I am looking to run a console application triggered from Autosys every X minutes.
The following commands do not seem to provide this capability
start_times: Exact time each day a job will run [cannot be
used with start_mins]
start_mins: Minutes after each hour a job will execute
[cannot be used with start_times]
The solution that I can see at the moment is to set start_mins : 0,5,10,15,20,25,30,35,40,45,50,55
This is ok if the time interval is 5 minutes, but becomes a little cumbersome if the interval is 1 or 2 minutes.
Is there any way to configure Autosys to easily repeat a job every x minutes ?
there is only one way for autosys to start a job every minute -
start_mins: 0,1,2..59
Related
For example, I created a new DAG at 11:30. The DAG is supposed to run on the first minute of every hour (Cron: 0 * * * *). I expected the DAG to start at 12:00. However the DAG first started at 13:00. The Second started at 14:00 while Next run in UI is 13:00
I know that, it's mentioned in docs
**Let’s Repeat That**, the scheduler runs your job one schedule AFTER the start date, at the END of the interval.
But I don't know What's the use of doing that? Why don't run at the start of the day like cron job?
Does anyone have any documentation explaining this issue?
thanks
I am quite confused about time concept in airflow. As far as I know, there are at least 5 different time concept in airflow:
UTC time in web UI
Execution time, e.g. start_date, in DAG file
RUN time.
time saved in database
system time
Could anyone explain which are the same and which are different?
The UTC time in the Web UI(1) is the system time (5) being presented.
The run time (3) shows when a task was actually run in UTC and this is the time saved in the database (4) as it corresponds to when a certain task was actually ran in a DAG.
The start_date should be a static non-changing date as it corresponds to when the DAG was first started and should be backfilled if certain parameters like catchup or depends_on_past is set to true.
So 5 and 1 are the same, 3 and 4 are the same, and 5 is it's own entity of time.
I have a chain with several jobs, and sometimes a certain job that normally takes about 2 hours, finishes in less than 2 minutes.
What I would like to do is to kill this job if it ends in less than 2 minutes so that the chain won't proceed.
Is this possible?
Thanks
Well you don't really want to kill anything, do you? If you do, see BMC's note (including video) on using ctmkilljob.
In this case your next job is dependent on 2 things, the predecessor job being ok and the duration of the predecessor job. Add another input condition to your next job (in addition to the existing condition) to represent the >2 minutes duration.
On the job that needs to run for more than 2 minutes, add a Notify when the Exectime exceeds 2 mins (or 60 mins or whatever you decide is the threshold) and get it to shout to an entry in your shout destination table.
On the Control-M Server create a new program entry in your shout destination table and create a small script referenced by the shout. The script should use the ctmcontb utility to create a new ODAT condition that your next is waiting on. If you have a look at the BMC help note for ctmkilljob (and just substitute in ctmcontb) then you'll see how to do this.
Job B
Condition s(Job A, 9.00)
Start time: 9:15
Job A has run within 9 hours. Sometimes the Job A starts running again just before the scheduled start time of Job B (9:15). Because of this, Job B cannot start as Job A is RU instead of SU.
What kind of condition on Job B would solve this issue?
Is your jobB should be really dependent with the success of jobA? if yes, then all you need to do is to wait for the jobA to be completed, and jobB will automatically start after that. If no, you can simply remove the jobA in the run condition of jobB and remain the start time of jobB as 9:15. Also, are your jobs reside in the same box? Is your jobA has multiple run time in a day?
I have an Hive script that I am executing using Oozie coordinator every 10 minutes. When I launched my Oozie coordinator-app, suppose I have started at 08:03, the first workflow starts at that time, the next 08:13, and then 08:23, and so on.
What I want is to execute the workflow every clock time hh:mm, where mm is divisible by 10. Assuming the same scenario above, what I want to happen is this: the first workflow will execute at 08:10, and then 08:20, and so on.
How do I do this in Oozie? How about when every 5 minutes (The last m of the minute is either 5 or 0)? Thanks for your input.
In order to run a coordinator job at a frequency, you can use the following directives
<coordinator-app name="app" frequency="10" start="2015-07-10T12:00Z" end="2016-01-01T00:00Z" timezone="UTC" xmlns="uri:oozie:coordinator:0.1">
This would run every 10 minutes, starting exactly at 12:00 UTC time today. The same goes for running every 5 minutes, just replace frequency="10" with frequency="5". To to have it run every Nth minute divisible by M, you will have to ensure that your start parameter is set correctly.
Another option if you are using a more recent version of Oozie (4.1.0) would be to use the cron like scheduler. This would allow you to schedule Oozie coordinators in a cron-like fashion if you're familiar. See http://blog.cloudera.com/blog/2014/04/how-to-use-cron-like-scheduling-in-apache-oozie/ and https://issues.apache.org/jira/browse/OOZIE-1306