If keep active is given as 0, how long the job waits for the pre-requisites to be met?
If keep active is given as 1, how long the job waits for the pre-requisites to be met?
My requirement is job should come into plan on certain time on first day and if pre-requisites aren't met that should go away from plan by the same time following(next) day.
Control-M works around the fundamental concept of New Day Processing (NDP). NDP sets up your scheduled jobs for the upcoming day and does housekeeping on the jobs that were there during the previous "day". The timing of your NDP is set by the CM Admin. Say, for example, your, your NDP is at 8 am and you have a new job that waits until 1 pm, has an input condition and keep active = 0. Should the input condition not show up then your job will be cleared out (unexecuted) at 8 am the next day.
If you have keep active set as 1 (or more) the job will survive NDP and potentially run (upon receiving the input cond) the next day. However it will wait on the from time (1 pm in our example) unless you use the "allow submission past next New Day" option. You can also set a "to" time, which sets the latest possible time for when the job could submit - unfortunately that would also limit you on the "first" day. To cleanup on the second day (in the way you describe) I think you'd have to create another job to manipulate the original (via ctmpsm ot the api).
I just ran an airflow DAG. When I see the airflow last run date, it displays the last but last run date. It catches my attention when I hover over the "i" icon it shows the correct date. Is there any way to solve this? Sounds like nonsense but I end up using it for QA of my data.
This is probably because your airflow job has catchup=True enabled and a start_date in the past, so it is back-filling.
The Start Date is the real-time date of the last run, whereas the Last Run is the execution date of the airflow job. For example, if I am back-filling a time partitioned table with data from 2016-01-01 to present, the Start Date will be the current date but the Last Run date will be 2016-01-01.
Please include your DAG file/code in the future.
Edit: If you don't have catchUp=True enabled, and the discrepancy is approximately one day (like in the picture you sent), then that is just due to the behaviour of the scheduler. From the docs, "The scheduler runs your job one schedule_interval AFTER the start date, at the END of the period."
if you run a DAG on a schedule_interval of one day, the run stamped 2016-01-01 will be triggered soon after 2016-01-01T23:59. In other words, the job instance is started once the period it covers has ended.
I have a Control-M job that runs every 20 min. Everything works great during that day's run. The issue is when we auto-order the next days jobs. If the current day's jobs are running we get 2 copies of the jobs running at the same time.
Is there a way to not start the new job if the previous day's job is executing?
The job starts every 20 minutes, but how long does it run? Set the end window for "submit between" time a few minutes before the new day build.
Let's say your new day builds at 0400. Since the job is intended to run every 20 minutes, you can have it run as late as 0340. Set the "to" time in Activity Window to 0340 and the job won't autosubmit after that time. The new day will build at 0400 and the new version of the job will start then - 20 minutes after the previous start.
You can also add a control resource to the job to prevent two of them from running at the same time. I don't know another way to do it. That's not a can of worms I'd open unless the activity period settings just won't work the way you want them to.
If you're referring to last day's job execution bleeding into current day, causing resource contention, your best bet (as Rob pointed out) is to define a resource with max count of 1 to be required by the job, so next day job instance cannot start until prev day job completes and release the resource. Alternatively you can have the job post a Condition for order date + 1, and have order date condition also be an In condition for the job.
I have 2 jobs in autosys: Job 1 and Job 2. Job2 will only run if Job1 is a success.
Now, in a normal scenario, Job1 and Job2 will run in succession as part of a daily batch.
But, sometimes, I need to re run Job1 without kicking off Job2. But if I just re run Job1, its success will automatically kick off Job 2.
So, I put Job2 'On Hold' and run Job1. All good so far.
Now, it is my experience that when I put Job 2 'Off Hold', it will immediately start running. I don't want that.
http://autosys-tutorials.blogspot.ca/2011/04/autosys-quick-reference.html
What I want is for Job2 to go into a Runnable state so that it gets picked up in the next batch run.
What status should I set Job2 to?
I am not sure whether you got answer for your doubt, there is one more option to do not let the ON_HOLD job to run immediately when off_hold it.
change the job 2 status from ON_HOLD to INACTIVE, so job 2 will not start immediately and same time job 2 will start running once condition(job 1 went to success state) met when job is in INACTIVE state.
If you mark the job 2 to ONICE, it will allow the dependent job to start if it has.
Regards,
Kaliraja (HP Autosys Team)
In this situation you should put Job2 'On Ice' before re-running Job1, instead of putting it 'On Hold'. From the link you posted:
The difference between "on hold" and "on ice" is that when an "on
hold" job is taken off hold, if its starting conditions are already
satisfied, it will be scheduled to run, and it will run. On the other
hand, if an "on ice" job is taken "off ice," it will not start, even
if its starting conditions are already satisfied. This job will not
run until its starting conditions reoccur.
You can put job2 on hold and run job1. After the completion of JOb1, change the status of Job2 to success,so that it can start next time as per schedule and based on its conditions.
Put Job_2 as 'On_Ice' before starting Job_1
You can place job 2 ON HOLD, But mark the job as success/Failure (ensure Job2 has no other dependency on this state)
You can put job2 on hold and run job1. After the completion of JOb1, change the status of Job2 to INACTIVE so that it can start next time as per schedule.
You could use override_job subcommand to bypass the execution only one time. The next schedule will automatically pick up the job to run.
This example shows how to define a one-time job override. The following script runs the job RunData with no conditions (where some had been previously specified) and outputs the results to a different output file:
UNIX:
override_job: RunData
condition: NULL
std_out_file: "tmp\SpecialRun.out"
If you want to cancel a override job defined, then use the below -
override_job: RunData delete
About your concern that you don't want to put Job2 'ON_ICE' because this will run the remaining jobs (job3 - job10), I think this scenario will not happen unless your main box is running. Since you've mentioned that you are going to run the job1 as standalone, the main box should not be running since, if so, job1 will be automatically triggered. Having said this, make sure that the main box is not running when you want to run the job1 alone. Else, the thing that your are saying will happen.
Options for the scenario:
1- Place job2 on "No Execution" each time you want to run ONLY job1, later remove it.
2- Place look-back as success condition as s(job1,0) on job2 for permanent fix to meet your adhoc request & permanent scheduled run if you are placing it on-hold & later removing it.
3- Place it on-Ice, but make sure there is no dependent job for Job2.
Put these jobs in the same box, put job 2 ON_HOLD, FORCESTART the box, wait for job 1 to finish with SUCCESS, KILLJOB the box, take job 2 OFF_HOLD, change status of all jobs in box to SUCCESS.
I am looking for a help in converting a IBM Tivoli script to Autosys since we have migrated from Tivoli to AUtosys.
Below is the script in Tivoli -
SCHEDULE SX On RUNCYCLE YEARLY VALIDFROM 02/01/2015 "FREQ=YEARLY;INTERVAL=1;"
UNTIL 0550 +5 DAYS
CARRYFORWARD
...
...
I need to convert this same script so that it can work in Autosys. Not sure, what Tivoli script means and what this Tivoli is doing and how I can convert it in run_calendar in AUtosys script
What about UNTIL and carryforward option ?
Thanks.
This is a TWS Schedule (a set of jobs) that is set to run once a year starting 02/01/2015. It will start on that day at new plan generation and will run until 05:50 am after 5 days. CARRYFORWARD means that this schedule will not disappear at new plan generation that comes everyday.
This is not a script per se, this is a container of jobs. The Autosys calendar must look like a yearly job.