i have an autosys box which runs at 7AM and 8AM daily. I understand that if 7AM execution does not finish till 8AM, My box will not be triggered at 8AM again.
I need to send an email from autosys to business stating that previous job is already running that is why next execution is not happening at 8AM. How can i do that? Please try to give the JIL file for it.
Thanks in advance
It all depends how have you licensed Autosys CA schedulers for your project. In our firm, we have configuration setup wherein if an Autosys job fails/terminates/maxruns/etc., it creates high severity incident.
Coming to your query, there's no straight forward way that i am aware of. First, normally we do not schedule same job in frequent intervals, unless it is a reporting job.
Still, if you want the job to generate some sort of alert if first execution exceeds start time of second execution, then you can add max_run_alarm or term_run_time in your JIL as per requirement and set it for 60 min since that's the interval between two executions.
Here is what i have added my script JIL for a data loader job :
max_run_alarm: 150
alarm_if_fail: y
alarm_if_terminated: y
box_terminator: y
term_run_time: 780
Which parameter you want to use will depend on your requirement. If you want to kill the first job then use term_run_time parameter so that second execution "might" start or use max_run_alarm parameter if you just want to send a MAX ALARM notification.
I use both! Hope this was useful.
Related
I am trying to understand autosys job. Suppose I have Job A that runs every 15 minutes. Suppose for some reason if Job A takes more than 15 minutes, will another instance of it run or it will wait for the job to finish before running another instance?
In my experience, if the previous job run is still running, another instance will not run if the next scheduled time comes. The next time the job runs is when the previous run is finished and the next scheduled time comes.
Another user also experienced this according to this answer.
I did not find any AutoSys documentation that officially confirms what happens in this situation, but I guess the best way to find out is to test it on your AutoSys instance.
I have experienced this first hand and can confirm that there won't be two instances in the mentioned scenario. The job will wait on the previous run to complete and will immediately kick off the next instance if the time condition is met before the previous completes.
But this will be the case only when the job is in running state, if the job is in any other state it will kick off based on the given start_time condition.
I am scheduling an Oozie MapReduce job to run every 15 minutes. I wonder what would happen if each job will take longer than that set time? Will it result in a job backlog? Or Oozie will create a new task / thread / fork for the new job while the previous one is still running?
Oozie won't run the next job before the previous one is over. If the first job takes more than 15 minutes to execute then the next one will be run after scheduled time. So scheduled time and running time may be different in Oozie.
EDIT:
Anyway, the described behaviour is default only and can be changed. You can set concurrency property from controls block to more than 1, and the next job will be run even the first one is still running. Check my answer on similar question
Currenty I have requirement in my enviroment for the autosys email notifition.
Requirement: If the job runs more than the specified time it should trigger an email.
What I am trying is using max_run_alam, but I am not successful.
Lets say i have a job that runs for 10mins(lets say the time as 10.00). i set max_run_alarm as 3. i should get an email at 10.03 where i can goahead and see why the job is running more than the max_run_alarm. if i use max_run_alarm i am able to see in the logs triggering that alarm, but I cannot spend all day monitoring the logs to see which job is taking long as i have many jobs. my question is am i using max_run_alarm in the correct way or is there something else i am missing or is there entirely different way for the emails to generate.
pls advise.
We are using autosys R11 at work. I believe the triggering of emails is already automated in higher versions of autosys, but, in our version, to send automatic emails after a certain time, we create two extra autosys jobs. One autosys job starts at the same time as the job you want to "monitor". This job contains a 'sleep' command. (in your example, the command would be "sleep 180" for the job to run for 3 minutes until completion). The second extra job is the sending of the email and only starts after successful completion of the sleep-job.
To prevent the mail from being send every time the autosys box starts, you have to add your first job as BOX_SUCCESS condition. The sleep-job will run to completion, but the mail-job went from the "ACTIVATED" state to the "INACTIVE" state because the autosys box isn't RUNNING anymore.
the benefits of coordinating by an absolute time is that (insofar as the jobs take a consistent amount of time) the output will be ready for others at some time (e.g. update a dashboard during the night for people to see in the morning).
the benefits of coordinating by a relative frequency is that, if oozie (or it's server) are down, no jobs are skipped (e.g. a daily job might run 2 hours late, but not 22 hours late).
how can i do something like:
start="2009-01-01T21:00Z"
frequency="${coord:days(1)}"
run-if-skipped="true"
i.e. when all's well, jobs run daily at 9pm. if something happens to oozie (e.g. the server is rolled) between 8pm and 10pm, once oozie comes back up at 10pm, the job should run at 10pm, and then tomorrow at 9pm as normal.
https://github.com/yahoo/oozie/wiki/Oozie-Coord-Use-Cases
Not sure that I fully understand the question.
If server is down, and you re-start you coordinator, so it will start from the coordinator start time.
Also you can make you job run every hour, check if the output folder exist - that stop . for that use Decision Control Node
I want to make a report of start and end times of a Autosys job from last three months.
How can i get it. Do i need to check archived history or logs?
If yes, please lemme know the details.
TIA
Autosys internally uses Oracle or Sybase database. As long as the data is available in the DB you can fetch it using autorep command. To get past run time use -r handle.
For example: autorep -J JobA -r -30
The above will give you last 30th run time for the job.
However, due to performance bottleneck that may arise due to historical data in the DBs the DBAs generally purge the data after a while. I have seen period of 1 day to 7 days based on the number of the jobs and database instance power.
Other approximate way would to be use the log files created by autosys if the option stdout is specified with unique filenames.
For example: you can have the attribute as std_out: $JOB_NAME.out.date +%m.%s
In this case the log file will be created as soon as the job starts which you can get from the filename using text function on unix,etc.
For the end-time, you can use the last modified time - this is where the approximate part comes in as the time would depend if your job had an echo to the log file or not. It can either be close or far based on the command of the script.
This method will not let you know the times for the box jobs as they never have a log attribute, for that you can depend on the first job in the box.