Currenty I have requirement in my enviroment for the autosys email notifition.
Requirement: If the job runs more than the specified time it should trigger an email.
What I am trying is using max_run_alam, but I am not successful.
Lets say i have a job that runs for 10mins(lets say the time as 10.00). i set max_run_alarm as 3. i should get an email at 10.03 where i can goahead and see why the job is running more than the max_run_alarm. if i use max_run_alarm i am able to see in the logs triggering that alarm, but I cannot spend all day monitoring the logs to see which job is taking long as i have many jobs. my question is am i using max_run_alarm in the correct way or is there something else i am missing or is there entirely different way for the emails to generate.
pls advise.
We are using autosys R11 at work. I believe the triggering of emails is already automated in higher versions of autosys, but, in our version, to send automatic emails after a certain time, we create two extra autosys jobs. One autosys job starts at the same time as the job you want to "monitor". This job contains a 'sleep' command. (in your example, the command would be "sleep 180" for the job to run for 3 minutes until completion). The second extra job is the sending of the email and only starts after successful completion of the sleep-job.
To prevent the mail from being send every time the autosys box starts, you have to add your first job as BOX_SUCCESS condition. The sleep-job will run to completion, but the mail-job went from the "ACTIVATED" state to the "INACTIVE" state because the autosys box isn't RUNNING anymore.
Related
I am using JobRunr to run my background jobs and in this I am providing the users to setup recurring jobs using an endpoint like below:
#PostMapping("/schedule-recurring")
public String scheduleRecurring(#RequestBody ExecutionJob executionJob) {
return BackgroundJob.scheduleRecurrently(executionJob.getId(),executionJob.getCronExpression(), ()
-> jobService.executeSomeJob(executionJob, JobContext.Null));
}
These jobs could run in 5 mins, 10 mins or it might sometimes take upto 4 hours. This all depends on how many records to process. Right now, I am in a phase where I have only one background job server since I am building a POC for this. However, in future we plan to scale it to support 1 instance per customer having one or multiple background job server based on the client license.
My issue here is that, I have 2 recurring jobs which run 1 hour apart from each other. The first recurring job executes for more than 1 hour and deems the second job unexecuted, because this job is not triggered as there is no available Background Job Worker to address this request. I am thinking of adding a check to the job to trigger itself only if a Background Job Worker is available. But is there a better idea where-in the schedule-recurring method itself adds a condition to queue the job if a Background Job worker is not available?
Thanks in advance.
i have an autosys box which runs at 7AM and 8AM daily. I understand that if 7AM execution does not finish till 8AM, My box will not be triggered at 8AM again.
I need to send an email from autosys to business stating that previous job is already running that is why next execution is not happening at 8AM. How can i do that? Please try to give the JIL file for it.
Thanks in advance
It all depends how have you licensed Autosys CA schedulers for your project. In our firm, we have configuration setup wherein if an Autosys job fails/terminates/maxruns/etc., it creates high severity incident.
Coming to your query, there's no straight forward way that i am aware of. First, normally we do not schedule same job in frequent intervals, unless it is a reporting job.
Still, if you want the job to generate some sort of alert if first execution exceeds start time of second execution, then you can add max_run_alarm or term_run_time in your JIL as per requirement and set it for 60 min since that's the interval between two executions.
Here is what i have added my script JIL for a data loader job :
max_run_alarm: 150
alarm_if_fail: y
alarm_if_terminated: y
box_terminator: y
term_run_time: 780
Which parameter you want to use will depend on your requirement. If you want to kill the first job then use term_run_time parameter so that second execution "might" start or use max_run_alarm parameter if you just want to send a MAX ALARM notification.
I use both! Hope this was useful.
Lets say i have a job that runs for 10mins(lets say the time as 10.00). i set max_run_alarm as 3. i should get an email at 10.03 where i can goahead and see why the job is running more than the max_run_alarm. if i use max_run_alarm i am able to see in the logs triggering that alarm, but I cannot spend all day monitoring the logs to see which job is taking long as i have many jobs. my question is am i using max_run_alarm in the correct way or is there something else i am missing or is there entirely different way for the emails to generate.
There is an added functionality where you can configure the email id where all Autosys notification to be sent out.
Incase of any event such as Job Failure / Termination / Run Alarms etc are notified to that email address.
This is configured by the Autosys Admin or the Scheduling Team which i call them as.
I am trying to understand autosys job. Suppose I have Job A that runs every 15 minutes. Suppose for some reason if Job A takes more than 15 minutes, will another instance of it run or it will wait for the job to finish before running another instance?
In my experience, if the previous job run is still running, another instance will not run if the next scheduled time comes. The next time the job runs is when the previous run is finished and the next scheduled time comes.
Another user also experienced this according to this answer.
I did not find any AutoSys documentation that officially confirms what happens in this situation, but I guess the best way to find out is to test it on your AutoSys instance.
I have experienced this first hand and can confirm that there won't be two instances in the mentioned scenario. The job will wait on the previous run to complete and will immediately kick off the next instance if the time condition is met before the previous completes.
But this will be the case only when the job is in running state, if the job is in any other state it will kick off based on the given start_time condition.
When i changed the start time of a coordinator job in job.properties in oozie, the job is not taking the changed time, instead its running in the old scheduled time.
Old job.properties:
startMinute=08
startTime=${startDate}T${startHour}:${startMinute}Z
New job.properties:
startMinute=07
startTime=${startDate}T${startHour}:${startMinute}Z
The job is not running at the changed time:07th minute,its running at 08th minute in every hour.
Please can you let me know the solution, how i can make the job pickup the updated properties(changed timing) without restarting or killing the job.
You can't really change the timing of the co-ordinator via any methods given by Oozie(v3.3.2) . When you submit a job the contents properties are stored in the database whereas the actual workflow is in the HDFS.
Everytime you execute the co-ordinator it is necessary to have the workflow in the path specified in properties during job submission but the properties file is not needed. What I mean to imply is the properties file does not come into the picture after submitting the job.
One hack is to update the time directly in the database using SQL query.But I am not sure about the implications of it.The property might become inconsistent across the database.
You have to kill the job and resubmit a new one.
Note: oozie provides a way to change the concurrency,endtime and pausetime as specified in the official docs.