I am new to Autosys and facing difficulty setting up some jobs. I have a box job containg a few command jobs. One of those command jobs may or may not run. The problem is when this job doesn't run(it remains in activated state), it keeps the box running. I have to terminate this job or the box every time such situation arises.
Is there a way to handle this?
Thanks
The answer depends on how your jobs are supposed to work, so posting JIL or a more detailed description would help, but:
You can add a "box_success" JIL attribute on your box job to define conditions that will cause the box to complete even without all the jobs running or completing.
You could consider moving the optional job outside of the box so that this issue goes away. But consider that the box could then complete while the optional job is running so make sure that you don't then have jobs running out of sequence or overlapping when they shouldn't.
Related
I'm having a very weird airflow bug.
Problem
I have a dag that has a bash operator as step 1 and a KubernetesPodOperator as step 2. The issue is regarding the KubernetesPodOperator. Basically, I was giving the task image X for quite some time, I recently changed the image the task is receiving to Y.
The issue I'm having is within TaskInstanceDetails the image is correct: Y. However, in the Rendered Template, the image starts correct X, however, as soon as the task starts running, it changes the image to Y.
I know this is very vague, I can't provide a whole lot more, but I'm just more looking for possibilities of things that could be happening, as I'm out of ideas.
What I've Tried
Delete Serialized Dags from DB
Delete Rendered Task Details from DB
Airflow db reset
Airflow db init (After nuking the whole thing)
Deleting the EC2 nodes and trying with new ones
EDIT
So, I tried runnning airflow tasks render dag_id task_id execution_date and the result here is image X !! Image Y is only superimposed on dag runs.
Answering my own question here in case anyone runs into this issue. Very simple fix... I carelessly had a different image name for the workers to be run with on the kubernetes_pod_template file. Changing that solved the issue.
In transaction DB05 you can check the selectivity of different fields of a table. For huge tables you should start it in the background, to avoid timeouts.
If I do this however, how can I find the result two hours later, when the job finished?
If this question belongs to a sister site, please move it there.
Background jobs: SM37
If you display the finished job, select it and click "Spool" (Pushbutton, above, about in the middle). Now you get to the spool, select it again and click the "Glasses" Icon. Yous should see the same list, like you would have run the report online.
I have several jobs than will run in sequence. It is possible to create a dependency between them only for completion, but not that the prior job has to complete successfully?
If a job fails this should remain red and go to the next job and continue running.
It is mandatory that this jobs to run in sequence and not in paralel.
As Mark outlined you can simply create an On-Do action within the parent job to add a condition when the job ends Not OK. The parent job will still go red and the successor job will kick off.
See below for an example:
yes, on the actions tab you create and On/Do step and say when Not OK the job should add the output condition. In this way the next job will run (in sequence) regardless of what happens to the predecessor job.
I have a box job that is dependent on another job finishing. The first job normally finishes by 11pm and my box job then kicks off and finishes in about 15 minutes. Occasionally, however, the job may not finish until much later. If it finishes later than 4am, I'd like to have it send an alert.
My admin told me that since it is dependent on a prior job, and not set to start at a specific time, it is not possible to set a time-based alert. Is this true? Does anybody have a workaround they can suggest? I'd rather not set the alert on the prior job (suggested by my admin) as that may not always catch those instances when my job runs longer.
Thanks!
You can set a max run alarm time which will alert if that time is exceeded
We ended up adding a job to the box with a start time of 4am that checks for the existence of the files the rest of the job creates. We also did this for the jobs predecessors to make sure we are notified if we are at risk of not finishing by 4am.
I submitting jobs to the batch process one after the other.
How do i control such that the second batch job runs only when the first one is finished.
Right now both the jobs executes simultaneously which i dont want to happen
There are two options. You can do this through code, or just via manual setup. Manual method is fairly easy, just go to (Basic>Inquiries>Batch Job), create a new batch job and save it. Then click "View Tasks" and create a new task, where this will be your first batch task. Choose your class, description, batch group, etc., then save. Click "parameters" to setup the parameters.
After that, you can setup your dependent task. Make sure your tasks both have descriptions. Add your second batch task and save. Then in the lower left corner, you click on your task that you want to have a condition, then add a row there and setup your conditions so that one task won't go until the second has completed.
Via X++ code, you would create a BatchHeader where you setup basically the same thing we just did manually. You use the .addDependency to make one task dependent on the completion of the other. This walkthrough will get you started with a job to create the batch header, and you'll just have to play around to get the dependency working.