I need to somehow get the start_time for the current job run dynamically from Autosys. It would be passed as an argument into our Python script. Current time isn’t good enough, it has to be the time for which the job is currently triggered which was supplied in the Jil. I haven’t found anything other than $AUTO_JOB_NAME that would get any jil attributes.
command: python3 /path/path/script.py —-job_id ${AUTO_JOB_NAME} —-conf file.conf
I want to add something like:
command: python3 /path/path/script.py —-job_id ${AUTO_JOB_NAME} —-conf file.conf —-time ${CURRENT_START_TIME}
And no current Autosys time is not what I’m looking for.
I already looked through documentation and it seems the only values defined in the runtime environment for a job are $AUTORUN and $AUTO_JOB_NAME
Why don't we use current time in the remote agent server where the Python script is hosted. It would be the same as tge job start time yoy are looking for. May differ my milliseconds, but you may experiment on that
Related
Is it possible on Firebase or Parse to set up something kinda like a cron job?
Is there a way to set up some sort of timed operation that runs over the stored user data?
For example, I'm writing a program that allows people to RSVP for lunch everyday. If you have RSVPed by noon, then you get paired up with somebody else who has also RSVPed. Using JavaScript, the user can submit their RSVP in the browser.
The question is, can Firebase/Parse execute the code to match everyone at 12:00pm every day?
Yes, this can be done with Parse. You'll need to write your matching function as a background job in cloud code, and then you'll need to schedule the task in the dashboard. In terms of the flexibility in scheduling, it's not as flexible as cron but you can definitely run a task at the same time every day, or every x minutes/hours.
Tasks can take 15 mins max to execute before they're killed, so depending on the size of your database or the complexity of your task, you may need to break it up into different tasks or make it resumable.
Just to confirm about Firebase:
As #rickerbh said, it can be done with Parse, but currently there is no way for you to run your code on Firebase's server. There are 2 options for you 2 solve this:
You could use Firebase Queue and run your code in Node.js
You could use a different library such as Microsoft Azure (I still haven't tried this yet, I'm not sure if it provides Job Scheduling for Android)
However, Firebase is working on something called Firebase Trigger, which will solve our problem, however it is still not released with no confirmed release date.
When i changed the start time of a coordinator job in job.properties in oozie, the job is not taking the changed time, instead its running in the old scheduled time.
Old job.properties:
startMinute=08
startTime=${startDate}T${startHour}:${startMinute}Z
New job.properties:
startMinute=07
startTime=${startDate}T${startHour}:${startMinute}Z
The job is not running at the changed time:07th minute,its running at 08th minute in every hour.
Please can you let me know the solution, how i can make the job pickup the updated properties(changed timing) without restarting or killing the job.
You can't really change the timing of the co-ordinator via any methods given by Oozie(v3.3.2) . When you submit a job the contents properties are stored in the database whereas the actual workflow is in the HDFS.
Everytime you execute the co-ordinator it is necessary to have the workflow in the path specified in properties during job submission but the properties file is not needed. What I mean to imply is the properties file does not come into the picture after submitting the job.
One hack is to update the time directly in the database using SQL query.But I am not sure about the implications of it.The property might become inconsistent across the database.
You have to kill the job and resubmit a new one.
Note: oozie provides a way to change the concurrency,endtime and pausetime as specified in the official docs.
I want to make a report of start and end times of a Autosys job from last three months.
How can i get it. Do i need to check archived history or logs?
If yes, please lemme know the details.
TIA
Autosys internally uses Oracle or Sybase database. As long as the data is available in the DB you can fetch it using autorep command. To get past run time use -r handle.
For example: autorep -J JobA -r -30
The above will give you last 30th run time for the job.
However, due to performance bottleneck that may arise due to historical data in the DBs the DBAs generally purge the data after a while. I have seen period of 1 day to 7 days based on the number of the jobs and database instance power.
Other approximate way would to be use the log files created by autosys if the option stdout is specified with unique filenames.
For example: you can have the attribute as std_out: $JOB_NAME.out.date +%m.%s
In this case the log file will be created as soon as the job starts which you can get from the filename using text function on unix,etc.
For the end-time, you can use the last modified time - this is where the approximate part comes in as the time would depend if your job had an echo to the log file or not. It can either be close or far based on the command of the script.
This method will not let you know the times for the box jobs as they never have a log attribute, for that you can depend on the first job in the box.
Autosys job retail_daily_job runs at 8:00 GMT. It is dependent on success of runner_daily_job.
Condition is If runner_daily_job is not success by 7:30 GMT, then status of retail_daily_job should be made to fail.i.e retail_daily_job should fail.
How to do this in autosys? what is the command to be used in jil file?
Thanks and Regards,
Simi
Not easy to do. Autosys doesn't support a negation condition for example NOT SUCCESS. I would try creating a job that would run at 0730 that would change the status of the retail_daily_job job to FAILURE if the runner_daily_job is FAILURE or TERMINATED or RUNNING.
Agree with clmccomas, not easy due to lack of certain operators.
We get around this by having our jobs create "status" files that other jobs can reference if need be. I've found that trying to parse/depend on output from autorep can be troublesome but is also doable. So we usually have a job status directory and create dated folders
/yyyymmdd/jobname_
The your command job would check the existing status file that's there. Often you may only want to write the status on completion.
The suggestion of having a job to alter statuses might work out well too.
Itseems like you are making the situation a bit complex.
If you want retail_daily_job to run based on the success of runner_daily_job, then simply put the condition as success(runner_daily_job).
So as you said if runner_daily_job is not success by 7:30 GMT (which means failure), then automatically the status of retail_daily_job will become to fail.
Hope this clarifies your question.
we have scheduled one script to run for every 5 mins.
how we can check that whether the script is running for every 5 minutes in linux?
If anyone knows, please Reply..
Thanks in Advance.
We have created the script. But we dont run that script. Our Support team will run this for every minutes. If they found any error, then they will update to us.
How we can cofirm that they r running this script properly or not?
If you don't want to modify your script and you've scheduled it in cron, you could change your cron line to:
*/5 * * * * /home/me/myscript.sh; date >> /tmp/mylog
And check /tmp/mylog - a new line should be added with the date and time every run.
Maybe by making the script log in a logfile with the timestamp, so you can verify if the timestamp is OK, something like:
date >>/tmp/myprocess.log
at the top of the script (or in the loop if that's how you're "running" it) then you can examine the log file to check.
Maybe you could just add a log with a timestamp in your script ?
Then you could see if the script was effectively run every 5 min.
Without changing the script?
atop is a process monitor package that can record history of programs being run, and as root will catch terminations as well. See http://www.atcomputing.nl/Tools/atop/whyatop.html
Also consider the process accounting tools http://www.faqs.org/docs/Linux-mini/Process-Accounting.html