I want to schedule a job using autosys R11. I use start_time to specify the job start time and also use start_mins to specify the interval.
But now, I want to schedule a job from 1:00 PM to 5:00 PM at regular interval (10 mins) so that the job will run at 1:00, 1:10,1:20,1:30,....,2:00,2:10,2:30 etc. How do I specify the end time (5:00 PM)?
Could anyone help me with this?
Thanks,
Veena
Use a run_window: "13:00 - 17:00" and start_min: 0, 10, 20, 30, 40, 50
Related
We are working on airflow currently. If we have any daily dags which has to run as we follow current date minus 1
if the dag(daily dag 00 10 * * *) should have first run on 20 March 2022 then we provide start date as
'start_date': datetime(2022, 3, 19)
if we have to run on weekly basis or monthly basis, can any pls suggest how to provide start date ?
If the dag should run first on 19th march 2022(every saturday) cron
expression : 00 10 * * 6
what can i provide the start date in this case ? please suggest
is that something like 'start_date':
datetime(2022, 3, 12) or
'start_date': datetime(2022, 3, 11) ?????
is there any way i can get start date based on frequency with some python code ?
I have the following DAG set up:
default_dag_args = {
'start_date': datetime.datetime(2021, 6, 25, 0, 0),
'email': 'foobar#foobar.com',
'email_on_failure': True,
'email_on_retry': False,
'retries': 1,
'retry_delay': datetime.timedelta(minutes=30)
}
with models.DAG(
'foobar',
schedule_interval = "30 5,7,9 * * *",
default_args=default_dag_args,
catchup=False) as dag:
The behaviour that I want to have is that the DAG will execute at 5:30, 7:30 and 9:30 UTC every day. The behaviour that I'm seeing is that the 5:30 run executes at 7:30 UTC, the 7:30 run executes at 9:30 and the 9:30 run executes at 5:30 the next day.
I think I kind of have a vague idea of why this is happening based on the docs - 9:30 marks the end of the schedule period and so the 9:30 run executes at the beginning of the next period. I can't figure out how to get the behaviour I want though. The DAG doesn't have any reference to the schedule time in the code, it just needs to run at 5:30, 7:30 and 9:30 and the 'run time' as Airflow considers it doesn't matter.
Is there any way to get a DAG to run at absolute times? If not, what schedule can I set to get the behaviour I desire?
Airflow is not a cron job scheduler. Airflow calculates start_date + schedule_interval and execute the job at the end of the interval. The reason behind this is explained in this answer.
In your case:
start_date=datetime(2021,06,25) with schedule_interval = "30 5,7,9 * * *" gives:
1st tasks with execution_date 2021-06-25 5:30 will start running on 2021-06-25 7:30
2nd task with execution_date 2021-06-25 7:30 will start running on 2021-06-25 9:30
3rd task with execution_date 2021-06-25 9:30 will start running on 2021-06-26 5:30
4th task with execution_date 2021-06-26 5:30 will start running on 2021-06-26 7:30
5th task with execution_date 2021-06-26 7:30 will start running on 2021-06-26 9:30
6th task with execution_date 2021-06-26 9:30 will start running on 2021-06-27 5:30
7th task with execution_date 2021-06-27 5:30 will start running on 2021-06-27 7:30
8th task with execution_date 2021-06-27 7:30 will start running on 2021-06-27 9:30
9th task with execution_date 2021-06-27 9:30 will start running on 2021-06-28 5:30
and so on...
Note that you still get 3 runs per day (except the first date) as you expect it just a matter of understanding how scheduling works. If you want to get 3 runs on the first date as well then change your start_date to datetime(2021,06,24,9,30). The execution_date is a logical date. If needed you can reference relevant dates within your DAG code usings macros - for example:
I mentioned that the 6th run execution_date is 2021-06-26 9:30 using macros with that runs can give you:
prev_execution_date is 2021-06-26 7:30
next_execution_date is 2021-06-27 5:30
Note: your code has catchup=False so the exact dates I wrote here won't be the same but that effects only on the first run. The following runs will follow the same logic.
I run discrete event simulations which cover more than a year. All events need to be tracked on a second time scale.
Now, I want to schedule the events according to a timetable, calculate overtime, ... and compare it to the real labor time in the end.
I thought whether there is an elegant way to implement schedules in R such that the following tasks/questions can be answered within one line of code:
Schedule work according to a timetable.
Given a timetable, how many labor hours do we expect per day/week/month/... Calculate the overtime per day/week/month/...
Create simple plots to visualize working time, overtime, ... and compare it to the planned work load.
On an abstract level, assume all of the timetables are given more or less in the following way:
Name Days_of_year Available Time
Name1 Jan-07 – Oct-18 Mon, Tues, Wed, Thurs, Fri, Sat, Sun 8:45 - 18:00
Name2 Jan-01 – Dec-31 Mon, Tues, Wed, Thurs, Fri 20:00 - 7:00
I am not looking for obvious answers like "create an arry and somehow you get everything out of that" but an implementation that:
Makes schedules independent of time resolution (days, hours, minutes, seconds).
Schedules/timetables are not stored with redundant information.
Any result (e.g. total year labor, ...) should be easily transferred to any given unit.
Easy filter possibilities by month, wday, ...
Ideally, calendar functionalities (holidays, ...) can be easily used.
Perhaps as part of lubridate?
Use case view ("schedule functions" indicated by s_):
Time: s_hm(„7:45“), s_md(„Jan-23“) or s_md(„01-23“). Please note that those "times" do not make sence as dates or date-times, but in the framework of schedules they perfectly make sence!
Interval: s_interval. See first thoughts below.
Duration: s_interval/dhours(1) => Duration in hours
Functions:
Max(), Min() on any s_interval
%s_in%
Given a date, when does somebody work:
date(s) %s_in% s_interval
=> return TRUE/FALSE (array)
Given a schedule, when is somebody available in an interval:
s_interval %s_in% interval
=> return class interval - a subset of interval!
Given an interval, was the labor according to schedule? Calculate the idle-, over- and working time:
interval %s_in% s_interval
=> return class s_interval
Given two schedules, consider them as one unit. How is idle-, over- and working time distributed:
(s_interval1 %+% s_interval2)
=> return class s_interval
Some thoughts:
schedule in case of planning labor time:
s_interval $tzone UTC
$daysofyear 286 days (Better alternative: format month-day e.g. Jan-07 – Oct-18)
$wday Mon, Tues, ... Sun (According to the start time of a shift/lecture...)
$time 12 h (Better alternative: format time span* e.g. 8:00 – 18:00. NOT period which results in hours)
$overtime 0 h (A priori, a schedule does not plan overtime)
$idle 0 h (A priori, a schedule only plans labor time)
schedule when real time-data is available:
s_interval$tzone UTC
$daysofyear 286 days (Better alternative: format month-day e.g. Jan-07 – Oct-18)
$wday c(Mon, Tues, Wed, ...) (According to the start time of a shift/lecture...)
$time c(12 h, 1 h, 0.5 h, ...) (Better alternative: format time span* e.g. 8:00 – 18:00. NOT period which results in hours)
$overtime c(2 h, 0 h, -1 h, ...) total time = time + overtime. Array of lubridate intervals*?
$idle c(4 h, 8 h, 0h, ...) pure working_time = time – idle + overtime. Array of lubridate intervals*?
* Use intervals not periods such that interval/dhours(1) possible.
If the tzone is the same we could calculate e.g.
s_interval1 %+% s_interval2 =
s_interval$tzone UTC
$daysofyear 286 days, 123 days
$wday c(Mon, Tues, Wed, ...), c(Sat, Sun)
$time c(12 h, 1 h, 0.5 h, ...), c(-1 h, 2.5 h)
$overtime c(2 h, 0 h, -1 h, ...), c(0 h, 5 h)
$idle c(4 h, 8 h, 0h, ...), c(4 h, 8 h)
There are related posts about this topic visualization of schedules with some interesting answers concerning timeline and gantt packages and how to filter dates.
However, they are not very conclusive.
As I am quite new to R, I don't know, how to start such a task the best way and understanding the structure of a package like lubridate is quite advanced...
I developed a package which gives the desired functionality (even with plots). At the moment, the package is private, but let me know if you are interested.
I am interacting with a Remote Server. This Remote Server is in a different Time Zone. Part of the Authentication requires me to produce the:
"The number of seconds since January 1, 1970 00:00:00 GMT
The server will only accept requests where the timestamp
is within 600s of the current time"
The documentation of erlang:now(). reveals that it can get me the the elapsed time since 00:00 GMT, January 1, 1970 (zero hour)
on the assumption that the underlying OS supports this. It returns a size=3 tuple, {MegaSecs, Secs, MicroSecs}. I tried using element(2,erlang:now()) but the remote server sends me this message:
Timestamp expired: Given timestamp (1970-01-07T14:44:42Z)
not within 600s of server time (2012-01-26T09:51:26Z)
Which of these 3 parameters is the required number of seconds since Jan 1, 1970 ? What aren't i doing right ? Is there something i have to do with the universal time as in calendar:universal_time() ? UPDATEAs an update, i managed to switch off the time-expired problem by using this:
seconds_1970()->
T1 = {{1970,1,1},{0,0,0}},
T2 = calendar:universal_time(),
{Days,{HH,Mins,Secs}} = calendar:time_difference(T1,T2),
(Days * 24 * 60 * 60) + (HH * 60 * 60) + (Mins * 60) + Secs.
However, the question still remains. There must be a way, a fundamental Erlang way of getting this, probably a BIF, right ?
You have to calculate the UNIX time (seconds since 1970) from the results of now(), like this:
{MegaSecs, Secs, MicroSecs} = now().
UnixTime = MegaSecs * 1000000 + Secs.
Just using the second entry of the tuple will tell you the time in seconds since the last decimal trillionellium (in seconds since the UNIX epoch).
[2017 Edit]
now is deprecated, but erlang:timestamp() is not and returns the same format as now did.
Which of these 3 parameters is the required number of seconds since Jan 1, 1970 ?
All three of them, collectively. Look at the given timestamp. It's January 7, 1970. Presumably Secs will be between 0 (inclusive) and 1,000,000 (exclusive). One million seconds is only 11.574 days. You need to use the megaseconds as well as the seconds. Since the error tolerance is 600 seconds you can ignore the microseconds part of the response from erlang:now().
How can I find the time difference in seconds between two dates in prolog?
datetime(2001,03,04,23,00,32).
datetime(2001,03,04,23,01,33).
timediff(D1,D2,Sec).
Sec=61
SWI-Prolog offers several predicates that convert human-readable time representations into seconds from Epoch (at 1970-01-01). Having the time represented as a number of seconds turns the difference calculation into a simple subtraction operation. For example:
timediff(DateTime1, DateTime2, Sec) :-
date_time_stamp(DateTime1, TimeStamp1),
date_time_stamp(DateTime2, TimeStamp2),
Sec is TimeStamp2 - TimeStamp1.
Usage:
?- timediff(date(2001, 03, 04, 23, 0, 32, 0, -, -),
date(2001, 03, 04, 23, 1, 33, 0, -, -), Sec).
Sec = 61.0.
This gets a little awkward what with the months not being the same length and leap years having extra days.
To start you off, I'm going to give you a version of a predicate that will only take into account hours, minutes, and seconds:
timediff(time(Hours1, Minutes1, Seconds1), time(Hours2, Minutes2, Seconds2), Seconds) :-
Seconds is Seconds1-Seconds2 + 60*(Minutes1-Minutes2 + 60*(Hours1-Hours2)).
If you could run convert_time/2 or convert_time/8 backwards, this process would be much easier, as it would allow you to use the operating system's time conversion routines instead of writing your own.