I need to configure a autosys schedule which should run on every monday and 1st of every month. Is it possible to do in single JIL file.
I have used days_of_week: mo and what is the config used for date.
Is it possible to write a OR condition to trigger the job on either 1st or monday.
I would say just create a calendar with those requirements, you can also do that as extended calendar so you don't have to update the regular calendar ever.
Choice is yours :)
Related
I want to run a DAG on 2nd and 5th working day of every month.
eg1: Suppose, 1st Day of the month falls on Friday. In that case the 2nd working day of the month falls on 4th of that Month(i.e. on Monday) and 5th working Day falls on 7th of that month.
eg2: Suppose 1st Day of Month falls on Wednesday. In that case, the 2nd working Day will fall on 2nd Day of that Month, but the 5th Working Day will fall on 7th of that month (i.e. on Tuesday)
eg3: suppose 1st Day of Month falls on Sunday. In that case, the 2nd working day will fall on 3rd of that month and the 5th working day will fall on 6th of that month (i.e. on Friday)
So, how to schedule the DAG in Airflow for such scenarios.
#aiflow #DAG #schedule
I am looking for scheduling logic or code
Could you provide a code of your DAG?
It depends what operators are you using/willing to use.
Also, you might keep in mind bank holidays. Be sure that it is okay to run your airflow job on the 2nd day even if it is bank holiday?
You can schedule your dag daily and use python operator to validate if current date fits your restirctions. I would push this value to XCOM and then read and process this value in your DAG definition.
Another option would be to use bash operator in the beggining of flow and fail it in case current date violates your logic. This will not execute the rest of depented tasks.
Airflow uses cron definition of scheduling times so you must define your logic inside the code as cron can only run tasks on defined schedule and cannot do any calculations.
You can use custom timetables: https://airflow.apache.org/docs/apache-airflow/stable/howto/timetable.html
For a similar use case we implemented a branching operator with a logic to run the selected tasks when it is a specific workday of the month (I think we were using the workday package to identify specific holidays) and then this dag run daily. But the DAG had to complete some other tasks in all cases.
I have a job which runs everyday 4 times in an hour (total 48 times).
I want to prevent it from running only between 7 AM to 9 AM Sunday. How can I achieve it without creating two different jobs. I want to accomplish it in a single job. Please help.
Mention all the time the job should run in start_times like below
*replace .. with missing times
start_times: "00:00, 00:15, .... 06:30, 06:45, 9:15, 9:30, .. 23:45"
I suggest to create a calender defined as Every Sunday 07:00 till 09:00.
And then in the Job defination assign this calender to job attribute exclude_calendar along with your run_calendar.
so I have defined a periodic calendar with following periods,
Period A: to factor in start of every month
Period B: to factor in end of every month
Calendar
Now I am trying to schedule the job on every days marked as period A i.e. start of every month by doing the following settings,
Scheduling Setting
Schedule
My Problem
From July month the schedule is getting pre-shifted by 1. Any ideas why?
Also can someone guide me to a detailed documentation on periodic calendars and the values example?
Thanks.
I was not able to reproduce the specific issue you are having. I created the same periodic calendar (Per A = SoM, Per B = EoM) and my job schedule came out as expected.
Calendar
Job
Schedule
Solution
There is no need to use a periodic calendar for such a simple scheduling requirement. Making use of a periodic calendar incurs maintenance overhead as you need to ensure it is updated every year.
Instead of using a periodic calendar you could simply configure the job as below. This will result in a job that will always execute on the first day of each month, no maintenance required.
I have Job A that need by schedule to run on the 4th working day and 6th working day .
I created two extended calendars for 4th and 6th working day using the keywords WORKD#4 and WORKD#6 in the Date Conditions.
however I am not able to attached to extended calendars to one job .
could you please help me in this scenario how to run a job every month on the 4th and 6th working
Create only one extended calendar with Date Conditions "WORKD#4 or WORKD#6"
I need to put a scheduler in an app that will make a weekly schedule without dates. For example weekly dinner schedule. Or chores schedule, etc. Really just need day of week and time mapped.
Fullcalendar's Scheduler is the perfect format, except it deals with specific dates.
Is there a way I can use it for my use case? Scheduling weekly events without them being tied to specific dates?