I am invoking a task with cron expression got from UI in the controller.
taskScheduler.schedule(scheduledTask, new CronTrigger(cronExpression));
And the task is created with the given cron expression. But when I again hit the controller with different cron expression, it is creating a new task without stopping the existing task.
I wanted either to stop the old task and start the task with new cron expression or reschedule the existing task with new cron expression
Related
I have a running dag, Can I stop it using REST API ?
There is a method to delete a running dag:
https://airflow.apache.org/docs/apache-airflow/stable/stable-rest-api-ref.html#operation/delete_dag_run。
But There is not REST API how to stop a running dag?
Thanks!
You can set the running tasks to failed with Set a state of task instances note that this will stop the task from running and set the task status to failed but it will not invoke the on_failure_callable if you set one on your tasks nor it will invoke the on_kill method of the operator (if there is one).
If you want the DAG to not schedule any more tasks you can set it to Pause with Update a DAG
I am having a task which will listen for certain events and kick-off other functions.
This function (the listener) subscribes to a kafka topic and runs forever, or at least until it will get a 'stop' event.
Wrapping this as an airflow operator doesn't seem to work properly.
Meaning, if I send the stop event, it will not process it, or anything else for that matter.
Is it possible to run busy loop functions in airflow ?
No, do not run infinite loops in an Airflow task.
Airflow is designed as a batch processor - long/infinite running tasks are counter to it's entire scheduling and processing model and while it might "work", it will lock up a task runner slot.
I wrote my job using jsr-352 and deployed it on wildfly. How can I schedule one job with some delay after last end time like below time line where = is execution time and - is delay time:
===============--=====--========--
Note: maximum number of job execution is one
JBeret ejb scheduler supports repeating interval job executions, with a fixed interval duration or certain delay duration after the start of job execution. Delay after end of a job execution is currently not supported. If your job execution duration is relatively predictable, you can approximate it with either interval or delay after the start of a job execution.
To achieve this kind of job schedulgin with finer control, you can try the following:
schedule a single-action job schedule
configure a job listener in job.xml to watch for the end of the above job execution, and scheule the next single-action job execution with a short initial delay
specifically, the job listener's afterJob() method should be able to look up or inject TimerSchedulerBean, which is a local singleton EJB, and invoke its org.jberet.schedule.TimerSchedulerBean#schedule method. The job listener is reponsible for creating an instance of org.jberet.schedule.JobScheduleConfig, passing it when calling the ejb business method. The job listener should already have all info to create JobScheduleConfig.
I read from Oozie official site: Actions Are Asynchronous
All computation/processing tasks triggered by an action node are executed asynchronously by Oozie. For most types of computation/processing tasks triggered by workflow action, the workflow job has to wait until the computation/processing task completes before transitioning to the following node in the workflow.
Whereas on different page of the same site: Fs HDFS action
The introduction of FS action (synchronous action) told that:
The FS commands are executed synchronously from within the FS action, the workflow job will wait until the specified file commands are completed before continuing to the next action.
Why synchronous and asynchronous introduction is basically the same?According to my understanding from the operating system principle course, asynchronous means the function does not wait but continue the execution.
Excerpt From: Mohammad Kamrul Islam and Aravind Srinivasan. “Apache Oozie.”
Asynchronous Actions : All Hadoop actions and the <shell> action follow the “Action Execution Model”. These are called asynchronous actions because they are launched via a launcher as Hadoop jobs.
Synchronous Actions : The filesystem action, email action, SSH action, and sub-workflow action are executed by the Oozie server itself and are called synchronous actions. The execution of these synchronous actions do not require running any user code—just access to some libraries.
Essentially, In both the cases Oozie server waits for the completion of the action and then only moves to the next action in the DAG. The separation is mainly based on whether actinos executes on the same Oozie server or on the Hadoop cluster.
Here is list of Oozie Actions and their Action Execution Model.
Am I understanding this correctly?
ISchedulerFactory sf = new StdSchedulerFactory();
IScheduler sched = sf.GetScheduler();
sched.Start();
Is the StdSchedulerFactory a singleton?
What I want to achieve is in my global,asax on Application_Start to start up the quartz scheduler, and then later on in one of my classes create the job, the trigger, get a handle again to the scheduler, and schedule the job/trigger
But as it turns out, I need to re-start the scheduler, I thought it was already started?
Or is there now more than 1 scheduler running?
The typical scenarios when embedding the scheduler in ASP.Net are to:
Create your scheduler as a singleton and then access it using something like Scheduler.Instance.
If you use DI, have the container give you a reference to the (singleton and already started) scheduler.
Use a global variable and have it reference the started scheduler.