I have used Quartz.Net for queuing and sending emails from my application. I don't know how each scheduled job responds to application instance stopping, pausing or shutting down. The IJob interface has no method that can notify a running job about these events.
My question is how can I handle these cases when they occur so that the job can exit while leaving the application and the data in a stable state?
Make sure you call IScheduler.Shutdown(true) when your application shuts down. This will wait for all jobs to finish and ensure everything is cleaned up correctly.
Related
This is a design question about the handling of tasks during the shutdown of a firebase-queue based app running on Google Compute Engine.
The use case I am working with is automatically scaling queue-workers depending on the load at any given time. Specific to our project is the fact that our tasks are long-running.
In an ideal world, the queue worker would have an opportunity to finish its current tasks before the virtual machine running the worker is terminated. We are working with Google Compute Engine / instance groups to handle the scaling of our queue worker app. Firebase-queue does provide a promise based method to shutdown a queue worker (i.e. queue.shutdown()). This will stop the worker from accepting new tasks and will allow running tasks to finish prior to resolving the promise.
The problem I am facing is how to allow the queue worker to shutdown gracefully prior to instance termination (this problem would also occur during a rolling update). One way is to trigger the worker shutdown and have the worker trigger instance shutdown, but this does not seem like the best design because control is taken away from whatever service is triggering the scale down in the first place.
GCE does provide a service which will run a shutdown script prior to instance termination, however, it will forcefully shutdown an instance after about 90 seconds, which does not work for us.
I am interested in design ideas / patterns to follow here. Any help is much appreciated.
javafx.concurrent.Service uses internally a java.util.concurrent.ExecutorService to execute its Tasks. Instances of ExecutorService need to be shut down after usage. This does not seem to be the case for javafx.concurrent.Service. How and when does javafx.concurrent.Service shutdown its ExecutorService ?
I think your misunderstanding here comes from:
Instances of ExecutorService need to be shut down after usage.
Calling shutdown() prevents an ExecutorService from accepting new tasks. You only need to do that if it makes sense to do so (e.g. you are trying to exit the application and want to make sure that any new tasks submitted during application exit are ignored. This might happen with a scheduled executor service, for example).
The related shutdownNow() will additionally attempt to interrupt any currently running threads. So if your tasks are implemented to accept interrupts gracefully, calling shutdownNow() gives those tasks the opportunity to perform any cleanup operations necessary (closing connections, etc).
In many use cases, however, there is no need to call either of these methods. If you are assured no further tasks will be submitted to the executor, shutdown() is unnecessary. If your tasks are not long-running, shutdownNow() is unnecessary. When the application attempts to exit, any existing tasks will complete (presumably reasonably quickly) and then the application can exit.
Note that if your executor service uses daemon threads, then when the application attempts to exit, those threads will not prevent application exit (they will be terminated, without interruption). So if your tasks are short-lived, require no cleanup, and can be safely terminated at any point, this is a viable strategy.
There is nothing special about javafx.concurrent.Service here: in some sense it is a wrapper for an ExecutorService that provides additional functionality for interacting with the FX Application Thread. Just note that the default executor service provided uses daemon threads, so as above, if your tasks need cleanup you would likely provide a different executor service and shut it down gracefully in the application's stop() method.
When using a DNX(core) ASP.NET 5 application, the Configure() method can be used to subscribe to the following cancellation events:
IApplicationLifetime::ApplicationStarted
IApplicationLifetime::ApplicationStopping
IApplicationLifetime::ApplicationStopped
However, the only way I can see ATM to properly terminate an application is to call
IApplicationLifetime::StopApplication()
from within the application or press CTRL+C using Kestrel and running in a console.
Obviously the target here is to host the application as a background application (ideally as a cloud deployment).
Unfortunately I haven't been able to get a demo application to run on IIS.
When using IBM BlueMix, I've noticed that the stopping and stopped events are never fired and apparently the application is just stopped.
What is the proper way to deal with this? Are these events unreliable or am I doing something wrong?
Help is greatly appreciated. Thanks in advance!
Right now, it's not possible to detect when the process is being killed.
There's 2 ways that even is going to fire as of now.
CTRL-C in the console window of Kestrel
The Application is being disposed.
As far as it seems, they will have something ready when they hit RTM.
As for when a process gets killed, if BlueMix is closing on windows, it should send the proper WM_Close (0x0010) event and SIGTERM if you are under a Linux/OSX system.
If BlueMix is sending SIGKILL, then there's nothing that can be done about events.
LIBUV Update:
After digging a bit deeper, if you have any control over the signals that are sent, I recommend not sending any of those: SIGILL, SIGABRT, SIGFPE, SIGSEGV, SIGTERM and SIGKILL.
ASP.NET 5 is based on libuv and they might be limited to SIGINT or SIGHUP to terminate the application since libuv will not allow you to handle SIGTERM.
The thread will be started on each Application_Start event.
It will be a monitoring thread which is supposed to run constantly.
So even if the app shuts down, once it is restarted the thread will start too ensuring it runs all the time.
However I need to be sure that this thread will not be stopped / shut down while the application is running.
So in a few words, does anybody know if asp.net could shut down such a thread without actualy stopping / recycling the application.
As a matter of design, you shouldn't depend on asp.net to run threads like this. Little things like app recycling can cause you a lot of trouble.
Instead, create a windows service to execute the thread. This way you don't have to worry about it.
Update
I just wanted to add a little more information.
IIS has the ability to execute your app across multiple threads and processes. A standard site installation usually only has a single process (aka: web garden) assigned which spins up around 20 threads to handle request processing.
However, any IIS administrator can easily add more processes to the mix. They usually do this when a site can hose a single process either because request processing takes too long, or the number of handler threads isn't enough, or as a temporary measure if the app has enough problems that a single thread will hose the entire process fairly often.
If you have a thread being spun on app start, then it will create one for each worker process the site has. This may be unexpected behavior to you or your successors.
Also, monitoring apps are almost always completely separate to the application they are monitoring. One of the primary reasons is that in the event the monitored process dies, hangs, or otherwise becomes unresponsive then the monitoring app itself still needs to carry on and log this information. Otherwise the monitored process could very well hose the monitoring app itself.
So, do yourself a favor and move this to its own process. The best way to do this on an IIS server is to create a windows service and give it the appropriate execution rights to do what you need.
we have a scenario where in whil serving one asp.net request from iis from our code we have created a child thread from thread pool to servve some background task.the idea was to finsih the main thread wihich is processing the request without depending on our child thread task. But our doubt is while processing a request in asp.net will workerprocess wiat untill the thread finishes its task ?
No, if you've created a separate thread and don't have any code to wait on it, the request will certainly complete.
What I don't know is whether a worker process can be recycled while there are non-ASP.NET thread pool threads executing. I strongly suspect it can - so be aware that your child task could be terminated at any point. If this is a problem, you may wish to create a non-threadpool foreground thread.
No, the background thread will not wait for the main thread to finish, it will start immediately. If you use the Thread pool your task may have to wait for a thread to be available, but that is all.
You should however be aware that this may not be a good solution, for the application domain your asp.net site is running in may be recycled while your background thread is running. This will kill the background thread leaving the task incomplete. If it is possible, you should separate the background task into a service application that runs in a separate process. This will ensure that the task isn't killed while running.