I have a aspx web application that updates or adds files in a database. The clients access through the browser and one of the requirements is that they can start the update and be able to close the browser while the update continues. It appears to run for a little bit after I close the browser but then it stops. How can you keep the application running for asp.net?
That's something you could very well solve with WF (Workflow Foundation). Create a workflow for the task that should survive closing the browser. Workflows have their own threads and livecycles separate from ASP.NET.
The web application will keep running in the application pool, but this will be recycled eventually. As long as the users session runs the application should be kept alive, so by upping the session timeout you may fix the problem.
A better approach though would be to move the long-running task into a service instead, but that may require a rewrite of your application.
Usually for long-running or asynchronous processing, you want to dispatch the request to a back-end service to handle. Trying to keep the web-app alive to finish processing can lead to problems, especially with HTTP and session timeouts.
A common pattern for this is to put the request on a message queue and let a back-end service process it when it can.
I would create a separate windows service that you can push jobs onto from your web application, then check the status of the job(s) when the user logs in again.
The windows service won't be tied to the asp.net app domain so it will continue to run regardless of whats happening in your web application.
I've run into this pattern and you have to decouple the work from the HTTP request. The way we've solved it is to abstract the computing to be done as an event to be scheduled. So, say a user at a browser takes an action that requires a long lived (relatively) computation on the back end, this computation is given a name like 'doXYZForUser' and given a prameter vector like (userId, params...) and sent off to the work queue. Some time in the future the user logs in again and can see what the status of their job is.
I'm running a Java stack and a Java Message Service (JMS) but the principle is the same. The request from the browser queues up an event and the browser get an ACK back saying the event is on the work queue. The queue is managed by an entirely separately running process which in .NET I believe is just called the Message Queue. The job comes up on the queue gets processed and the results can be placed in a separate table containing a reference to the user that kicked off the job, so the next time they log in job status/results can be returned.
Related
We use SOLR for autocomplete search in our web application. We have a nightly process that adds a bunch of data to our database, and once that process finishes, we need to import that data from the database into our SOLR server.
Currently that import process is done manually in the following sequence:
Check if ondeck SOLR server is idle
Tell ondeck server to do a full import of the data, server status is "busy"
Poll ondeck server every so often and when it finishes, the status reverts to idle.
Swap the ondeck server (with the new information) with the live SOLR server.
Run a stored procedure on our database.
We have an administrative tools page on our web app. What I want to do is put a button on that page that kicks off that process (via AJAX call to our web application) and tells the user that the process has been kicked off. The entire process takes about an hour to do so I can't have the AJAX call run for an hour and then get back to the page telling the user that the process is finished.
I need it done asynchronously. The process has to be kicked off, once it gets kicked off, it informs the user that it's begun, and then the application has continue waiting on each step of the process and then call the next step until it finishes. What does the code in MVC 4 look like to make this happen? I've been looking at the tutorials on MSDN on async and await but I'm a little confused.
I may suggest you to take a look on Websokets (or long poling if you have to support old browsers). There is a really nice library called SignalR which will handle all complex staff for you. It use websockets if it is supported by client browser or automatically switch to long poling if it's not. It has good documentation and a lot of expamles on the Internet.
My application overview is
alt text http://img823.imageshack.us/img823/8975/modelq.jpg
ASP.Net webservice entertains requests from various applications for digital signing and verification via a client. The webservice will then route these requests to a smart card
When the system date changes, I want the following to happen.
New request from the clients are made to wait
Current work between webservice and smart card should get completed
If there is any prior pending requests then they should get completed.
The reason why I need the above things to happen is, I need to close the existing sessions between the smartcard and webservice. This should happen only when there is no signing/verification of files. I cannot just close all the sessions as it might affect a file being processed by any one of the threads. So I need to make sure that there are no current active threads between webservice and smart card.
I wrote a piece of code which gives the total number of active threads between webservice and smartcard.
int vWorkerThreads,vWorkerThreadsMax;
int vPortThreads,vPortThreadsMax;
System::Threading::ThreadPool ^ vThreadPool;
vThreadPool->GetAvailableThreads(vWorkerThreads, vPortThreads);
vThreadPool->GetMaxThreads(vWorkerThreadsMax, vPortThreadsMax);
ActiveThreadCount = vWorkerThreadsMax - vWorkerThreads;
This means, I also need to make the client requests wait?
CLEANUP MECHANISM: Close the PKCS#11 API using C_CloseAllSessions and C_Finalize call which will free up the library so that it cleans all the session objects. This should be done once everyday.
Any ideas on how I can perform such a task?
UPDATE:
I could have been much more clearer in my query. I want to make it clear that my aim is not to shutdown the ASP.NET webservice. My aim is to reset the smartcard. As I am accessing the smartcard via ASP.NET webservice, I need a mechanism to perform this task of resetting the smart card.
I am giving the current process below
Client detects Date change, At midnight
Client calls the function WebService_Close_SmartCard
Web Service receives the request WebService_Close_SmartCard and in turn
calls PKCS11_Close_SmartCard. This
Call will be served via one of the
available threads from the Thread
Pool. PKCS11_Close_SmartCard will
close all the existing current
sessions with the smartcard.
At this point, I want to make sure that there are no threads with
function calls such as
PKCS11_DigitalSign_SmartCard/
PKCS11_DigitalVerify_SmartCard talking
to smartcard, as
PKCS11_Close_SmartCard will abruptly
end the other ongoing sessions.
PS: I am new to ASP.NET and Multithreading.
The question was updated in a big way, so bear with me...
Given that no threads are being created directly\indirectly by your web method code:
Quesiton So you are not explicitly creating any new threads or using ThreadPool threads directly\indirectly, you are simply receiving calls to your web method and executing your code synchronously?
Answer Yes, you are correct. There is a client API which calls the webservice. Then the webservice manages the threads automatically(creats/allocates etc) inresponse to the client's demands.The webservice talks to a smart card by opening multiple sessions for encryption/decryption.
It is more helpful to rephrase the original question along the lines of "requests" rather than threads, e.g.
When the system date changes I want to re-start my ASP.NET application and ensure that all requests that are currently executing are completed, and that any outstanding\queued requests are completed as well.
This is handled automatically as there is a concept of a request queue and active requests. When your ASP.NET application is restarted, all current and queued requests are completed (unless they do not complete in a timely fashion), and new requests are queued and then serviced when a new worker process comes back up. This process is followed when you recycle the Application Pool that your ASP.NET application belongs to.
You can configure your application pool to recycle at a set time in IIS Manager via the "Recycle" settings for the associated Application Pool. Presumably you want to do this at "00:00".
Update
I think I can glean from your comments that you need to run some cleanup code when all requests have been serviced and then the application is about to shut down. You should place this code in the global "Application_End" event handler.
Update 2
In answer to your updated question. Your requirements are:
When the application is restarted:
New request from the clients are made to wait
Current work between webservice and smart card should get completed
If there is any prior pending requests then they should get completed.
This is supported by the standard recycling pattern that I have described. You do not need to deal with request threads yourself - this is one of the pillars of the ASP.NET framework, it deals with this for you. It is request orientated and abstracts how requests are handled i.e. serviced on multiple threads. It manages putting requests onto threads and manages the lifeclyle of those requests when the application is recycled.
Update 3
OK, I think we have the final piece of the scenario here. You are trying to shut down ASP.NET from your client by issuing a "CLOSED" web service call. Basically
you want to implement your own ASP.NET shut down behaviour by making sure that all current and queued request are dealt with before you then execute your clean-up code.
You are trying to re-invent the wheel.
ASP.NET already has this behaviour and it is supported by:
a. Application Recycling It will service outstanding requests cleanly and start-up a new process to serve new requests. It will even queue any new requests that are received whilst this process is going on.
b. Application_End A global application event handler where you can put your clean-up code. It will execute after recycling has cleanly dealt with your outstanding requests.
You do not need your "CLOSED" command.
You should consider letting IIS recycle your application as it has support for recycling at a specified daily time(s). If you cannot configure IIS due to deployment reasons then you can you use web.config "touching" to force a recycle out-of-bounds of IIS:
a. Have a timer running in the server which can check for the date change condtion and then touch the web.config file.
b. Still have the client call a "CLOSED" web method, but have the "CLOSED" method just touch the web.config file.
IIS, then "a" are the most desirable.
Honestly Microsoft have already thought about it. :)
Update 4
#Raj OK, let me try and rephrase this again.
Your conditions are:
You have a requirement to reset your smartcard once a day.
Before resetting your smartcard, all current and queued web service requests must be completed i.e. the outstanding requests.
After outstanding requests are completed, you reset your smartcard.
Any new requests that come in whilst this process is happening should be queued and then serviced after the smartcard has been reset.
These conditions allow you to complete existing requests, queue any new requests, reset your smartcard, and then start processing new requests after the card has been reset.
What I am suggesting is:
Place your smartcard reset code in "Application_End".
Configure IIS to recycle your application at "00:00". Ensure that in advanced settings for the associated Application Pool that you configure "Disable Overlapped Recycle = True".
At "00:00" application recycling ensures that all current and queued requests will be completed.
After "00:00" application recycling ensures that all new requests will be queued whilst requests in "3" are completed and the application performs shutdown steps.
After requests in "3" are completed, "Applicaton_End" will be called automatically. This ensures that your smartcard is reset after all current requests are completed.
Application recycling ensures that your application is re-started in a new process, and that new requests queued in step "4" start to be processed. The important thing here is that your reset code has been called in "5".
Unless there is some detail missing from your question, the above appears to meet your conditions. You wish to do "x,y,z" and ASP.NET has built-in support which can be used to achieve "x,y,z" and gives you mature, guaranteed and well-documented implementations.
I am still struggling to understand why you are talking about threads. I do multi-threaded development, but talking about threads instead of requests when thinking about ASP.NET adds unnecessary complexity to this discussion. Unless your question is still unclear.
Perhaps you are missing the point I'm making here. I am drawing a parallel between the behaviour you require when you call "CLOSED" from your client application, and what happens when you recycle an application. You can use recycling and "Application_End" to achieve the required results.
I am trying to help you out here, as trying to implement this behaviour yourself is unnecessary and non-trivial.
What are your most successful ways of running a long process, like 2 hours, in asp.net and return information to the client on the progress.
I've heard creating a windows service, httphandler and remoting can be successful.
Just a suggestion...
If you have logic that you are tyring to utilize already in asp.net... You could make an external app (windows service, console app, etc.) that calls a web service on your asp.net page.
For example, I had a similiar problem where the code I needed was asp.net and I needed to update about 3000 clients using this code. It started timing out, so I exposed the code through a web service. Then, instead of trying to run the whole 3000 clients at through asp.net all at once, I used a console app that is run by a nightly sql server job that ran the web service once for each client. This way all the time consuming processing was handled by the console app that doesn't have the time out issue, but the code we had already wrote in asp.net did not have to be recreated. In the end slighty modifying the design of my existing architecture allowed me easily get around this problem.
It really depends on the environment and constraints you have to deal with...Hope this helps.
There are two ways that I have handled this. First, you can simply run the process and let the client time out. This has two drawbacks: the UI isn't in synch and you are tying up an IIS thread for non-html purposes (I did this for a process that used to return quickly enough but that grew beyond time-out limits).
The better way to handle this is to write a "Service" application that handles the request as passed through a database table (put the details of the request there). Then you can create a window that gives the user a "window" into ongoing progress on the task (e.g. how many records have been processed or emails sent). This status window can either have a link to permit the user to refresh or you can automate the refresh using Ajax callbacks on a timer.
This isn't directly applicable but I wrote code that will let you run processes similar to "scheduled tasks" inside of ASP.NET without needing to use windows services or any type of cron jobs.
Scheduled Tasks in ASP.NET!
I very much prefer WCF service to scheduled tasks. You might (off the top of my head) pass an addr to the WCF service as a sort of 'callback' that the service can call with progress reports as it works.
I'd shy away from scheduled tasks... too course grained.
I'm encountering a situation where it takes a long time for ASP.NET to generate reply with the web page (more than 2 hours). It due to the codebehind running for a while (very long, slow loop).
Browser (both IE & Firefox) stops waiting for the reply (after about an hour) and gives generic cannot display webpage error (similar to what you would see if you'd try to navige to non-existing server).
At the same time asp.net app keeps going (I can see it in debugger) and eventually completes.
Why does this happen? Are there any settings in web.config to influence this? I'm hoping there's a timeout setting that I'm missing that's causing this.
Maybe a settings in IE or Firefox? But I think they wait while the server is keeping connection alive.
I'm experiencing this even when I launch app in debug mode (with compilation debug="true") on my local machine from VS (so it's not running on IIS, but on ASP.NET Dev Server).
I know it's bad that it takes so long to generate the page, but it doesn't matter at this stage. Speeding it up would take a lot of extra work and the delay doesn't really matter. This is used internally.
I realize I can redesign around this issue running logic to a background process and getting notified when it's done through AJAX, or pull it to a desktop app or service or whatever. Something along those lines will be done eventually, but that's not what I'm asking about right now.
Sounds like you're using IE and it is timing out while waiting for a response from the server.
You can find a technet article to adjust this limit:
http://support.microsoft.com/kb/181050
CAUSE
By design, Internet Explorer imposes a
time-out limit for the server to
return data. The time-out limit is
five minutes for versions 4.0 and 4.01
and is 60 minutes for versions 5.x, 6,
and 7. As a result, Internet Explorer
does not wait endlessly for the server
to come back with data when the server
has a problem. Back to the top
RESOLUTION
In general, if a page does not return within a few
minutes, many users perceive that a
problem has occurred and stop the
process. Therefore, design your server
processes to return data within 5
minutes so that users do not have to
wait for an extensive period of time.
The entire paradigm of the Web is of request/response. Not request, wait two hours, response!
If the work takes so long to do, then have the page request trigger the work, and then not wait for it. Put the long-running code into a Windows service, and have the service listen to an MSMQ queue (or use WCF with an MSMQ endpoint). Have the page send requests for work to this queue. The service will read a request, maybe start up a new thread to process it, then write a response to another queue, file, or whatever.
The same page, or a different, "progress" page can poll the response queue or file for responses, and update the user, assuming the user still cares after two hours.
For something that takes this long, I would figure out a way to kick it off via AJAX and then periodically check on it's status. The background process should update some status variable on a regular basis and store it's data in the cache or session when complete. When it completes and the browser detects this (via AJAX), have the browser do a real postback (or get by changing location.href), pick up the saved data, and generate the page.
I have a process that can take a few minutes so I spin off a separate thread and send the result via ftp. If an error occures in the process I send myself an error message including the stack trace. You may want to consider sending the results via email or some other place then the browser and use a thread as well.
I have written an HttpModule that spawns a background thread. I'm using the thread like a Scheduled Task that runs in-process, which is really handy.
What are best practices for keeping track of this thread? I've never done this before, and I'm a little confused about some aspects of it:
How do I know if the thread is still running? I see it do its job, but is there another way to know if it's still alive? I downloaded ProcMon, but w3wp.exe spawns a boatload of threads, so I had no idea which was my thread. I named it, but that didn't help.
How do I "catch" the thread if it dies? Is there some kind of Dispose method where I can have it write to the EventLog or something if it fails? A "dying declaration" or something?
How do I actively stop the thread? If I want it to stop running this background process, how do I kill it without having to bounce IIS?
Is there anyway to start it again, independently of the HttpModule? (I'm guessing the answer to this is no...)
Edit: Just to clarify, the intention is that my thread never goes away. It runs a function, then goes to sleep for a couple minutes, then wakes up and runs the function again. It's not like it's doing one task then ending.
When Jeff made Stackoverflow, he had a similiar issue.
His solution was to use the cache expiration. You'd put something in the cache and then when it expires an event is fired in a non-user facing thread. In the event handler for the expiration, you stick some code in to re-add the item to the cache and do whatever housekeeping work needs to be done for your application
Using this technique, your subquestions are easily answered:
You check the item is still in
cache.
If the item is not in
cache, re-add it.
Remove the
cache item from the cache.
Add
the item back to the cache.
You could make a small management page to configure these options.
This gives you a nice way to roughly time housekeeping processes in your web-application. It doesn't require a seperate Windows Service, which is a big win.
From my experience, you can get this to work "good enough", but not perfect. I would recommend to implement your repeating tasks in a windows service. Depending on what the tasks do, the Windows Service maybe wouldn't even have to talk to the web application or vice versa, e. g. if both work with the same database. Otherwise you could still use e. g. WCF for communication.
The big advantage is: The windows service will start with the OS, you can easily configure, start and stop it using the control panel, you have built-in monitoring via Windows Event Log, you can update the background service and the web application independently etc.
If this is not an option, e. g. because you are in a shared hosting environment, I would recommend the following:
Start your background thread in Application_start (Global.asax), and store the thread reference in a static variable.
Wrap every method called on your background thread with try/catch, because since .NET 2.0, every unhandled exception on a background thread will shut down the application. (It will be restarted on the next request, but it slows down the next request, kills all current sessions and caches, and of course no timer will be active until the next request.)
On every request (implemented has a HttpModule or in Global.asax again), check the Thread instance in the global variable (is it still != null, is the thread active and running etc.). If not, call the restart code. Use locking in the restart part to make sure that the thread will not be created twice at same time.
Even then you can't be sure that your background thread is always running, if you don't have regular traffic around the clock. Also keep in mind that in a shared hosting environment it is very common to shut down application pools if there is no activity for a few hours. You could try to improve this by setting up a scheduled task on a client machine on your own network doing a light-weight HTTP request on your application every few minutes, just to ensure that your application is always running.