What are your most successful ways of running a long process, like 2 hours, in asp.net and return information to the client on the progress.
I've heard creating a windows service, httphandler and remoting can be successful.
Just a suggestion...
If you have logic that you are tyring to utilize already in asp.net... You could make an external app (windows service, console app, etc.) that calls a web service on your asp.net page.
For example, I had a similiar problem where the code I needed was asp.net and I needed to update about 3000 clients using this code. It started timing out, so I exposed the code through a web service. Then, instead of trying to run the whole 3000 clients at through asp.net all at once, I used a console app that is run by a nightly sql server job that ran the web service once for each client. This way all the time consuming processing was handled by the console app that doesn't have the time out issue, but the code we had already wrote in asp.net did not have to be recreated. In the end slighty modifying the design of my existing architecture allowed me easily get around this problem.
It really depends on the environment and constraints you have to deal with...Hope this helps.
There are two ways that I have handled this. First, you can simply run the process and let the client time out. This has two drawbacks: the UI isn't in synch and you are tying up an IIS thread for non-html purposes (I did this for a process that used to return quickly enough but that grew beyond time-out limits).
The better way to handle this is to write a "Service" application that handles the request as passed through a database table (put the details of the request there). Then you can create a window that gives the user a "window" into ongoing progress on the task (e.g. how many records have been processed or emails sent). This status window can either have a link to permit the user to refresh or you can automate the refresh using Ajax callbacks on a timer.
This isn't directly applicable but I wrote code that will let you run processes similar to "scheduled tasks" inside of ASP.NET without needing to use windows services or any type of cron jobs.
Scheduled Tasks in ASP.NET!
I very much prefer WCF service to scheduled tasks. You might (off the top of my head) pass an addr to the WCF service as a sort of 'callback' that the service can call with progress reports as it works.
I'd shy away from scheduled tasks... too course grained.
Related
I need to run a long running task like report export, report import feature which will take few minutes to hour to complete the task. I want to start the task as soon as user gives command from the asp.net mvc view.
I found various web portals, but I couldn't find the exact solution I was looking for or might be I didn't got their idea (or, limited by my experience). I don't need something that runs at certain time interval or at certain time-stamp of the day/week. I only need the background job to be executed on user request and update the task as completed in DB after it's done. So, the background job need to be always triggered by user only.
I assume the ideal solution will be something like, Background service/job is always in ready state to accept trigger from my web application in some kind of port and as soon as it receives command from web application, it (a) either send everything to job when requested for action (b) or, just poke job to start its task (based on the updated db table entry made just before the call action) . So, the user can browse through other pages in the application and can view the progress of all such tasks in one of the view (where he can even cancel or see the completed task). Note, that there can be simultaneous request for the job from different web users which should be treated as separate job (As data export differs based on user access-rights).
I want to go with the simple method, with 100% assurance of task to be done/not done as recorded in db (no crashing of service). What are the best approaches (except azure jobs) as i want to implement the service within same server and I have access to the server/virtual machine to install windows services/packages if needed.
Take look at this great article that Hanselman wrote:
How to run Background Tasks in ASP.NET
Also Hangfire is a mature library that can make development of application long running process more easy.
Hangfire is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don't want to put in your request processing pipeline
client wants an asp.net page that has a button to fire off a database update from an external source with hundreds of records. This process takes a long time. He also wants status update as the process runs, like "processing 10 out of 1000 records". In reading various articles, I'm thinking of putting the database update code in a windows service. I've never worked with windows services before and I can't find many tutorials on how to fire off a windows service and poll it from an asp.net page. My questions are is this the best way to handle this process? And, does anyone have any examples on how they've accomplished this?
There are a few ways to approach this.
You're right in that executing a long-running task within the Web's worker process doesn't usually end well: it ties up resources, the app pool can get recycled, etc. In most of my projects of any complexity, I usually end up with 4 pieces: the database, a DLL with my model, a "Worker" that is a Windows service, and an ASP.NET Web site.
The "Worker" is a Windows service that is always running and uses Quartz.net to execute scheduled tasks using the same model that the Web site uses. These can be all sorts of periodic tasks that seem to crop up when maintaining a Web site of any complexity: VacuumExpiredPickTicketsJob, BackupAndFtpDatabaseJob, SendBackorderReminderEmailsJob, etc.
Writing a Windows service is not difficult in C# (there is a built-in template in Visual Studio, but you pretty much inherit from ServiceBase and you're off to the races), and libraries like TopShelf make it even easier to deploy them.
What is left is triggering the update from the Web site and communicating the results back to the user. This can be as simple or as complicated as you want it to be. If this is something that has to scale up to lots of users, you might use something like MSMQ to queue up update commands to the Windows service, and the Windows service would respond to that queue. I get the impression that that is probably overkill here.
For a handful of users, you could override your service's OnCustomCommand(int command) method to be the trigger. Your Web site would then use ExecuteCommand() of the ServiceController class to get the process started. Your Web site and service would agree on the parameter value that means "do that update thing," let's say 142 (since it has to be a number between 128 and 255 for reasons of history).
As for communicating progress back to the client, it's probably easiest to just have the Web page use a timer and an AJAX call to poll for updated progress data. You can get fancy with new stuff like WebSockets (bleeding edge stuff as I write this) and long polling, but regular polling will simply work for something that doesn't need to scale.
Hope this helps!
In addition to Nicholas' thorough answer, another option is to deploy your back end processes as command line scripts, and schedule them to run through Window's built in task scheduler, which has improved quite a bit in Windows Server 2008+. Or you can use any other host of task scheduler applications.
I find the command line approach to be easier for MIS staff to understand and configure, and to migrate to new servers, versus standard Windows services.
We require that in a ASP.Net application, a .Net process should be invoked every day at a specified time automatically. This process needs to interact with the database (SQL Server 2005) and generate billing on a daily basis. We are using a shared hosting hence we are not able to create a windows service or create SQL Server jobs. How can this be achieved without user intervention?
You could try the technique described here, used at StackOverflow itself (or at least it was used here at one point). In a nutshell:
At startup, add an item to the HttpRuntime.Cache with a fixed
expiration.
When cache item expires, do your work, such as WebRequest or what have
you.
Re-add the item to the cache with a fixed expiration.
To get it to run at a specific time instead of an interval, you could lower the interval and simply modify your working method to check the time itself.
As the comments in the original article linked above note, this isn't a perfect solution, and no one should prefer it over a proper scheduling technique if one is available. See When Does Asp.Net Remove Expired Cache Items? for some additional qualifications.
Yes, use Windows Scheduler. Depending on how it's configured you might need to be logged in for the scheduler to run.
You could always schedule a task to run a webservice..
http://weblogs.asp.net/jgalloway/archive/2005/10/24/428303.aspx
The scheduler would run a VBS file with the following..
Set oServerXML = CreateObject("Msxml2.ServerXMLHTTP")
oServerXML.Open "GET","http://my.hostedservice.com/myService.asmx/myService?aParam=Value
oServerXML.setRequestHeader "Content-Type","application/x-www-form-urlencoded"
oServerXML.Send
Set oServerXML = nothing
Can't be done, unfortunately.
IIS only responds to requests, and SQL Server only wakes up for jobs.
The closest you'll be able to do is to put your routine in an ASPX page, not linked from the site and not with an obvious name, and trigger it by a request from some other machine out on the Internet.
The other machine could be a Windows, Linux, Mac, whatever you have available, and all of those platforms have ways of scheduling events (service, cron, etc.) that can make the request to trigger the update on the server.
There are ways to run "services" in .Net by using cache expiration to trigger the task.
More at CodeProject
You can use a Scheduled Task, but this might not work in a shared hosting environment either.
You could setup a webservice or page on your website to kickoff the process, then have a scheduled task on a desktop machine hit that page/service once daily to start the process. Hacky, but it might work.
Being .NET ignorant, I would imagine there's some kind of .NET based scheduler framework available for this (much like Quartz for Java).
Or you could simply fire off a long running thread that spends the bulk of its time sleeping, wake up every minute, check the time, check it's list of "things to do", fire off the ones that need to be done. Level of sophistication being as far as you want to take it, but the primary goal of keeping the primary scheduling thread "alive", "at all costs".
What i can think about now are:
Create a dll which contain the
schedule logic you want, and make
sure that this dll schedule function
will not stop and will loop for ever,
then you will need a page on that
server this page will fire this dll
functions. "you will need to call
this page at least once to start the
scheduler".
Create an application "holds schedule logic" on another machine, may be your Home PC, and make your pc application call the functions on the server through webservices or pages
We have an ASP.Net application that provides administrators to work with and perform operations on large sets of records. For example, we have a "Polish Data" task that an administrator can perform to clean up data for a record (e.g. reformat phone numbers, social security numbers, etc.) When performed on a small number of records, the task completes relatively quickly. However, when a user performs the task on a larger set of records, the task may take several minutes or longer to complete. So, we want to implement these kinds of tasks using some kind of asynchronous pattern. For example, we want to be able to launch the task, and then use AJAX polling to provide a progress bar and status information.
I have been looking into using the BackgroundWorker class, but I have read some things online that make me pause. I would love to get some additional advice on this.
For example, I understand that the BackgroundWorker will actually use the thread pool from the current application. In my case, the application is an ASP.Net web site. I have read that this can be a problem because when the application recycles, the background workers will be terminated. Some of the jobs I mentioned above may take 3 minutes, but others may take a few hours.
Also, we may have several hundred administrators all performing similar operations during the day. Will the ASP.Net application thread pool be able to handle all of these background jobs efficiently while still performing it's normal request processing?
So, I am trying to determine if using the BackgroundWorker class and approach is right for our needs. Should I be looking at an alternative approach?
Thanks and sorry for such a long post!
Kevin
In your case it actually sounds like the solution you will be looking for is multifaceted (and not a simple in and done project).
Since you said that some processes can last for hours that is absolutely not something for ASP.NET to own. This should be ran inside a windows service and managed with native windows threading.
You will need to implement some type of work queue in your service and a way to communicate with the queue. One way is to expose a WCF service for all actions your service will govern. Another would be to have service poll a database table and pick up work from the table.
To be able express the status of the process you will want the ASP.NET application to be able to have some reference to the processID for example the WCF service returns a guid identifier. Then you have a method that when you give it the processID it will return the status of the process. You can then implement the polling of that service call using AJAX and display any type of modal you wish.
Another thing to remember is that you need to design your processes to have knowledge of where it is and where it will be when it is finished so it can track the state it's in. For example, BatchJobA is run and will have 1000 records to process. The service needs to know what record it's on or what the current % of competition is for it to be able to return information to the UI. For sql queries that take a very long time to execute this can be very problematic to accurately gauge where it is unless you do alot of pre and post processing of temp tables that you can in the middle of it read the status of the temp tables to understand where it is.
Based on what you are saying I think that BackgroundWorker is not a good choice.
Furthermore keeping this functionality as a part of your main app can be problematic, specifically because you do not want the submitted processing to be interrupted if the main app recycles. You can play with asynch processing but it still will be a part of the main app AppDomain - all of it will die if the app recycles.
I would suggest buidling a separate app implementing this functionality. In a similar situation I separated background processing to a Windows service and hosted a web service in it as a means of communication
You might consider a slightly different approach.
For example, have a command and control table in which you send commands like "REFORMAT PHONE NUMBERS" or whatever.
Then have a windows service monitoring that table. Whenever a record shows up, run the command.
This eliminates any sort of worry about a background thread. Further you have a bit more flexibility with regards to what's in the queue, order of operations including priority, etc. Finally, you would have a definitive list of what is running or needs to run.
As an option, instead of a windows service you might just use a SQL job to execute every so often to watch your control table and perform the requested action.
I have a aspx web application that updates or adds files in a database. The clients access through the browser and one of the requirements is that they can start the update and be able to close the browser while the update continues. It appears to run for a little bit after I close the browser but then it stops. How can you keep the application running for asp.net?
That's something you could very well solve with WF (Workflow Foundation). Create a workflow for the task that should survive closing the browser. Workflows have their own threads and livecycles separate from ASP.NET.
The web application will keep running in the application pool, but this will be recycled eventually. As long as the users session runs the application should be kept alive, so by upping the session timeout you may fix the problem.
A better approach though would be to move the long-running task into a service instead, but that may require a rewrite of your application.
Usually for long-running or asynchronous processing, you want to dispatch the request to a back-end service to handle. Trying to keep the web-app alive to finish processing can lead to problems, especially with HTTP and session timeouts.
A common pattern for this is to put the request on a message queue and let a back-end service process it when it can.
I would create a separate windows service that you can push jobs onto from your web application, then check the status of the job(s) when the user logs in again.
The windows service won't be tied to the asp.net app domain so it will continue to run regardless of whats happening in your web application.
I've run into this pattern and you have to decouple the work from the HTTP request. The way we've solved it is to abstract the computing to be done as an event to be scheduled. So, say a user at a browser takes an action that requires a long lived (relatively) computation on the back end, this computation is given a name like 'doXYZForUser' and given a prameter vector like (userId, params...) and sent off to the work queue. Some time in the future the user logs in again and can see what the status of their job is.
I'm running a Java stack and a Java Message Service (JMS) but the principle is the same. The request from the browser queues up an event and the browser get an ACK back saying the event is on the work queue. The queue is managed by an entirely separately running process which in .NET I believe is just called the Message Queue. The job comes up on the queue gets processed and the results can be placed in a separate table containing a reference to the user that kicked off the job, so the next time they log in job status/results can be returned.