Scenario
I have an web application which needs some calculations and processing on data. This job is a long running job(few hours). Job is initiated by user.
Requirement.
User Clicks on Process Data.
Some functions are called to start data processing.
Data Processing runs for hours.
User is given feedback of percentage completed etc.
Even if user logs off and then again log on he should get this feedback.
The requirement is somewhat similar to Spiceworks. Where it runs in background to detect the devices/computers in network and the user is notified in his page about the progress. But spicework uses windows service. We don't want to us windows service.
Now the question is.
What if user closes the page, will the task still run in background.
This task has to be completed fully.If terminated in between output will not have any meaning.
How to actually to design this long running process. In ASP.Net environment.
Also is there a way to show all/same user who logs in the status of processing.
There are multiple ways to schedule a job in the background. You can use SQL Job, Windows Service or Scheduled Tasks.
I would design it like this:
From my ASP.NET page - I will store an indication in the database for the job to start which will then be picked by the scheduled task. This task is nothing but a console application which pulls data from the database to see which tasks user initiated and then take the next action in there.. For the percetage complete you can store those values from your job into DB and your page will access the dB to show it to user anytime they come to the page.
Here is another thread where long running tasks in IIS are discussed:
Can I use threads to carry out long-running jobs on IIS?
Related
I need to run a long running task like report export, report import feature which will take few minutes to hour to complete the task. I want to start the task as soon as user gives command from the asp.net mvc view.
I found various web portals, but I couldn't find the exact solution I was looking for or might be I didn't got their idea (or, limited by my experience). I don't need something that runs at certain time interval or at certain time-stamp of the day/week. I only need the background job to be executed on user request and update the task as completed in DB after it's done. So, the background job need to be always triggered by user only.
I assume the ideal solution will be something like, Background service/job is always in ready state to accept trigger from my web application in some kind of port and as soon as it receives command from web application, it (a) either send everything to job when requested for action (b) or, just poke job to start its task (based on the updated db table entry made just before the call action) . So, the user can browse through other pages in the application and can view the progress of all such tasks in one of the view (where he can even cancel or see the completed task). Note, that there can be simultaneous request for the job from different web users which should be treated as separate job (As data export differs based on user access-rights).
I want to go with the simple method, with 100% assurance of task to be done/not done as recorded in db (no crashing of service). What are the best approaches (except azure jobs) as i want to implement the service within same server and I have access to the server/virtual machine to install windows services/packages if needed.
Take look at this great article that Hanselman wrote:
How to run Background Tasks in ASP.NET
Also Hangfire is a mature library that can make development of application long running process more easy.
Hangfire is an open-source framework that helps you to create, process and manage your background jobs, i.e. operations you don't want to put in your request processing pipeline
Background: This is largely a learning exercise. I am working on a task scheduling program. I have been asked to make it so that the application I am working on will send emails to users telling them they have messages waiting to be looked at. I already have the part that sends emails working right, but that only works when the user is actively on a page. I essentially want to make it so that the server decides to send emails at a certain time for instance midnight to tell users how many messages and meetings they have unread updates on during that day. I am not familiar with how to make the app do work when not receiving input from the user.
What I am looking for: How do I make ASP run a function on at a scheduled time.
Here we had difficulties with the functionality of a browser when scheduling the browser (starting with a specific page) on the server.
A Temporary solution was to make one page in your project that reloads after waiting 5 minutes <META http-equiv='refresh' content='6000'>, and start this page on a client or on the server. Note: The copy running on the server stops when rebooting the server.
The final solution was to make a .Net package using the right server-side objects, and schedule this package in task scheduler.
Use Windows Schedule Task to get IE to run the page at a scheduled interval.
e.g.
You will have to make sure that it runs unattended and with an account that is allowed to run it. Also you will want the page to close the window after having finished.
(How to configure and what Schedule Task looks like varies depending on your Windows Version)
I would like to know the best way to deal with long running processes started on demand from an ASP.NET webpage.
The process may consist of various steps (like upload files to the server, run SSIS packages on them, execute some stored procedures etc.) and sometimes the process could take up to couple of hours to finish.
If I go for asynchronous execution using a WCF service, then what happens if the user closes the browser while the process is running, how the process success or failure result should be displayed to the user? To solve this, I choose one-way WCF service calls, but the problem with this is I need to create a process table and store the result (and error messages if it fails in any of the steps and which steps have completed successfully) in that table which is an additional overhead because there are many such processes with various steps that the user can invoke from the web page and user needs to be made aware of the progress (in simplest case, the status can be "process xyz running") and once it is done, the output needs to be displayed to the user (for example by running a stored procedure).
What is the best way to design the solution for this?
As I see it, you have three options
Have a long running page where the user waits for the response. If this is several hours, you're going to have many usability problems, so I wouldn't even consider it.
Create a process table to store the results of operations. Run service functions asynchronously and delegate logging the results to the service. There can be a page that the user refreshes which gets the latest results of this table.
If you really don't want to create a table, then store all the current process details in the users' session state, and have a current processes page as above. You have the possible issue that the session might timeout, or the web app might restart and you'll lose all this.
I can't see that number 2 is such a great hardship. You could make the table fairly generic to encompass all types of processes: process details could just be encoded as binary or xml and interpreted by the web application. You then have the most robust solution.
I cant say what the best way would be but using Windows Workflow Foundation for such long running processes is definitely one way to go about it.
You can do tracking of the process to see what stage it is at, even persist it if you have steps where it is awaiting user input etc.
WF provides a lot of features out of the box (especially if your storage medium is SQL Server) and may be a good option to consider.
http://www.codeproject.com/KB/WF/WF4Extensions.aspx might help give you some more insight into the same.
I think you are in the right track. You should run the process asynchronously, store the execution somewhere (a table), and keep status of the running process in there.
Your user should see a pending display label while the process is executing, and a finished label with the result when the process finished. If the user closed the browser, she will see the result of her running process next time she logs in.
What are your most successful ways of running a long process, like 2 hours, in asp.net and return information to the client on the progress.
I've heard creating a windows service, httphandler and remoting can be successful.
Just a suggestion...
If you have logic that you are tyring to utilize already in asp.net... You could make an external app (windows service, console app, etc.) that calls a web service on your asp.net page.
For example, I had a similiar problem where the code I needed was asp.net and I needed to update about 3000 clients using this code. It started timing out, so I exposed the code through a web service. Then, instead of trying to run the whole 3000 clients at through asp.net all at once, I used a console app that is run by a nightly sql server job that ran the web service once for each client. This way all the time consuming processing was handled by the console app that doesn't have the time out issue, but the code we had already wrote in asp.net did not have to be recreated. In the end slighty modifying the design of my existing architecture allowed me easily get around this problem.
It really depends on the environment and constraints you have to deal with...Hope this helps.
There are two ways that I have handled this. First, you can simply run the process and let the client time out. This has two drawbacks: the UI isn't in synch and you are tying up an IIS thread for non-html purposes (I did this for a process that used to return quickly enough but that grew beyond time-out limits).
The better way to handle this is to write a "Service" application that handles the request as passed through a database table (put the details of the request there). Then you can create a window that gives the user a "window" into ongoing progress on the task (e.g. how many records have been processed or emails sent). This status window can either have a link to permit the user to refresh or you can automate the refresh using Ajax callbacks on a timer.
This isn't directly applicable but I wrote code that will let you run processes similar to "scheduled tasks" inside of ASP.NET without needing to use windows services or any type of cron jobs.
Scheduled Tasks in ASP.NET!
I very much prefer WCF service to scheduled tasks. You might (off the top of my head) pass an addr to the WCF service as a sort of 'callback' that the service can call with progress reports as it works.
I'd shy away from scheduled tasks... too course grained.
I have a aspx web application that updates or adds files in a database. The clients access through the browser and one of the requirements is that they can start the update and be able to close the browser while the update continues. It appears to run for a little bit after I close the browser but then it stops. How can you keep the application running for asp.net?
That's something you could very well solve with WF (Workflow Foundation). Create a workflow for the task that should survive closing the browser. Workflows have their own threads and livecycles separate from ASP.NET.
The web application will keep running in the application pool, but this will be recycled eventually. As long as the users session runs the application should be kept alive, so by upping the session timeout you may fix the problem.
A better approach though would be to move the long-running task into a service instead, but that may require a rewrite of your application.
Usually for long-running or asynchronous processing, you want to dispatch the request to a back-end service to handle. Trying to keep the web-app alive to finish processing can lead to problems, especially with HTTP and session timeouts.
A common pattern for this is to put the request on a message queue and let a back-end service process it when it can.
I would create a separate windows service that you can push jobs onto from your web application, then check the status of the job(s) when the user logs in again.
The windows service won't be tied to the asp.net app domain so it will continue to run regardless of whats happening in your web application.
I've run into this pattern and you have to decouple the work from the HTTP request. The way we've solved it is to abstract the computing to be done as an event to be scheduled. So, say a user at a browser takes an action that requires a long lived (relatively) computation on the back end, this computation is given a name like 'doXYZForUser' and given a prameter vector like (userId, params...) and sent off to the work queue. Some time in the future the user logs in again and can see what the status of their job is.
I'm running a Java stack and a Java Message Service (JMS) but the principle is the same. The request from the browser queues up an event and the browser get an ACK back saying the event is on the work queue. The queue is managed by an entirely separately running process which in .NET I believe is just called the Message Queue. The job comes up on the queue gets processed and the results can be placed in a separate table containing a reference to the user that kicked off the job, so the next time they log in job status/results can be returned.