I'll try to be brief.
What is the best practice for calling a routine from an asp.net web application that initiates a lengthy 'background' process that must be run?
For example, I want to click a button on my webpage that says "Run data conversion" (for example). This data conversion routine may take 20-40 minutes to run, so it seems to me putting all that code into a asp.net web page is not the way to go....there is no need to run this background process thru the IIS server. A sperate service or app etc seemx the way to go...
The web app, and the background process will both run on my dedicated Win2003 server so I have lots of options - but what is the best one?
If you control the server, I would suggest creating a windows service - take the data conversion task outside of ASP.NET altogether; you wouldn't want ASP.NET restarting halfway through a 40 minutes conversion routine.
There are quite a few options, basically it boils down to you needing any sort of process that you can communicate with. The options I can think of off the top of my head are; a web-service running under another application pool; a windows service; a command-line process started by your ASP.NET code.
The next question is how to communicate with the other process. If you're using a database you could set up two common-tables that both processes can access. The website would place the request for work into a table that the second process would monitor. A second table could then be used for the results, which the web-site would monitor. Another option would be to use something like the Windows Communication Foundation (WCF) or .NET remoting to send events between the processes.
Related
We are referencing a 3rd party proprietary CLI DLL in our .net project. This DLL is only an interface to their proprietary C++ library. Our project is an asp.net (MVC4/Web API) web application.
The C++ unmanaged library is rather unstable. Sometimes it crashes with e.g. dangling pointers. We have no way of solving it, and using this library is a first-class customer requirement.
When the application crashes, the application pool in IIS doesn't respond anymore. We have to restart it, and doing so takes a couple minutes (yes, that long!).
We would like to keep this unstable DLL from crashing our application. What's the best way of doing it? Can we keep the CLI DLL in a separate AppDomain? How?
Thanks in advance.
I think every answer to this question will be some kind of work around.
My workaround would be to not interact directly with the DLL from your web application.
Instead write your requests from the web application to either a Message Queue or a SQL table. You can then have another application such as a Windows Service which reads the requests, interacts with the DLL and then writes the results back for your web application to read.
I'm not saying that SQL / Message Queues are the right way, I'm more thinking of the general process flow.
I had this exact problem with a third party library that accessed protected memory for purposes of interacting with a hardware copy protection dongle. It worked fine in a console or winforms app, but crashed like crazy when called from an IIS application.
We tried several different things, some of which are mentioned in other answers on this page. But ultimately, the best solution for us was to us a very old technology - .Net Remoting. I know - it's somewhat frowned on these days. But it fit this particular need quite well.
The unstable code was placed in a Windows Service application. The web application made remoting calls to this service, which relayed the commands to the third-party library.
Now I'm sure you could do the same thing with WCF, sockets, etc. But remoting was quick and easy to setup, and since we only talk to the same server it works without opening any ports. It just talks on a named pipe.
It does mean a second service to install besides the web application, but that was acceptable in my particular use case.
If you did something similar, and the third-party code actually crashed the service, you could probably write some code in your main application to bring it back up.
So perhaps a process boundary is more useful than an App Domain when you have unstable code to wrangle.
I would first increase the IIS process recyling rate, maybe the the DLL code fails after a certain number of calls, or after the process reaches a certain amount of memory usage.
You can find information on the configuration of IIS 7.0 recycling options here: http://technet.microsoft.com/en-us/library/cc753179(v=ws.10).aspx
In your case I would recycle the process at a specific time, when you know there is less load on the application. And after a certain number of requests (lower than the default) to try and have "fresh" process most of the time.
The recycling process is graceful in the sense that the the old process is not terminated until the one that will replace it is ready, so there should be no noticeable downtime.
More information about the recycling mechanism here: http://technet.microsoft.com/en-us/library/cc745955.aspx
If the above does not solve the problem I would wrap the calls in my own code that manages the unstable DLL execution.
This code should recover from the failures for example by repeating the failing calls until a result is obtained and failing with a graceful error if it is not possible after a number of attempts.
Internally the calls to the unstable DLL could be made in a spawned thread or even the code could be in an new external executable that you could launch with Process.Start.
This last option has more overhead but it might be your only option. See this SO question for more information on this: How do you handle a thread that has a hung call?
I suggest following solution.
Wrap this dll with another web application. Can be one of the following ones. Since you already use web api, it is most suitable for you.
Simple ASMX Web Service
WCF Service
Asp.Net MVC - WEB Api Service
Control your p-invoke code so that you do not have any bug? See following articles.
The Black Art of P/Invoke and Marshaling in .NET
P/Invoke Revisited
Publish this application to IIS with different application pool.
Use standard techniques suggested before like. I suggest configure recycling IIS for both memory and scheduled times.
IIS process recycling rate
How to limit the memory used by an application in IIS?
I have web app code in VS2010 that is manually executed every night. One of the developers manually runs the code in VS, when the web page opens, presses a few buttons etc.. and executes the code to get our required results. How can we automate this process so as to eliminate any human element. Ideally, I am looking for a way to have the code execute automatically at a given time during the day. What is involved in getting something like this to work?
A WCF service is a possible solution:
http://msdn.microsoft.com/en-us/library/ms734712.aspx
Windows has scheduled tasks which is good at... scheduling tasks.
Do you (or plan to) have a big test of GUI tests? There are entire tools dedicated to GUI automation testing. I'd recommend looking into one of those if this is going to be a big part of your overall test strategy.
If this is a one-time thing, you could schedule the running of a simple C# application that hosts a Web Browser control, that points to your web site. In this sense, the Web Browser control acts as your browser. You can send JavaScript commands to it etc.
Web Browser documentation:
http://msdn.microsoft.com/en-us/library/system.windows.forms.webbrowser.aspx
Another alternative is to schedule an AutoHotKey script to simulate the key pressing and mouse clicks. This works well. I used to use it to rack up Farmville points (no joke).
http://www.autohotkey.com/
A Windows service seems better in the sense of "scheduled" and "no intervention." A WCF service is still, by default, listening and waiting for interaction (hosting the service). Or just schedule a simple app or script to do what you need.
Can you tell us a little more about this process? My initial recommendation would be to transition that code from a web application into a command line utility and then use windows task scheduler. If for some reason there are heavy dependencies within that web app that make that impossible I would consider taking those button click events and turning them into web services that you could then call programmatically from a command line application.
I'm developing a .NET 4 application that requires a backend worker thread to be running. This thread consists mostly of the following code:
while (true) {
//Check stuff in database
//Do stuff
//write to database / filesystem
Thread.sleep(60000)
}
The ASP.NET app is just a frontend for the database.
My question is around where the best place to put this worker loop would be. It seems my immediate two choices would be (1) to spin it off from the Application_Start method, and just let it run, or (2) bundle it in a separate process (Windows service?)
(1) would obviously need some logic in the ASP.NET code to check it's still running, as IIS might kill it. It's also quite neat in that the whole application logic is in one, easily deployable package.
(2) is much more segregated, but feels a lot messier.
What's the best approach?
I would strongly opt for the Windows Service if possible. Background threading in ASP.NET comes with a lot of baggage.
The lifetime of your background process is at the mercy of IIS. If IIS decides its time to recycle the App Pool, your background process will restart. If IIS decides to stop the App Pool due to inactivity, your background process will not run.
If IIS is configured to run as a Web Garden (multiple processes per AppPool), then your background thread could run more than once.
Later on, if you decide to load balance your website (multiple servers running the site), then you may have to change your application to ensure the background threading is only happening on one server).
And plenty more.
Consider something simple like Hangfire and then think about the design points in this related answer.
I' am building some RSS web service in ASP.net (using IIS as the web server). In it I wand to create some king of RSS reader.
I 'am creating some process that will retrieve the content from the RSS feed every 3 hours.
I want to create a control panel that will give me the ability to start/stop the process, and will have some simple dashboard that will sum the current activity.
I 'am looking for the best way to do that.
I thought about creating a Windows Service on the server, but there are security issues in starting and stopping the service from a web interface.
What is the right way to do it?
you could create your service like this
https://blog.stackoverflow.com/2008/07/easy-background-tasks-in-aspnet/
the current state (start/stop) could be set with a static variable (which is the same for the whole AppDomain) or from a config file (which would be better because it survives a AppDomain restart)
A windows service is the right way to perform scheduled tasks on a web server, no question, but I'd argue that a web-based control panel isn't really necessary. Can't you just RDP to the server when you want to start / stop / restart the service?
I am writing a web application in ASP.NET 3.5 that takes care of some basic data entry scenarios. There is also a component to the application that needs to continuously poll some data and perform actions based on business logic.
What is the best way to implement the "polling" component? It needs to run and check the data every couple of minutes or so.
I have seen a couple of different options in the past:
The web application starts a background thread that will always run while the web application does. (The implementation I saw started the thread in the Application_Start event.)
Create a windows service that is always running
What are the benefits to either of these options? Are there additional options?
I am leaning toward a windows service because it is separated and can run on a different server (more scalable) as well as there is more control over when it is started/stopped, etc. However, I feel like the compactness of having the "background" logic running in the process of the web application might make the entire solution more understandable.
I'd go for the separate Windows service primarily for the reasons you give:
You can run it on a different server if necessary.
You can start and stop it independently of the web site.
I'd also add that it could well have some impact on the performance of the web site itself - something you want to avoid.
The buzz-word here is "separation of concerns". The web site is concerned with presenting the data to the user, the service with checking the integrity of the data.
You can also update the web site and service independently of each other should you need to.
I was going to suggest that you look at a scheduled task and let Windows control when the process runs, but I re-read your question and noted that you wanted the checks to run every couple of minutes. The overhead of starting the process might be too great in this case - though some experimentation would probably prove this one way or the other.
If you use a scheduled task there's also the possibility that you could start the next check before the current one has finished - something you can code for if you're in complete control.
Why not just use a console app that has no ui? Can do all that the windows service can and is much easier to debug and maintain. I would not do a windows service unless you absolutely have to.
You might find that the SQL Server job scheduler sufficient for what you want.
Console application does not do well in this case. I wrote a TAPI application which has to stay in the background and intercept incoming calls. But it did it only once because the tapi manager got GCed and was never available for the second incoming call.