I have an ASP.Net website that has custom internal threads, for periodically occurring tasks.
If I get an exception on one of these threads, it is not caught in Global.ASAX's Application_Error() function. It is allowed to bubble up to IIS and I find out about it by reviewing the Event Viewer logs. If I catch the exception then Log4Net will direct an email to me and I should find out about the error relatively quickly.
Is there a way I can trap exceptions on these threads? The app needs to be 'always-on', so an exception that drops the application is a show-stopper.
In a comment you mentioned:
This is a web-site rather than web app.
"Web site" vs. "web app" seems like a moot distinction at this point. There's enough complexity in the code that it's an "application" by pretty much any definition of the word. To that point, if the application host doesn't meaningfully manage thread faults for you (and I wouldn't expect a web application host to do so) then you have to manage them manually.
In this case I see that as one of two options:
Option 1: Don't let your threads end in a faulted state. Whatever your top-level worker item for any given thread is (a method invoked at the start of the thread, a loop repeating operations, etc.), that needs to be essentially fault-proof. No exception should get past that. Which means it needs to be dead simple (so as to not throw exceptions of its own) and needs to catch any and all exceptions from the operation(s) it invokes.
Once caught, do with them as you please. Roll back a unit of work, notify someone of the error, etc.
Option 2: Move the long-running thread operations out of the web application, since web applications aren't really suited for ongoing background processes. A Windows Service or scheduled Console Application is a much more suited application host for that logic.
Yes, bit of a re-write there though.
Is it? It shouldn't be. That's really a matter of how the code was originally architected, not related to the application hosts themselves. Invoking a business operation from one application host is the same as invoking it from another. If the logic is tightly coupled to the application technology, that's a separate problem. And there's no quick fix to that problem. The good news is that once you fix that problem, other problems (like the one which prompted this question) are quick fixes.
Related
Does it cause all the threads,I mean all the users to stop and increase the wait time http request queue and start to affect the availability of the application ?
If so should we make sure there is no exception left over in an asp.net application to ensure scalability of the application.
Some unhandled exceptions can terminate w3wp.exe. Therefore, you should handle exceptions. Application_Error can't handle which are thrown on another thread (background workers, fire and forget taks). You should use http module for these kind of errors. But it also can't catch stackoverflow erros. You should use some decarations for this, if you want handle exceptions on Application level. Here is more detail.
Other than this I think exceptions will not effect the other users. Exception and performance.
Basically, exceptions shouldn't happen often unless you've got
significant correctness issues, and if you've got significant
correctness issues then performance isn't the biggest problem you
face.
I liked this sentence :)
Does it cause all the threads,I mean all the users to stop and
increase the wait time http request queue and start to affect the
availability of the application
It depends on the exception. For example, OutOfMemoryException will affect all users.
On the other hands, FileNotFoundException wouldn't affect other user.
If so should we make sure there is no exception left over in an
asp.net application to ensure scalability of the application.
Well, we all developers try to write bug free code, but sometimes things slip out of our hand. It is why we use logging to catch run-time exceptions - such as Log4Net, NLog.
I created a service with a RESTful API in ASP.NET, hosted in IIS. Inside this service, I would like to create an actor system with Akka.NET.
Upon creating the actor system:
var actorSystem = ActorSystem.Create("myActorSystem");
The following exception is thrown:
A first chance exception of type 'System.InvalidOperationException' occurred in System.Web.dll
Additional information: An asynchronous operation cannot be started at this time. Asynchronous operations may only be started within an asynchronous handler or module or during certain events in the Page lifecycle. If this exception occurred while executing a Page, ensure that the Page is marked <%# Page Async="true" %>. This exception may also indicate an attempt to call an "async void" method, which is generally unsupported within ASP.NET request processing. Instead, the asynchronous method should return a Task, and the caller should await it.
The actor system is inherently a concurrent system with asynchronous messages being exchanged between actors. As explained here, this actor system would not survive IIS taking down the AppDomain, which is probably why the aforementioned exception is thrown.
This article explains how to run background tasks in ASP.NET. However, I don't see how I could use this for my actor system, as I have no control over the lifecycle of background tasks that might be created by Akka.NET.
Is there a way to make this work, or should I abandon the idea of having an actor system in an ASP.NET application?
EDIT: I also saw a question on Stackoverflow about implementing a REST service using Akka. Any advice about a solution similar to the Spray toolkit, but working for Akka.NET would be welcome.
I've used Akka.NET and Akka.Remote inside ASP.NET MVC applications that are doing up to 1000 requests per second on EC2 - so I'll share some of the tips and tricks I used to get it up and running successfully. Had a prototype version that even used Akka.Cluster but ended up not shipping that version.
Best place to call ActorSystem.Create is inside Global.asax Application_Start().
Hang onto a static reference to the ActorSystem object inside Global.asax itself, using a static field or property. Helps ensure that the ActorSystem itself doesn't get garbage-collected in long-running applications.
Create a separate static helper class to initialize any top-level actors your applications needs - i.e. actors at the top of the /user/ hierarchy. This class should also provide actor paths that your ASP.MVC controllers and action methods can use for Tell and Ask operations.
Creating the ActorSystem is a bit of an expensive operation, because lots of system-level stuff gets fired up at once. It's definitely best to do this once at application startup and then just cache the result inside the Application class.
Creating individual actor instances is cheap - you should be able to do this no-problem inside ASP.NET MVC action methods. If you see this error come up again, please let us know what part in the request-handling process this error occurred and with which version of ASP.NET.
Edit: added some updated guidance for how to do this on ASP.NET Core
https://petabridge.com/blog/akkadotnet-aspnetcore/
Keep your ActorSystem as a shared property in some static class container - this way you may access it from the rest of your application. Actor system initialization/disposal can be done by:
Global.asax - use ActorSystem.Create(...) inside Global.asax Application_Start and dispose it with system.Shutdown() on Application_End.
OWIN - create actor system in OWIN's Startup.Configuration method and shut it down by binding to host.OnAppDisposing event (how-to link).
Remember that IIS will startup your web app only after first request and tear it down automatically after some time when it's idle. Therefore make sure, that your deployment script will ping application after publishing and set idle timeout (link) for long enough if you want your Akka actor system to run continuously.
Second option
Separate your Actor System logic and deploy it, for example, as a Windows Service (or Linux deamon). Turn on Akka.Remoting for it and create a proxy client, which will forward all application long-running sensitive tasks to external service. Similar solution is often used for things such as schedulers or event buses, when your application logic must be working continuously.
In my 3.5 .net web application I have a background thread that does a lot of work (the application is similar to mint.com in that it does a lot of account aggregation on background threads). I do extensive exception handling within the thread performing the aggregation but there's always the chance an unhandled exception will be thrown and my entire application will die. I've read some articles about this topic but they all seem fairly outdated and none of them implement a standard approach. Is there a standard approach to this nowadays? Is there any nicer way to handle this in ASP.NET 4.0?
Arguably, the entire application should die if you have an unhandled exception. An unhandled exception means that your program is in an unknown/indeterminate state, and any further processing or user interaction could cause corruption of the program's state, or worse, data corruption.
You're doing the right thing handling exceptions within your thread work. As far as I know, there is no way for a .NET application to "gracefully" deal with unhandled exceptions on background threads - they will always terminate the process.
Certain .NET Framework classes, such as the BackgroundWorker component and the Task Parallel Library in .NET 4 make multithreading easier and handle a lot of the dirty work of exception handling for you, so if it's possible for you to use those instead of implementing your own multi-threaded code, then you should definitely do so. But if those aren't able to help you in a given circumstance, if you must use the ThreadPool or a pure Thread, then be sure not to let any unhandled exceptions escape.
You can always put a try/catch block around your worker thread at a very high level... like right when the thread starts. I'm assuming this is what you're doing already, or something like it. But just keep in mind that you definitely don't want to turn an unknown error into a silent unknown error, because then it's going to be much harder to track down when something goes wrong. Be sure you are logging the exception to the EventLog or your custom app log if you want to just catch it and forget it.
Like Aaronaught says, the application should die when something unexpected happens. But I don't see a problem with just letting your background thread exit/die instead of bringing down the whole application process (in fact, I don't think Aaron is correct here, it won't kill the entire process) I think your question can basically be translated as "is there something magical in ASP.NET that will suddenly solve issues I don't even know about yet?" and the answer to that of course, is no. But you already knew that. :)
I am getting started with ASP.NET web development and was wondering what the differences are when multi-threading a standard winforms application versus a web-based application written in asp.net that will run in IIS. Is there any difference and if so, what are the limitations (and conversely any positives) of threading a web application,
Thank you in advance
In the Windows GUI world, you always have the "UI thread" which you can use to communicate with the user. For example, you can start a BackgroundWorker in the UI thread, which will raise an event in the UI thread after completing its work: In a Windows UI application, you can be pretty sure that the UI thread is still there, unless the user has closed the application.
In the web world, you have no equivalent main thread. There are just web requests: Sometimes there are none and sometimes there are many at the same time; it is even possible that one single request will be handled by multiple threads without you knowing it. If you start a background thread with a lengthy operation, you need to either
delay finishing the web request until the background thread has completed -- which means a slow response time for the user and somehow defeats the purpose of a background thread or
regularly (during future requests) check the state of the thread and inform the user when it has finished.
Of course, that's only a problem if you want some user interaction after the thread has finished. If you don't, just start it and it will eventually finish (unless someone restarts IIS).
Threading isn't really an issue in the web world. There isn't any state and each request is a new instance of the web page. There are ways to track state between requests such as cookies, sessionstate, viewstate.
Recently, the book on threading for Winforms application (Concurrent programming on Windows by Joe Duffy) was released. This book, focused on winforms, is 1000 pages.
What gotchas are there in ASP.NET threading? I'm sure there are plenty of gotchas to be aware of when implementing threading in ASP.NET. What should I be aware of?
Thanks
Since each http request received by IIS is processed separately, on it's own thread anyway, the only issues you should have is if you kick off some long running process from within the scope of a single http request. In that case, I would put such code into a separate referenced dependant assembly, coded like a middle-tier component, with no dependance or coupling to the ASP.Net model at all, and handle whatever concurrency issues arose within that assembly separately, without worrying about the ASP.Net model at all...
Jeff Richter over at Wintellect has a library called PowerThreading. It is very useful if you are developing applications on .NET. => Power Threading Library
Check for his presentations online at various events.
Usually you are encouraged to use the thread pool in .Net because it of the many benefits of having things managed on your behalf.....but NOT in ASP.net.
Since ASP.net is already multi-threaded, it uses the thread pool to serve requests that are mapped to the ASP.net ISAPI filter, and since the thread pool is fixed in size, by using it you are basically taking threads away that are set aside to do the job of handling request.
In small, low-traffic websites, this is not an issue, but in larger, high-traffic websites you end up competing for and consuming threads that the ASP.net process relies on.
If you want to use threading, it is fine to do something like....
Thread thread = new Thread(threadStarter);
thread.IsBackground = true;
thread.Start();
but with a warning: be sure that the IsBackground is set to true because if it isn't the thread exists in the foreground and will likely prevent the IIS worker process from recycling or restarting.
First, are you talking about asynchronous ASP.NET? Or using the ThreadPool/spinning up your own threads?
If you aren't talking about asynchronous ASP.NET, the main question to answer is: what work would you be doing in the other threads and would the work be specific to a request/response cycle, or is it more about processing global tasks in the background?
EDIT
If you need to handle concurrent operations (a better term than multi-threaded IMO) for a given request/response cycle, then use the asynchronous features of ASP.NET. These provide an abstraction over IIS's support for concurrency, allowing the server to process other requests while the current request is waiting for work to complete.
For background processing of global tasks, I would not use ASP.NET at all. You should assume that IIS will recycle your AppPool at a random point in time. You also should not assume that IIS will run your AppPool on any sort of schedule. Any important background processing should be done outside of IIS, either as a scheduled task or a Windows Service. The approach I usually take is to have a Windows Service and a shared work-queue where the web-site can post work items. The queue can be a database table, a reliable message-based queue (MSMQ, etc), files on the file system, etc.
The immediate thing that comes to mind is, why would you "implement threading" in ASP.NET.
You do need to be conscious all the time that ASP.NET is multi-threaded since many requests can be processed simulatenously each in its own thread. So for example use of static fields needs to take threading into account.
However its rare that you would want to spin up a new thread in code yourself.
As far as the usual winforms issues with threading in the UI is concerned these issues are not present in ASP.NET. There is no window based message pump to worry about.
It is possible to create asynchronous pages in ASP.NET. These will perform all steps up to a certain point. These steps will include asynchronously fetching data, for instance. When all the asynchronous tasks have completed, the remainder of the page lifecycle will execute. In the meantime, a worker thread was not tied up waiting for database I/O to complete.
In this model, all extra threads are executing while the request, and the page instance, and all the controls, still exist. You have to be careful when starting your own threads, that, by the time the thread executes, it's possible that the request, page instance, and controls will have been Disposed.
Also, as usual, be certain that multiple threads will actually improve performance. Often, additional threads will make things worse.
The gotchas are pretty much the same as in any multithreaded application.
The classes involved in processing a request (Page, Controls, HttpContext.Current, ...) are specific to that request so don't need any special handling.
Similarly for any classes you instantiate as local variables or fields within these classes, and for access to Session.
But, as usual, you need to synchronize access to shared resources such as:
Static (C#) / Shared(VB.NET) references.
Singletons
External resources such as the file system
... etc...
I've seen threading bugs too often in ASP.NET apps, e.g. a singleton being used by multiple concurrent requests without synchronization, resulting in user A seeing user B's data.