We are referencing a 3rd party proprietary CLI DLL in our .net project. This DLL is only an interface to their proprietary C++ library. Our project is an asp.net (MVC4/Web API) web application.
The C++ unmanaged library is rather unstable. Sometimes it crashes with e.g. dangling pointers. We have no way of solving it, and using this library is a first-class customer requirement.
When the application crashes, the application pool in IIS doesn't respond anymore. We have to restart it, and doing so takes a couple minutes (yes, that long!).
We would like to keep this unstable DLL from crashing our application. What's the best way of doing it? Can we keep the CLI DLL in a separate AppDomain? How?
Thanks in advance.
I think every answer to this question will be some kind of work around.
My workaround would be to not interact directly with the DLL from your web application.
Instead write your requests from the web application to either a Message Queue or a SQL table. You can then have another application such as a Windows Service which reads the requests, interacts with the DLL and then writes the results back for your web application to read.
I'm not saying that SQL / Message Queues are the right way, I'm more thinking of the general process flow.
I had this exact problem with a third party library that accessed protected memory for purposes of interacting with a hardware copy protection dongle. It worked fine in a console or winforms app, but crashed like crazy when called from an IIS application.
We tried several different things, some of which are mentioned in other answers on this page. But ultimately, the best solution for us was to us a very old technology - .Net Remoting. I know - it's somewhat frowned on these days. But it fit this particular need quite well.
The unstable code was placed in a Windows Service application. The web application made remoting calls to this service, which relayed the commands to the third-party library.
Now I'm sure you could do the same thing with WCF, sockets, etc. But remoting was quick and easy to setup, and since we only talk to the same server it works without opening any ports. It just talks on a named pipe.
It does mean a second service to install besides the web application, but that was acceptable in my particular use case.
If you did something similar, and the third-party code actually crashed the service, you could probably write some code in your main application to bring it back up.
So perhaps a process boundary is more useful than an App Domain when you have unstable code to wrangle.
I would first increase the IIS process recyling rate, maybe the the DLL code fails after a certain number of calls, or after the process reaches a certain amount of memory usage.
You can find information on the configuration of IIS 7.0 recycling options here: http://technet.microsoft.com/en-us/library/cc753179(v=ws.10).aspx
In your case I would recycle the process at a specific time, when you know there is less load on the application. And after a certain number of requests (lower than the default) to try and have "fresh" process most of the time.
The recycling process is graceful in the sense that the the old process is not terminated until the one that will replace it is ready, so there should be no noticeable downtime.
More information about the recycling mechanism here: http://technet.microsoft.com/en-us/library/cc745955.aspx
If the above does not solve the problem I would wrap the calls in my own code that manages the unstable DLL execution.
This code should recover from the failures for example by repeating the failing calls until a result is obtained and failing with a graceful error if it is not possible after a number of attempts.
Internally the calls to the unstable DLL could be made in a spawned thread or even the code could be in an new external executable that you could launch with Process.Start.
This last option has more overhead but it might be your only option. See this SO question for more information on this: How do you handle a thread that has a hung call?
I suggest following solution.
Wrap this dll with another web application. Can be one of the following ones. Since you already use web api, it is most suitable for you.
Simple ASMX Web Service
WCF Service
Asp.Net MVC - WEB Api Service
Control your p-invoke code so that you do not have any bug? See following articles.
The Black Art of P/Invoke and Marshaling in .NET
P/Invoke Revisited
Publish this application to IIS with different application pool.
Use standard techniques suggested before like. I suggest configure recycling IIS for both memory and scheduled times.
IIS process recycling rate
How to limit the memory used by an application in IIS?
Related
I have to see if I can work around a known thread safety issue on third party component. The plan was to let an ASP.Net app to talk to the third party component via a WCF service, this was based on the assumption that I will be able to assign the WCF service to it's own application pool, restrict the pool to one working process and the working process to one thread. Requests to the service will have to wait for their turn, but that's OK because we expect them to require very little time and to be rare as well.
Problem is, I can't find anything that suggests how to achieve this part: "restrict the working process to one thread".
I have found a few useful pages, but no solution:
On IISforums, the discussion seem to suggest that I can achieve this in IIS 7+ but does not mention IIS6.
On MSDN Blogs and linked MSDN books chapter (can't put the link because I'm new here!) the discussion is mostly about what you can set via the machine.config file, which, if I'm understanding it correctly, is going to apply to all AppPools/Worker processes and is not what I'm trying to do, I would like to control what one single app can do, leaving the other applications untouched.
Questions:
Is it possible to achieve what I'm trying to do? (assign the WCF service to it's own application pool, restrict the pool to one working process and the working process to one thread" via IIS 6 configurations)
If not, can this be achieved programmatically somehow? For example, using locks or other threading-related tricks within my WCF service implementation.
It doesn't make much sense to me.. A web server have to be multithreaded by definition, because it must handle different income messages at the same time, if there is only one thread once in use any new request will fail.
What about to wrap the component in a class with a SynchonizationAttribute, so only one thread can access the component? Even if this will make your solution less scalable, at least it may work
http://msdn.microsoft.com/en-us/library/system.runtime.remoting.contexts.synchronizationattribute(v=vs.110).aspx
I'm porting a cpu-heavy .net 4.0 windows application to a .net 4.0 wcf service. Basically I just imported the .net classes to the wcf service.
All is working well except for performance at the wcf service - a task that takes 6267947 ticks (2539ms) uses 815349861 ticks (13045ms) on the aspx.net wcf service running locally on the same develop machine.
I allready have uploaded the service + a test client to appharbor where the performance is as bad as on my local machine - the link to my test app is: http://www.wsolver.com/. Any ideas on how I can improve performance?
Check any dependencies on your service that may be constructed at Request Time. These Include constructor dependencies and field/property dependencies. Maybe one of them is causing the delay? If this is the case consider using a singleton to instantiate the long running class.
Have you confirmed that subsequent requests still cause the delay?
Also create a band new service that does something simple like Datetime.Now.toString() and see if it has the same problem.
Please take a look at the articles and whitepapers below. I think they should give you enough concrete performance considerations to explore, and likely some very practical settings to tweak, optimize, or change.
Performance Tuning WCF Services
Optimizing WCF Web Service Performance
Using ServiceThrottlingBehavior to Control WCF Service Performance
Transport Quotas
Optimizing IIS Performance
ASP.NET Performance Overview
A Performance Comparison of Windows Communication Foundation (WCF) with Existing Distributed Communication Technologies
If you need to do time-consuming initialization of a complex datastructure, you should to that once in Application_Start() and assign the generated datastructure to a static variable on the MvcApplication object. Doing it just once on application start is going to be much faster that doing it in each request.
I would take a full memory dump during the 13 seconds (or several using procdump) and then acutally look at what is occurring in the process (windbg and sos.dll). Then, you can narrow down which code is the culprit.
I take it that the dictionary tree is only loaded once, into cache? You're not loading it on every call are you?
Throughout the process of developing my Application, the first-response time has got worse and worse, It is now taking 10 minutes to load! I am using Web-Deploy to speed up publishing my changes, and from what i've read on MSDN, i understand that this delay is due to compilation and loading assemblies.
It's an ASP.NET MVC3 Application which uses EF CodeFirst, MVC-MiniProfiler etc. Im wondering if its one of these assemblies that is slowing things down.
Is there a way to track down the long running process plaguing my development/testing process?
As a side note, the issue is nowhere near as bad in the Azure Emulator.
Using Windows Azure SDK 1.4 and later, you have the option to enable Profiling for you application (beside the IntelliTrace). You can read about some of the options available (in 1.5) from my blog post here where you will also find a good screenshot showing the option to enable either IntelliTrace or Profiling.
The trick is that you can only have one of them running (either ItelliTrace or Profiling). So I suggest you first run the ItelliTrace and inspect ItelliTrace logs for any exceptions during your application execution. Then do another deployment using Profiling to catch which are the most time consuming methods.
Please note that enabling IntelliTrace /Profiling is only accomplishable during deployment process, and cannot be changed with simple WebDeploy, so you'll have to make at least two deployments for test.
It's hard to say what the slowdown is - as Awais mentioned, IntelliTrace is your friend. However, the delay might be unavoidable (I have seen this a number of times). If this is the case, you can add a startup script that will "prime" IIS, preventing the problem when the first user hits the site.
I am writing a web application in ASP.NET 3.5 that takes care of some basic data entry scenarios. There is also a component to the application that needs to continuously poll some data and perform actions based on business logic.
What is the best way to implement the "polling" component? It needs to run and check the data every couple of minutes or so.
I have seen a couple of different options in the past:
The web application starts a background thread that will always run while the web application does. (The implementation I saw started the thread in the Application_Start event.)
Create a windows service that is always running
What are the benefits to either of these options? Are there additional options?
I am leaning toward a windows service because it is separated and can run on a different server (more scalable) as well as there is more control over when it is started/stopped, etc. However, I feel like the compactness of having the "background" logic running in the process of the web application might make the entire solution more understandable.
I'd go for the separate Windows service primarily for the reasons you give:
You can run it on a different server if necessary.
You can start and stop it independently of the web site.
I'd also add that it could well have some impact on the performance of the web site itself - something you want to avoid.
The buzz-word here is "separation of concerns". The web site is concerned with presenting the data to the user, the service with checking the integrity of the data.
You can also update the web site and service independently of each other should you need to.
I was going to suggest that you look at a scheduled task and let Windows control when the process runs, but I re-read your question and noted that you wanted the checks to run every couple of minutes. The overhead of starting the process might be too great in this case - though some experimentation would probably prove this one way or the other.
If you use a scheduled task there's also the possibility that you could start the next check before the current one has finished - something you can code for if you're in complete control.
Why not just use a console app that has no ui? Can do all that the windows service can and is much easier to debug and maintain. I would not do a windows service unless you absolutely have to.
You might find that the SQL Server job scheduler sufficient for what you want.
Console application does not do well in this case. I wrote a TAPI application which has to stay in the background and intercept incoming calls. But it did it only once because the tapi manager got GCed and was never available for the second incoming call.
I want to add a scheduled task to a client's ASP.NET app. These posts cover the idea well:
https://blog.stackoverflow.com/2008/07/easy-background-tasks-in-aspnet/
What is the Best Practice to Kick-off Maintenance Process on ASP.NET
"Out of Band" Processing Techiniques for asp.net applications
My question has two parts: First, will IIS unload the application if there isn't enough request activity despite the Cache activity? My client doesn't enjoy as much traffic as stackoverflow so they can't rely on user requests to keep the app 'active'. Obviously, I can't schedule tasks in an unloaded app.
Second, if so, is there a way to prevent IIS from unloading the app outside of configuration or external 'stay-alive' requests? My client's host doesn't allow much configuration tweaking and a stay-alive utility introduces the deployment complexity I'm trying to avoid with an ASP.NET Cache solution.
Thanks a bunch.
Edit/Conclusion: TheXenocide's solution is exactly correct given the question. However, I've decided it is a really bad question. The temptation to cut corners is always looming. I've regained my senses and told my client to use a website monitoring tool to keep the site active. In addition, the scheduled task is going in a windows service despite the extra deployment hassle.
Unfortunately, outside the range of changing timeout configuration (which I believe to be possible in Web.config, though I don't know what is and isn't allowed on hosting providers, most of which use Medium Trust) I don't believe there is any other method to keep the application from ending beyond web requests. One thing you might try that may be a little more simple than using some keep-alive service on a local machine might be to add some logic to Session_Start/Session_End that ensures there is always at least one session active; you can use the WebRequest class from within your application to call your own site and it should still start a new session.
Good luck, and let us know what you do :)
UPDATE: these details now very much depend on which version of IIS and which version of .NET you're running in. Newer versions of each have methods of configuring "always running" applications.