I have a ongoing issue with my website, where it appears the W3wp.exe crashes and in turn anyone attempting to connect to the site will receive a SQL server timeout error.
The error is this...
System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
What actually happens is when this occurs, the CPU usage on the server maxes out to 100% and stays there. During this time if I try and connect to the homepage the site will appear to be loading and then timeout. This is also the case when i try and connect to the database through management studio.
I get the above SQL timeout error from ELMAH when it emails me exceptions. When I do a iisreset I can once again connect to SQL server and the CPU usage drops immediately.
Now I have already posted a few questions oh here about the issue in regards to the SQL time out exception. So ill list what I have done and what im using...
• Im using Linq to SQL.
• SQLCacheDependancy.
• IIS 7.5 & SQL Server 2008 R2.
• Disposing of all objects.
• The call to the Linq to SQL dataContext is always wrapped in a USING Keyword.
These timeout errors occur infrequently, maybe once every 2 days, but today when I got up, my inbox was full of time out exceptions indicating one occurred overnight.
The other odd thing is that when for example it happened last night, it was around 4am, when I tried it this morning at 8am the site worked fine, so I appears that overtime the W3wp.exe, SQL server or both manage to either recycle or reset themselves.
Can anyone see what maybe causing this behaviour ? or point me in the right direction of what maybe the cause ? or perhaps even a possible solution ?
Any help is most appreciated
Truegilly
I would guess the timeout is because of the high CPU spike, not the other way round. If your IIS and SQL are on the same machine, try to find out the process (from task manager) which is hogging the processor.
If it is the web server, setup perform counters, especially the ones related to .NET Garbage collection, and see how frequently it is getting executed. Also check the Gen2 object size. You would have to collect this data starting from when it is running fine till CPU maxes out, and compare what is "growing" in the counters. Check if you have object tracking enabled in the data context, and also how long your using(datacontext){ ... } are.
If it is the sqlserver that is holding the CPU, then try to use the following Microsoft article to analyze the issue.
Related
My asp.net application crashes sometimes in live server while we are working on it. All users are facing [Exceptions]yellow error screen.And Throws dozens of exception in exception log.
[Get] find 0 Table rows
This exception occurs on randomly not on specific location or specific click.
But when I restart my application on IIS the application working fine. but again some hours same problem appears.
When I try to Re-login Than application don't find my credentials. but after restart IIS. I successfully login in to application with same credentials.
Summary: not find the specific reason of crashes. Working fine when i restart IIS.
drop down list are not find data from datasource
Grid table find 0 rows when access to table.
when same grid and drop down list is access after IIS restart it has data.
Note: its feels that we lost connection from database. but actually it
has connection but find no data in it
Try to catch exception and check server response time
After a lot of search i find the depth of the problem and the problem is memory leak.
Memory leaks in .NET applications have always been a programmer’s
nightmare. Memory leaks is the biggest problems when it comes to
production servers. Productions servers normally need to run with the
least down time. Memory leaks grow slowly and after sometime they
bring down the server by consuming huge chunks of memory. Most of the
time people reboot the system, make it work temporarily, and send a
sorry note to the customer for the downtime
some use full links are:
link 1
link 2
The proper solution of the problem is very time consuming and need a skill full person.
Solution for temporary basis is when you application is down you need to reset ISS programatically.
Process.Start(#"C:\WINDOWS\system32\iisreset.exe", "/noforce");
or
Process p = new Process(); p.StartInfo = new
ProcessStartInfo("iisreset.exe"); p.Start();
I know this is not a good solution but this can put the pressure off from your head.
For proper solution check this link
I am loading 100,000 plus pieces of data, which should take like 30 to 40 minutes to analyze and complete. We use SQL Server 2005. Currently after a specified time (trying to get the minutes it uses), the SQL Server says the connection was reset.
Is there some setting where this can be changed so my queries can run for an hour?
SqlCommand.CommandTimeout property will let you set it in code, however unless you've got clever you could get a webserver timeout as well.
Connection reset could even be your network boys, killing it for no activity.
I'd be looking at reorgansiing this somehow. Like a request to do the work and kick off athread to do it. A status page to see where it's at. When finished provides a link to the result, or somesuch.
May be your problem is related to Worker Process(w3wp.exe). In web application After some time the worker process will be recycle. So please check this setting of "IIS". you can also reset the the timeout of "Connection Timeout" in web.config or change "CommandTimeout" in code.
Hi i got a web application(asp.net) where we just started getting "System.InvalidOperationException: Timeout expired." when trying to get a new sql connection.
So my guess some where in the code a connection is created but never disposed, but how would i go about to find where this happens? sadly most of the database communication does not use a datalayer it works directly with the sql data types...
Is there any thing i could enable in the web.config to trace what connections are open for longer then x seconds and where they where opened?
find everywhere your sqlconnection is used. Ensure it is in a using() block to automatically dispose of it. There is nothing built in to the web.config with this unfortunately. You may be able to try out ants memory profiler to help track this down. for ex:
http://www.developerfusion.com/review/59466/ants-memory-profiler-51/
Ok i found a way to track them down by using
EXEC SP_WHO2
DBCC INPUTBUFFER(SPID)
SP_WHO2 gives you information about the connections and by using DBCC INPUTBUFFER you can find out what command they ran last time.
.NET framework will not solve your architecture and logging issues so you will have to find the problem yourselves. What can help you:
Performance counters can show you utilization of connection pool
SQL Server Management Studio can show you connections in use, their activity and last executed statement. In Object Explorer select the server and in context menu go to Reports > Standard Reports > Activity xxx
Another approach is turning off the pool and using some profiler to collect information about your application and track not disposed connections.
Greetings!
I have an ASP.NET app that scrapes data from a handful of external pages, parses the relevant bits and displays them in a table. Total data retrieved is 3-4MB and the resulting page is about 1MB. I am using synchronous WebRequest GetResponse for the retrieval, but the same problem existed using an asynchronous BeginGetResponse/EndGetResponse process.
There is no database access, no session storage, no caching, but an in-memory list of about 100 objects (total 1MB of data), plus a good amount of AJAX (AjaxControlToolkit). This issue appears on the very first run of the app, even if I have restarted IIS.
The issue:
When I run the app on my dev computer, the maximum commit charge is about 1.5GB. The biggest user, measured by Task Manager's VM Size, is WebDev.WebServer.exe (600MB). The app runs perfectly.
When I run it on my rent-a-server (IIS 7.5, 1GB RAM), the maximum commit charge is over 3.8GB. The biggest user is w3wp.exe at 2.7GB. IIS grinds to a halt and spits out a timed-out error page.
Given my limited server budget and the hope of having multiple simultaneous users, I'm kind of in a panic.
Is this normal? If I bump the server RAM up to 4GB, will that be enough?
Will multiple users require even more memory?
Could the culprit be AJAX or the list of objects?
Thanks for any insight you can provide.
Did you try running this in your dev environment under IIS 7.5?
Make sure debug="false", not "true" in your web.config
I think you need to dig out some debugging tools and capture a worker process dump of your production server, you won't be able to properly diagnose this issue with just PerfMon and Task Manager.
I posted this answer on Stack Overflow a while back which should get you started:
CLR out Of Memory Exceptions
Well, after some hard work, I have identified the culprit: our old nemesis the Endless Loop. Of course, if the development environment had thrown an exception, I would have caught and excised the problem - but it didn't.
I would say the lesson learned here is to understand that different versions of IIS and DevEnv respond to errors differently and that we must test the app in the same configuration in which it will be deployed.
Thanks everyone for your feedback.
Just curious if anyone else has got this particular error and know how to solve it?
The scenario is as follow...
We have an ASP.NET web application using Enterprise Library running on Windows Server 2008 IIS farm connecting to a SQL Server 2008 cluster back end.
MSDTC is turned on. DB connections are pooled.
My suspicion is that somewhere along the line there is a failed MSDTC transaction, the connection got returned to the pool and the next query on a different page is picking up the misbehaving connection and got this particular error. Funny thing is we got this error on a query that has no need whatsoever with distributed transaction (committing to two database, etc.). We were only doing select query (no transaction) when we got the error.
We did SQL Profiling and the query got ran on the SQL Server, but never came back (since the MSDTC transaction was already aborted in the connection).
Some other related errors to accompany this are:
New request is not allowed to start
because it should come with valid
transaction descriptor.
Internal .Net Framework Data Provider error 60.
MSDTC has default 90 seconds timeout, if one query execute exceed this time limit, you will encounter this error when the transaction is trying to commit.
A bounty may help get the answer you seek, but you're probably going to get better answers if you give some code samples and give a better description of when the error occurs.
Does the error only intermittently occur? It sounds like it from your description.
Are you enclosing the close that you want to be done as a transaction in a using TransactionScope block as Microsoft recommends? This should help avoid weird transaction behavior. Recall that a using block makes sure that the object is always disposed regardless of exceptions thrown. See here: http://msdn.microsoft.com/en-us/library/ms172152.aspx
If you're using TransactionScope there is an argument System.TransactionScopeOption.RequiresNew that tells the framework to always create a new transaction for this block of code:
Using ts As New Transactions.TransactionScope(Transactions.TransactionScopeOption.RequiresNew)
' Do Stuff
End Using
Also, if you're suspicious that a connection is getting faulted and then put back into the connection pool, the likely solution is to enclose the code that may fault the connection in a Try-Catch block and Dispose the connection in the catch block.
Old question ... but ran into this issue past few days.
Could not find a good answer until now. Just wanted to share what I found out.
My scenario contains multiple sessions being opened by multiple session factories. I had to correctly rollback and wait and make sure the other transactions were no longer active. It seems that just rolling back one of them will rollback everything.
But after adding the Thread.Sleep() between rollbacks, it doesn't do the other and continues fine with the rollback. Subsequent hits that trigger the method don't result in the "New request is not allowed to start because it should come with valid transaction descriptor." error.
https://gist.github.com/josephvano/5766488
I have seen this before and the cause was exactly what you thought. As Rice suggested, make sure that you are correctly disposing of the db related objects to avoid this problem.