First of all, we have a MVC web application which uses NHibernate (version 3) and an Oracle 11g database.
The application is working, but when we publish in the production server a curious scenario happens:
The user access the application and perform a task, for example, select a link into the Menu.
The user waits a couple of minutes (2-3 minutes).
The user perform another task, for example, reload the same page or select another link into the Menu.
The application fails with a ORA-12571: TNS:packet writer failure exception.
The user just refresh the error page, the application works.
The first thing we tried to do was to isolate the problem, so we published the application into another server with the exactly same configuration:
Same binaries, of course.
Same Oracle x64 client version, even the minor version.
Same Windows Server 2008 version with IIS 7.5.
Same IIS configuration (we compared the windows/system32/inetsrv/config files using WinMerge).
Accessing the same production database.
And for our surprise we couldn't reproduce the problem.
Please, someone have a clue of what is going on?
The problem is related to the connection pool of the server's oracle client. It seems that it is delivering invalid connections to the web application, while in the other servers it does not happens.
The solution is not very interesting, but putting the Validate Connection = True inside the connection string resolved the issue. I am aware of the performance penalty of this, but I am out of options.
PS: using this flag, each connection is validated by the connection pool service before delivering it for the client application. This is not very nice, since a database round-trip will happen for every connection request.
Related
From time to time, some requests on my website starts to hang on the RequestAcquireState state of the Session module. When that spiral begins all requests timeout and we need to restart the IIS on the affected server.
I investigated it a lot and the only conclusion I got is that somehow a deadlock is happening while the application tries to access user data stored in Session.
The only option I can think of to fix this issue is to either reduce or stop using Sessions in my application. This is definetely part of the plan, but it will take a while before we can complete that.
We run 6 machines with IIS 7.5, out of proc StateServer and server affinity on in our Load Balance.
Any hints on how to workaround this issue or fix it at all without having to remove Sessions entirely?
Lock mechanism exist on both provider and session module (IIS Session Module). You can develop custom session module, but you still need provider without locking or You can develop custom provider without locking but you still need IIS session module and it is not so simple to implement at that level.
The Solution is UnlockedStateProvider [aka Unlocked]
Follow the white rabbit :P (Check the demo project, it explains everything.)
The answer is Hotfix Rollup 2828841 for .NET Framework 4.5 , here all the explanation:
http://forums.asp.net/t/1888889.aspx/2/10?Question+regarding+a+possible+bug+within+NET+4+5
and here the download link
It works for me on IIS 7.5 Windows Server 2008 rs x64 , asp.net web forms application with lot of ajax request.
I just found out today that if you have a long running request (or in my case, an infinite loop), then all subsequent requests will be locked, because by default ASP.NET locks on session.
So if you have users with requests in RequestAcquireState, then check if there is a request in ExecuteRequestHandler that is locking the session, and thus preventing other requests from starting.
There is a discussion here on how to prevent locking on session.
(Basically, create most of your pages as Session-Read-Only, and modify session as rarely as you can.)
Is it possible those users have another long running request and the requests you see piling up are actually secondary requests? By default, ASP.NET will lock Session until a request is complete. If a second request comes in before the first one is complete, it will have to wait. If you are using MVC, you can change this behavior by adding an attribute to your controller.
[SessionState(SessionStateBehavior.ReadOnly)]
This makes Session read-only, removing the locking behavior allowing subsequent requests to be processed.
I have a SQL Server instance that is throwing the "Named Pipes Provider" error, code 40. I know that this is a relatively common error on new installs and there's a convoluted process to fix it. However, the odd part of this error for me is that (1) it only happens intermittently and (2) it only happens with my ASP.NET application. I have other apps that are accessing this server just fine without any errors.
I came across a vague reference to this error happening when your connection string is part of the ASPX code (as opposed to the web.config) and that doesn't make sense but it's the only option I could come up with. Anyone else seen this?
Sounds like a security context issue? How is the asp.net application accessing the sql server? Integrated windows security or sql security? If the earlier check that the identity of the application (asp.net process) which is defined by the Application Pool identity has login rights on the DB.
From http://blogs.msdn.com/b/sql_protocols/archive/2007/03/31/named-pipes-provider-error-40-could-not-open-a-connection-to-sql-server.aspx :
Summary, give checklist:
Is your target server started?
Is your target server listening on NP? Which Pipe?
Has your client enabled NP? Use the same pipe to connect as Server?
Are you making local connection? If so, what is the instance, default or remote?
Did you put correct instance name in the connection string? Remember, Sqlexpress is a named instance.
Did you enable remote connection? Firewall? IPSec? "File and Printer Sharing" opened? Can access server?
Can you make basic connection by using or \? Use sqlcmd or osql.
The setup: 2 web servers and a seperate state server
I have two production web servers in a load balanced configuration. The ASP.NET web app they host shares state (like a web farm) using this line in their web.configs:
<sessionState mode="StateServer" stateConnectionString="tcpip=9.9.9.9:42424" cookieless="false" timeout="60"/>
9.9.9.9 is the IP of the machine the asp.net session state service is running on (ok it's not 9.9.9.9 really, changed to protect the innocent). It's a third machine (the database server, actually.
It worked fine until...
The error: website down!
Suddenly the site went down, just showing a generic asp.net error page ('turn custom errors off to see this error' or whatever).
The app's log recorded the actual error message:
An unhandled exception occurred Unable to make the session state request to the session state server. Please ensure that the ASP.NET State service is started and that the client and server ports are the same. If the server is on a remote machine, please ensure that it accepts remote requests by checking the value of HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\aspnet_state\Parameters\AllowRemoteConnection. If the server is on the local machine, and if the before mentioned registry value does not exist or is set to 0, then the state server connection string must use either 'localhost' or '127.0.0.1' as the server name.
So it appears that the web app was unable to contact the state server (9.9.9.9).
I "tried turning if auf and then onnegen" - restarting the state server fixed the problem.
Why?
I really want know what happened and why so I can prevent it happening again.
So far all I have are two theories:
A windows update, to .net framework 4, was applied around that time on the state server. So maybe the update did something to the asp.net state service? The windows event viewer showed that .net 4 had logged a warning around then:
Updates to the IIS metabase were aborted because IIS is either not installed or is disabled on this machine. To configure ASP.NET to run in IIS, please install or enable IIS and re-register ASP.NET using aspnet_regiis.exe /i.
Some kind of temporary network problem between the prod web sites and the state server? They do sit right next to each other in the same physical rack though.
??? Any other ideas, anyone?
Anyone seen this before, or able to correct me on anything?
Has this happened since? The easy answer is that the problem was with the db server, not the web app. Are there any relevant errors in the log on the db server?
The fact that both apps threw an error indicates that a common resource was the problem. We chased a similar issue for a good solid week awhile back, and eventually found a faulty fiber channel gadget. (that's below my OSI level, not sure about the details).
Start–> Administrative Tools –> Services
Right-click over the ASP.NET State Service and click “start”
It is working fine We need to follow the steps
Had a similar issue before when our Infrastructure team tried sneaking in an install of 3.5 when they forgot to install it on our Production box. Not bouncing a server after a framework update is just going to cause all kinds of weird problems.
I am seeing a very strange problem on one of my production boxes. We have an application hosted in IIS 6 on a single machine with an apache web server in front of it. My application is using ASP.NET Membership for authentication and relies on session state. I am seeing a problem right now where some server requests will hit a null exception when trying to access any session variables, but if the request is retried, the page hits no exceptions and behaves properly.
I believe this has something to do with the Session ID cookie either getting corrupted or lost on the request, but I have no idea what could cause that. The reason I believe this is because it seems like ASP.NET is not seeing the cookie and creating a new session, which would explain why the variables are null. When it gets the exception, it cannot write the new SessionID cookie back to the client, so the client still maintains the original SessionID. Then when the next request is sent, the original SessionID cookie is used, which now ASP.NET finds and is able to retrieve the session state. This is pure speculation, but seems to fit the symptoms.
Also this site is using no other cookies other then the ones required by ASP.NET Membership and ASP.NET Session, so I am well under the cookie limit for IE. The site has been functioning properly for about 8 months and this issue just popped up recently. I have tried IIS resets and actually rebooting the machine, but nothing has seemed to help the problem.
Updates:
Here are some clarifications that were asked for.
1.)Our Apache server is the only thing exposed to the internet. All requests occur over HTTPS to this box. The Apache box then forwards all request past over HTTP to our application server. This is being done for security reasons. We have looked to see if Apache might have been the issue, but there seems to be no error in the Apache logs.
2.)The null exception is occurring when trying to access an object stored in session that the application expects to be there as opposed to the exception happening with the actual session object itself.
We have found the root of the issue. It looks like the IIS meta-base got corrupted on our app server. The best way to fix this issue is to do a clean install of IIS, but because of business constraints, this is not an option for us. So another solution is to actually just create a new App Pool for the application to run under. According to some people with more IIS expertise than myself, this will fix the problem in the short term, but it is very likely that the same thing will happen to this App Pool. So we are required to create new App Pools if this starts occurring again.
We've got a web system running SQL Server 2005 for the back end, and ASP.Net for the front end (using .net 2.0).
Every now and then, the system barfs out the error in the title: 'SQLOLEDB' failed with no error message available, result code: E_FAIL(0x80004005).
The web system runs just fine 24/7, and then every now and then will toss this out on a select or some such. I've tried re-running the exact select that throws the error, but (of course) it works fine when I do it. And, to answer the obvious follow-up question, no we haven't done any code changes or upgrades to speak of lately.
Has anyone ever run into this before? Nosing around on google seems to only turn up situations where Access has some kind of file issue (permissions, missing data file, etc.)
Firstly, it's probably not SQL Server throwing out the error, and if it is, it's probably not while running the SQL statement itself, but if it is, it's almost certainly going to be peculiar to a login that doesn't have permissions, not the SQL command itself.
The 0x80004005 error is a general permissions failure, and it can occur in just about anything, but it's most likely to be in another layer.
I've seen it in authentication when the account the application is running under does not have access to the network to open a connection to the SQL Server. The SQL Server never even sees the request, but the client will say that the server cannot be found.
If it's intermittent, that could point to an intermittent domain controller issue, but that would depend on how your Windows Servers and Active Directory are set up, and how your application is connecting to SQL Server.
To solve this one, your netadmins will need to really know their Windows logging functionality and track it down for you.