Http Error 503 in asp.net because of global.asax file - asp.net

I am using godaddy for my website hosting I have used around one year without any problem,
right now I have a Http Error 503.
please look at the attached image.
I am using this request to make the website login(session state online) not logout quickly.
I have added session timeout to the web.config and i have used timer in the website.
just the request in global.asax files work fine.
Many thanks for the help
global.asax file

From the GoDaddy community:
503 Service Unavailable errors are most often caused by either your website having met its maximum concurrent connection limit or if your hosting account consumes all of its available resources.
Presumably your thread uses up too many resources. Perhaps there is even a thread limit on your application, it may not be allowed to spawn new threads.

Related

Some requests on IIS hang for minutes and end in a lost connection

I have an awkward issue with IIS 10.0 on Windows Server 2016 and ASP.Net 4.5.2 and MVC 5.2.7.
At times, certain requests do not receive a response and run for minutes, maybe 10 or so, before ending in a lost connection (PR_CONNECT_RESET_ERROR in Firefox on Windows, NSURLDomainError in Firefox on iOS). These are mostly POST requests. When this issue occurs, other GET requests will receive a swift response and a correct result. Normally, POST-request do no take long to be processed, typically less than 3 seconds.
Recycling the associated worker process will make the issue go away, for hours or days.
When today inspected the web server when the issue was going on, I saw little CPU usage, less than 10%, memory 56%, the worker process a modest 615 MB. I saw neither logging in the W3C log of these requests, nor in my custom application logs.
I added the Web-Request-Monitor conform How do I see currently executing web request on IIS 8, but in doing so, the the worker process probably got recycled, as the issue is not currently occurring.
There are a reverse proxy and an access manager between the internet and my web server. I suppose they can have something to do with this issue, but it certainly is related to IIS, as recycling helps.
All of this is happening on a acceptation web server running a newer version of my application. I am not aware of any big changes to the application's architecture that could be involved. Also, there will be very little traffic from other clients, if none at all.
What could be next steps to investigate this issue further?
Update
This issue was definitely caused by log4net. However, it was not related to the log4net.Internal.Debug setting. It was caused by two application domains accessing the same log file. This occasionally resulted in concurrency issues with accessing the log file. It appeared that log4net could not properly handle this and got stuck while writing to the log file.
This log file was configured with the RollingFileAppender option. Since we also used AdoNetAppender, we decided to remove file logging all together.
Original
I have found a probable cause. I'll report the steps I took to investigate the issue.
I activated the Worker Processes feature in IIS.
When, after a couple of days of waiting, the issue started again, I found long running requests. They all had State ExecuteRequestHandler and Module Name ManagedPipelineHandler. They had Time Elapsed of hundreds of seconds.
I also activated the Failed Requests Tracing with a rule for long running requests with a Time Taken of 1 minute.
After a couple of days, I started to receive failed request reports. The failed request all have a GENERAL_SET_RESPONSE_HEADER event as their last event.
I added additional debug logging events for each requests. When debugging in my development environment, at one point, I started to see the hanging behaviour there, on one of the new logging statements(!). The application uses log4net.
I captured a stack trace:
log4net.dll!log4net.Appender.AppenderSkeleton.DoAppend(log4net.Core.LoggingEvent loggingEvent) log4net.dll!log4net.Util.AppenderAttachedImpl.AppendLoopOnAppenders(log4net.Core.LoggingEvent loggingEvent) log4net.dll!log4net.Repository.Hierarchy.Logger.CallAppenders(log4net.Core.LoggingEvent loggingEvent) log4net.dll!log4net.Repository.Hierarchy.Logger.Log(System.Type callerStackBoundaryDeclaringType, log4net.Core.Level level, object message, System.Exception exception) log4net.dll!log4net.Core.LogImpl.DebugFormat(string format, object arg0)
The DoAppend method uses lock(this), which may very well cause hangs.
I also found out that the config setting log4net.Internal.Debug was set to true, which I do not want under normal circumstances and this may be related. I did not attempt to understand the log4net code, but I remember that logging initially did not work, in the acceptance environment, so the setting may very well have been set to true then, causing the issue to start.
Another indication that this is happening with log4net is that when the issue last occurred, I realized that logging of level standard, only occurs in some POST requests. I found a POST-request that does not log and requests to it where handled normally, while the other POST-requests still hung.
For now, I have set log4net.Internal.Debug to false and will wait to see what happens.
IIS recycle fix this issue doesn't mean that this is an IIS issue because all asp.net application run in .net runtime unless it is proved that the request is hang in IIS module.
So you may need to wait this issue happen again, then create a Failed request tracing rule for time-taken. Then it will tell us this issue is happening on IIS pipeline module or .net runtime.
If all request hang in .net runtime. Then you may have to capture a hang dump and do a deep analysis via WINDGB and mex extension. It will tell us what's happening there.

Suspicious PUT and GET requests in IIS logs

The Situation
I have come across some very suspicious PUT and GET requests in my IIS server logs. After Googling the requesters address, I have found information linking the IP's to known hacking teams. After each PUT there is an immediate GET for the same resource that was attempted to be uploaded to my server.
Question 1:
Would this be considered a remote code execution attack?
Additional Testing Completed By Me:
The IIS logs show that the response given for the PUT request was 412 'Invalid file type all files are not uploaded'
I have turned on Failed Request Tracing and attempted to upload text files using CURL and this is the same response I am provided with and was not able to upload a file.
Question 2:
What can I do to help prevent these type of attacks from being successful?
I can turn on IIS request filtering, but I am concerned that if I deny PUT my IIS application may be negatively impacted for any future web services.
Question 1: Would this be considered a remote code execution attack?
It is impossible to determine the intentions of the attacker from the information given. They could be looking to gain code execution, or they may simply settle for uploading their own content to your server for you to host, or to try and deface your site with their content.
Question 2: What can I do to help prevent these type of attacks from being successful?
Server configuration and patching. The best advice I could give you is to reduce the attack surface - only enable the features you need. If you're not using PUT in your application, then disable it. Only reenable it if needed. Make sure you have the latest updates for your OS installed.
Security is a wide subject. You need everything from secure code when developing applications to rigorous security testing after.

ASP.NET deployed application stops responding

We have an ASP.NET deployed application running on IIS 7. Lately we started having problems with the website, which usually starts at high traffic times, and the issue is that the page stops loading without showing an error. It essentially continues spinning and does not load. IIS reset would usually fix the issue but we have tried everything to resolve it with no success. Below are additional information about what we have already tried.I can intentionally put the website into this state by running 25 concurrent users to the landing page after which I would have to reset iis because it would stop responding. I am thinking this might have to do with a settings in IIS. Maximum Concurrent Connection is set to the default which is 4294967295. Kind of at a lost here.
We turned on Failed log tracing on IIS. The data in the error log did not provide anything conclusive.This might be partly related to the request not failing completely hence no log was created. Most errors were collected based on time the page took to respond.
I have also looked at the app pool and host log files and found nothing out of place
25 concurrent users is nothing. What's the back end stack? There's not too many details listed here but I'd start with looking at each stage the request is in and in addition enable failed request tracing. Mike (formerly from the IIS team) has a great write up on this, in a nutshell though:
Troubleshoot hanging requests on IIS in 3 steps
View requests
%windir%system32inetsrvappcmd list requests /elapsed:30000
Enabled Failed Request Tracing (modify if not using Default Web Site of course)
%windir%system32inetsrvappcmd configure trace "Default Web Site" /enablesite
%windir%system32inetsrvappcmd configure trace "Default Web Site" /enable /path:test.aspx /timeTaken:00:00:30
Then hopefully you can find some details via
appcmd list traces | findstr "yourpage.aspx"

Bogus URL access causing server to hang

When a unavailable url is accessed, we internally raise an exception and email to support team. We do this to identify is there is hidden error in our web application. Couple of days back, suddenly there was a huge number of access to unavailable url which added load to server and casued SMTP to queue large exception emails. This attempt has brought IIS completely down and none of the applications are accessible.
How to prevent this? Is there any other option like firewall etc to disallow continues request from same ip. I have seen this behavior in google. How can we achieve that?
I'd suggest caching already sent notifications. Before your application sends email, it can check if this error is already reported.
And you can set the cache validity to, say 1 minute, so you get maximum 1 same email/minute.
It is quite easy to implement in ASP.NET.

Lose ASP.NET Session for one request and then regains it on next

I am seeing a very strange problem on one of my production boxes. We have an application hosted in IIS 6 on a single machine with an apache web server in front of it. My application is using ASP.NET Membership for authentication and relies on session state. I am seeing a problem right now where some server requests will hit a null exception when trying to access any session variables, but if the request is retried, the page hits no exceptions and behaves properly.
I believe this has something to do with the Session ID cookie either getting corrupted or lost on the request, but I have no idea what could cause that. The reason I believe this is because it seems like ASP.NET is not seeing the cookie and creating a new session, which would explain why the variables are null. When it gets the exception, it cannot write the new SessionID cookie back to the client, so the client still maintains the original SessionID. Then when the next request is sent, the original SessionID cookie is used, which now ASP.NET finds and is able to retrieve the session state. This is pure speculation, but seems to fit the symptoms.
Also this site is using no other cookies other then the ones required by ASP.NET Membership and ASP.NET Session, so I am well under the cookie limit for IE. The site has been functioning properly for about 8 months and this issue just popped up recently. I have tried IIS resets and actually rebooting the machine, but nothing has seemed to help the problem.
Updates:
Here are some clarifications that were asked for.
1.)Our Apache server is the only thing exposed to the internet. All requests occur over HTTPS to this box. The Apache box then forwards all request past over HTTP to our application server. This is being done for security reasons. We have looked to see if Apache might have been the issue, but there seems to be no error in the Apache logs.
2.)The null exception is occurring when trying to access an object stored in session that the application expects to be there as opposed to the exception happening with the actual session object itself.
We have found the root of the issue. It looks like the IIS meta-base got corrupted on our app server. The best way to fix this issue is to do a clean install of IIS, but because of business constraints, this is not an option for us. So another solution is to actually just create a new App Pool for the application to run under. According to some people with more IIS expertise than myself, this will fix the problem in the short term, but it is very likely that the same thing will happen to this App Pool. So we are required to create new App Pools if this starts occurring again.

Resources