We have an ASP.Net web application on IIS7 that is used to upload Excel files and then load them into a SQL database by running jobs on the SQL server. The app will wait until the job completes then show the user a message. Due to some larger files being used the app is throwing the error below.
Network Error (tcp_error)
A communication error occurred: "" The Web Server may be down, too
busy, or experiencing other problems preventing it from responding to
requests. You may wish to try again at a later time. For assistance,
contact your network support team.
The app uses an asp:View to progress from various steps. I have tried to bump the session timeout and httpRuntime executionTimeout values to account for how long the job takes to run but it does not appear to have any effect. I know the job completes but the app isn't showing that feedback to the user. I think the error is thrown as the app hits the logic to display the user the view showing all the final messages.
I can only guess that a) there is another setting I'm not aware of for timeout, b) another config file setting is overruling my web config setting for the app, or c) the asp:View is counting all the various steps as one long process and not reseting the "clock" as each step is completed.
As I said, the file upload fine, and the job completes fine, the app just can't advance to that last step where it shows the user the view upon the end. Any ideas on what I can look for to fix this issue? My only other option would be to rewrite the app to not wait for the job to finish and handle notifying the user some other way.
Update 1
After further testing it appears the error is from the ASP.Net custom code we created that does a SQL bulk copy and not the running of the SQL job. The current test runs around 220 seconds testing locally but causes a timeout on a test server.
Update 2
After more research I'm inclinded to think user pevgeniev is correct and this is just a limiting factor of the browser. The only thing that prevents me from marking this as answered is I don't know why file uploads don't appear to have the same issue.
If you're getting this error in the browser, than the timeout is on the client side, and there isn't much you could do server side. As you've suggested, you could rewrite the app, so that it polls for the result from the client, rather than expecting to finish the task in a single request.
Related
Environment: Windows Server 2003; IIS 6, ASP.NET 2.0.50727
I'm going crazy with a brand new web server that we set up (note that this problem doesn't happen on our other web servers which have the same configuration). When loading and asp.net app the first time, the page hangs for over a full minute before showing the page in the browser. After it loads the first page, everything runs very quickly.
Note 1: You will probably say that the application is being compiled for the first time. But I've ruled that out. I put trace messages EVERYWHERE in the app and all the trace messages run within a second of requesting the page. Thus, the app compiles and runs immediately. But when the app is finished rendering the page and my last trace message is printed, nothing happens. IIS is doing something behind the scenes for a full minute before transferring the finished page along http to the user's browser.
Note 2: We found that after hitting the app the first time and things run fine, if we wait an hour then we get the delay again. Thus, IIS has something in its cache that it clears out after an hour and causes our site to stall again.
Note 3: Between each test we stop/start IIS to force it to hang upon loading the app.
Note 4: We watched the Task Manager to see if IIS was spiking and taking up a lot of resources processing something. But that wasn't it. We did see a very quick spike to 50% immediately before the browser showed the page, but for the previous 60 seconds there was only 1% usage on the server.
Note 5: On another test I created a HelloWorld.html page and this does not cause IIS to hang. Thus, it has something to do with calling the ASP.NET library the very first time it sends a rendered page across http. Also, since the app has already been compiled and runs instantly, it's just the part of asp.net that sends the rendered page to the user's browser that causes the delay.
Any ideas? We are a a loss here. All of our other web servers are setup the same way and work fine, but this is a new install. So there must be a configuration setting that was missed or maybe something needs to be installed?
Thanks,
Brian
If you have access to the servers, then make sure that app pool recycling is actually logged to the event logs
cscript adsutil.vbs get w3svc/AppPools/DefaultAppPool/LogEventOnRecycle
you can set it to log everything with
cscript adsutil.vbs Set w3svc/AppPools/DefaultAppPool/LogEventOnRecycle 255
See more here
Then check if there were any recycles.
App initialization, creation the worker process, threads, load the app domain and all the references dll's can take some time, that's normal, but that 1 minute delay is something else probably.
Try to precompile the app on the server and see if that helps
aspnet_compiler -m /LM/W3SVC/[site id ]/Root/[your appname]
If you want to dig deeper, you can check the event trace ETW.
logman query providers
Save the IIS /ASP.NET related Guids to a file like iisproviders.txt
logman start ExampleTrace -pf iisproviders.txt -ets -rt
reproduce
LogParser "SELECT * FROM ExampleTrace" -i:ETW
logman stop ExampleTrace -ets
You can find more hereTroubleshooting appdomain restarts and other issues with ETW tracing
I would also check the w3wp.exe with procexp if it has a TCP connection time out or with Procmon for other clues.
If you have experience with windbg, then you can make a request to the app then quickly attach the debugger to the process
windbg -p [process id of the app pool]
.loadby sos mscorwks
g
and take it from there. If there are exceptions, process crash, etc you should be able to catch it...
Once we had a weird server issue like this and a .NET reinstall solved the problem, still not sure what was the culprit.
Could be some aspnet.config settings on this box that are different from others. Have you tried copying over their config files to this server? There appears to be certificate options along with registry modifications that you can do to remove some lag time during the initial load of a page (precompiling aside)
See here and here
One thing you might want to check on is if there are any database access going on on your page load. That might be blocking the creation of the page during initial page load. Then when the query is cached (either by the db engine or another cache mechanism like memcached), subsequent page loads work as normal.
As per your last comment,
I could stop/start IIS multiple times and the app always ran instantly. I thought it was fixed for good. But now I just tried again (it has been sitting idle for the past couple of hours) and now it is back to hanging on the first request.
This could mean that the cache has expired and thus needs to hit the database once again, causing the delay in page load.
I have an ASP.Net C# web application, running on IIS, that I'm supporting which involves generation of word documents. Some of these word documents take a very long time (i.e. upwards of 20-30 minutes) to generate. What I notice while testing on my dev server is that the server closes the connection long before the process completes, the server-side ASP.Net code itself enters a loop and updates the status of a boolean value when the word doc generation completes.
My workaround for this is to keep the connection alive by implementing a dynamically animated wait screen ( using jquery and ajax) on the client-side that's updated by a repeated asynchronous AJAX call to the server that checks on the status of the operation from a server-side web method. I'm asking about that piece in another question.
Is the solution I'm looking at implementing the best approach to this problem? Are there more efficient or common methods for keeping the connection alive during a long running server-side operation? Any help or insight is appreciated, thanks.
UPDATE:
I tried Brian's suggestion, unfortunately I still get the same error from Chrome that no data is being sent from the server and the entirety of the error is as follows:
No data received Unable to load the webpage because the server sent no
data.
Here are some suggestions: Reload this webpage later.
Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection
without sending any data.
I'll try again by setting the connection timeout in the advanced website settings as described and increasing the connection idle setting.
Ideally, you'd use a socket to notify the client when the process completes. Look at socket.io or native web-socket implementations on how to do this.
You can control Idle time within IIS 7. This is done by going to IIS management; select application pools; then right click on the pool your using for your website. Click the "Advance settings" here you will be able to control idle time out and some other settings for your website. Hope this is what your looking for.
I have a aspx web application that updates or adds files in a database. The clients access through the browser and one of the requirements is that they can start the update and be able to close the browser while the update continues. It appears to run for a little bit after I close the browser but then it stops. How can you keep the application running for asp.net?
That's something you could very well solve with WF (Workflow Foundation). Create a workflow for the task that should survive closing the browser. Workflows have their own threads and livecycles separate from ASP.NET.
The web application will keep running in the application pool, but this will be recycled eventually. As long as the users session runs the application should be kept alive, so by upping the session timeout you may fix the problem.
A better approach though would be to move the long-running task into a service instead, but that may require a rewrite of your application.
Usually for long-running or asynchronous processing, you want to dispatch the request to a back-end service to handle. Trying to keep the web-app alive to finish processing can lead to problems, especially with HTTP and session timeouts.
A common pattern for this is to put the request on a message queue and let a back-end service process it when it can.
I would create a separate windows service that you can push jobs onto from your web application, then check the status of the job(s) when the user logs in again.
The windows service won't be tied to the asp.net app domain so it will continue to run regardless of whats happening in your web application.
I've run into this pattern and you have to decouple the work from the HTTP request. The way we've solved it is to abstract the computing to be done as an event to be scheduled. So, say a user at a browser takes an action that requires a long lived (relatively) computation on the back end, this computation is given a name like 'doXYZForUser' and given a prameter vector like (userId, params...) and sent off to the work queue. Some time in the future the user logs in again and can see what the status of their job is.
I'm running a Java stack and a Java Message Service (JMS) but the principle is the same. The request from the browser queues up an event and the browser get an ACK back saying the event is on the work queue. The queue is managed by an entirely separately running process which in .NET I believe is just called the Message Queue. The job comes up on the queue gets processed and the results can be placed in a separate table containing a reference to the user that kicked off the job, so the next time they log in job status/results can be returned.
I have an application that scans an input directory every five seconds and when a job (i.e. file) is placed in the directory the application reads it, processes it and outputs another file(s) in an output directory.
My question is, If I wanted to put a web based front end on this application, how would I wait for the processing to be complete?
User submits job
job is placed in input directory
......What am I doing on the web page here?
processing occurs
output file is generated
......How do I know that the job is finished?
The two solutions I came up with were:
poll output directory every x seconds from the webpage
use ajax to poll a webservice or webpage that reports back whether the output file is present
Is there a better design for the server? In other words, how would TCP or Named Pipes help in this situation? (Cannot use remoting due to a DCOM object.)
A solution we have done commercially in the past is basically the daemon writes to a log (typically DB), with a date/time stamp, about what its doing, and the web frontend just shows the latest X amount of entries from the log, with a little toggle to hide all of the "Looked in directory, no file found" messages, worked fairly well, we upgraded it later on with AJAX (Timer that reloaded every 20 seconds).
I don't think that Named Pipes are going to make it any easier to get the web client to poll automatically, but it might make the server better able to notify another process that the conversion has completed--and ultimately queue a message to the web browser.
You can try having the web client poll every few seconds to see if the file process has completed, alternatively you could have something like Juggernaut "push" commands out to the page. Juggernaut works using Flash to open a socket on the web browser that continually feeds JavaScript from the server. It could be responsible for sending a command to alert the browser that the file has completed and then issue a redirect.
I'm encountering a situation where it takes a long time for ASP.NET to generate reply with the web page (more than 2 hours). It due to the codebehind running for a while (very long, slow loop).
Browser (both IE & Firefox) stops waiting for the reply (after about an hour) and gives generic cannot display webpage error (similar to what you would see if you'd try to navige to non-existing server).
At the same time asp.net app keeps going (I can see it in debugger) and eventually completes.
Why does this happen? Are there any settings in web.config to influence this? I'm hoping there's a timeout setting that I'm missing that's causing this.
Maybe a settings in IE or Firefox? But I think they wait while the server is keeping connection alive.
I'm experiencing this even when I launch app in debug mode (with compilation debug="true") on my local machine from VS (so it's not running on IIS, but on ASP.NET Dev Server).
I know it's bad that it takes so long to generate the page, but it doesn't matter at this stage. Speeding it up would take a lot of extra work and the delay doesn't really matter. This is used internally.
I realize I can redesign around this issue running logic to a background process and getting notified when it's done through AJAX, or pull it to a desktop app or service or whatever. Something along those lines will be done eventually, but that's not what I'm asking about right now.
Sounds like you're using IE and it is timing out while waiting for a response from the server.
You can find a technet article to adjust this limit:
http://support.microsoft.com/kb/181050
CAUSE
By design, Internet Explorer imposes a
time-out limit for the server to
return data. The time-out limit is
five minutes for versions 4.0 and 4.01
and is 60 minutes for versions 5.x, 6,
and 7. As a result, Internet Explorer
does not wait endlessly for the server
to come back with data when the server
has a problem. Back to the top
RESOLUTION
In general, if a page does not return within a few
minutes, many users perceive that a
problem has occurred and stop the
process. Therefore, design your server
processes to return data within 5
minutes so that users do not have to
wait for an extensive period of time.
The entire paradigm of the Web is of request/response. Not request, wait two hours, response!
If the work takes so long to do, then have the page request trigger the work, and then not wait for it. Put the long-running code into a Windows service, and have the service listen to an MSMQ queue (or use WCF with an MSMQ endpoint). Have the page send requests for work to this queue. The service will read a request, maybe start up a new thread to process it, then write a response to another queue, file, or whatever.
The same page, or a different, "progress" page can poll the response queue or file for responses, and update the user, assuming the user still cares after two hours.
For something that takes this long, I would figure out a way to kick it off via AJAX and then periodically check on it's status. The background process should update some status variable on a regular basis and store it's data in the cache or session when complete. When it completes and the browser detects this (via AJAX), have the browser do a real postback (or get by changing location.href), pick up the saved data, and generate the page.
I have a process that can take a few minutes so I spin off a separate thread and send the result via ftp. If an error occures in the process I send myself an error message including the stack trace. You may want to consider sending the results via email or some other place then the browser and use a thread as well.