I have an application that scans an input directory every five seconds and when a job (i.e. file) is placed in the directory the application reads it, processes it and outputs another file(s) in an output directory.
My question is, If I wanted to put a web based front end on this application, how would I wait for the processing to be complete?
User submits job
job is placed in input directory
......What am I doing on the web page here?
processing occurs
output file is generated
......How do I know that the job is finished?
The two solutions I came up with were:
poll output directory every x seconds from the webpage
use ajax to poll a webservice or webpage that reports back whether the output file is present
Is there a better design for the server? In other words, how would TCP or Named Pipes help in this situation? (Cannot use remoting due to a DCOM object.)
A solution we have done commercially in the past is basically the daemon writes to a log (typically DB), with a date/time stamp, about what its doing, and the web frontend just shows the latest X amount of entries from the log, with a little toggle to hide all of the "Looked in directory, no file found" messages, worked fairly well, we upgraded it later on with AJAX (Timer that reloaded every 20 seconds).
I don't think that Named Pipes are going to make it any easier to get the web client to poll automatically, but it might make the server better able to notify another process that the conversion has completed--and ultimately queue a message to the web browser.
You can try having the web client poll every few seconds to see if the file process has completed, alternatively you could have something like Juggernaut "push" commands out to the page. Juggernaut works using Flash to open a socket on the web browser that continually feeds JavaScript from the server. It could be responsible for sending a command to alert the browser that the file has completed and then issue a redirect.
Related
We have an ASP.Net web application on IIS7 that is used to upload Excel files and then load them into a SQL database by running jobs on the SQL server. The app will wait until the job completes then show the user a message. Due to some larger files being used the app is throwing the error below.
Network Error (tcp_error)
A communication error occurred: "" The Web Server may be down, too
busy, or experiencing other problems preventing it from responding to
requests. You may wish to try again at a later time. For assistance,
contact your network support team.
The app uses an asp:View to progress from various steps. I have tried to bump the session timeout and httpRuntime executionTimeout values to account for how long the job takes to run but it does not appear to have any effect. I know the job completes but the app isn't showing that feedback to the user. I think the error is thrown as the app hits the logic to display the user the view showing all the final messages.
I can only guess that a) there is another setting I'm not aware of for timeout, b) another config file setting is overruling my web config setting for the app, or c) the asp:View is counting all the various steps as one long process and not reseting the "clock" as each step is completed.
As I said, the file upload fine, and the job completes fine, the app just can't advance to that last step where it shows the user the view upon the end. Any ideas on what I can look for to fix this issue? My only other option would be to rewrite the app to not wait for the job to finish and handle notifying the user some other way.
Update 1
After further testing it appears the error is from the ASP.Net custom code we created that does a SQL bulk copy and not the running of the SQL job. The current test runs around 220 seconds testing locally but causes a timeout on a test server.
Update 2
After more research I'm inclinded to think user pevgeniev is correct and this is just a limiting factor of the browser. The only thing that prevents me from marking this as answered is I don't know why file uploads don't appear to have the same issue.
If you're getting this error in the browser, than the timeout is on the client side, and there isn't much you could do server side. As you've suggested, you could rewrite the app, so that it polls for the result from the client, rather than expecting to finish the task in a single request.
We use SOLR for autocomplete search in our web application. We have a nightly process that adds a bunch of data to our database, and once that process finishes, we need to import that data from the database into our SOLR server.
Currently that import process is done manually in the following sequence:
Check if ondeck SOLR server is idle
Tell ondeck server to do a full import of the data, server status is "busy"
Poll ondeck server every so often and when it finishes, the status reverts to idle.
Swap the ondeck server (with the new information) with the live SOLR server.
Run a stored procedure on our database.
We have an administrative tools page on our web app. What I want to do is put a button on that page that kicks off that process (via AJAX call to our web application) and tells the user that the process has been kicked off. The entire process takes about an hour to do so I can't have the AJAX call run for an hour and then get back to the page telling the user that the process is finished.
I need it done asynchronously. The process has to be kicked off, once it gets kicked off, it informs the user that it's begun, and then the application has continue waiting on each step of the process and then call the next step until it finishes. What does the code in MVC 4 look like to make this happen? I've been looking at the tutorials on MSDN on async and await but I'm a little confused.
I may suggest you to take a look on Websokets (or long poling if you have to support old browsers). There is a really nice library called SignalR which will handle all complex staff for you. It use websockets if it is supported by client browser or automatically switch to long poling if it's not. It has good documentation and a lot of expamles on the Internet.
Environment: Windows Server 2003; IIS 6, ASP.NET 2.0.50727
I'm going crazy with a brand new web server that we set up (note that this problem doesn't happen on our other web servers which have the same configuration). When loading and asp.net app the first time, the page hangs for over a full minute before showing the page in the browser. After it loads the first page, everything runs very quickly.
Note 1: You will probably say that the application is being compiled for the first time. But I've ruled that out. I put trace messages EVERYWHERE in the app and all the trace messages run within a second of requesting the page. Thus, the app compiles and runs immediately. But when the app is finished rendering the page and my last trace message is printed, nothing happens. IIS is doing something behind the scenes for a full minute before transferring the finished page along http to the user's browser.
Note 2: We found that after hitting the app the first time and things run fine, if we wait an hour then we get the delay again. Thus, IIS has something in its cache that it clears out after an hour and causes our site to stall again.
Note 3: Between each test we stop/start IIS to force it to hang upon loading the app.
Note 4: We watched the Task Manager to see if IIS was spiking and taking up a lot of resources processing something. But that wasn't it. We did see a very quick spike to 50% immediately before the browser showed the page, but for the previous 60 seconds there was only 1% usage on the server.
Note 5: On another test I created a HelloWorld.html page and this does not cause IIS to hang. Thus, it has something to do with calling the ASP.NET library the very first time it sends a rendered page across http. Also, since the app has already been compiled and runs instantly, it's just the part of asp.net that sends the rendered page to the user's browser that causes the delay.
Any ideas? We are a a loss here. All of our other web servers are setup the same way and work fine, but this is a new install. So there must be a configuration setting that was missed or maybe something needs to be installed?
Thanks,
Brian
If you have access to the servers, then make sure that app pool recycling is actually logged to the event logs
cscript adsutil.vbs get w3svc/AppPools/DefaultAppPool/LogEventOnRecycle
you can set it to log everything with
cscript adsutil.vbs Set w3svc/AppPools/DefaultAppPool/LogEventOnRecycle 255
See more here
Then check if there were any recycles.
App initialization, creation the worker process, threads, load the app domain and all the references dll's can take some time, that's normal, but that 1 minute delay is something else probably.
Try to precompile the app on the server and see if that helps
aspnet_compiler -m /LM/W3SVC/[site id ]/Root/[your appname]
If you want to dig deeper, you can check the event trace ETW.
logman query providers
Save the IIS /ASP.NET related Guids to a file like iisproviders.txt
logman start ExampleTrace -pf iisproviders.txt -ets -rt
reproduce
LogParser "SELECT * FROM ExampleTrace" -i:ETW
logman stop ExampleTrace -ets
You can find more hereTroubleshooting appdomain restarts and other issues with ETW tracing
I would also check the w3wp.exe with procexp if it has a TCP connection time out or with Procmon for other clues.
If you have experience with windbg, then you can make a request to the app then quickly attach the debugger to the process
windbg -p [process id of the app pool]
.loadby sos mscorwks
g
and take it from there. If there are exceptions, process crash, etc you should be able to catch it...
Once we had a weird server issue like this and a .NET reinstall solved the problem, still not sure what was the culprit.
Could be some aspnet.config settings on this box that are different from others. Have you tried copying over their config files to this server? There appears to be certificate options along with registry modifications that you can do to remove some lag time during the initial load of a page (precompiling aside)
See here and here
One thing you might want to check on is if there are any database access going on on your page load. That might be blocking the creation of the page during initial page load. Then when the query is cached (either by the db engine or another cache mechanism like memcached), subsequent page loads work as normal.
As per your last comment,
I could stop/start IIS multiple times and the app always ran instantly. I thought it was fixed for good. But now I just tried again (it has been sitting idle for the past couple of hours) and now it is back to hanging on the first request.
This could mean that the cache has expired and thus needs to hit the database once again, causing the delay in page load.
i'm not sure I've used the correct terms in the title but this is my question.
If i have a stock standard ASP.NET webpage (webforms or mvc) and i wish to do some longish processing, I create a new process and fire that off.
Eg. save an uploaded file to a local folder then do some image manipulation on it
Eg. save an uploaded video to a local folder then do some video encoding on it.
So while this process is doing it's thing, will it effectively block on available connection hitting the IIS server, to be processed/handled?
Like, I heard there was a finite number of connections that can get processed at once ... which is the number of threads on the ... err thread-queue? Hmm.
Basically, can i have some background tasks going ahead and it doesn't impact the number of possible requests i can process.
If you spawn a new thread, you CAN continue processing the response to the user, and the connection will close and release the connection to the pool. We use this type of setup for error logging purposes where I work. The user gets a quick error message and we have a lengthy process that tries to figure out what happened and inform us through various mechanisms.
But don't take my word for it, spawn a thread with a long sleep and test it out for yourself.
You can have a look at Use Threads and Build Asynchronous Handlers in Your Server-Side Web Code and Asynchronous Pages in ASP.NET 2.0.
I'm encountering a situation where it takes a long time for ASP.NET to generate reply with the web page (more than 2 hours). It due to the codebehind running for a while (very long, slow loop).
Browser (both IE & Firefox) stops waiting for the reply (after about an hour) and gives generic cannot display webpage error (similar to what you would see if you'd try to navige to non-existing server).
At the same time asp.net app keeps going (I can see it in debugger) and eventually completes.
Why does this happen? Are there any settings in web.config to influence this? I'm hoping there's a timeout setting that I'm missing that's causing this.
Maybe a settings in IE or Firefox? But I think they wait while the server is keeping connection alive.
I'm experiencing this even when I launch app in debug mode (with compilation debug="true") on my local machine from VS (so it's not running on IIS, but on ASP.NET Dev Server).
I know it's bad that it takes so long to generate the page, but it doesn't matter at this stage. Speeding it up would take a lot of extra work and the delay doesn't really matter. This is used internally.
I realize I can redesign around this issue running logic to a background process and getting notified when it's done through AJAX, or pull it to a desktop app or service or whatever. Something along those lines will be done eventually, but that's not what I'm asking about right now.
Sounds like you're using IE and it is timing out while waiting for a response from the server.
You can find a technet article to adjust this limit:
http://support.microsoft.com/kb/181050
CAUSE
By design, Internet Explorer imposes a
time-out limit for the server to
return data. The time-out limit is
five minutes for versions 4.0 and 4.01
and is 60 minutes for versions 5.x, 6,
and 7. As a result, Internet Explorer
does not wait endlessly for the server
to come back with data when the server
has a problem. Back to the top
RESOLUTION
In general, if a page does not return within a few
minutes, many users perceive that a
problem has occurred and stop the
process. Therefore, design your server
processes to return data within 5
minutes so that users do not have to
wait for an extensive period of time.
The entire paradigm of the Web is of request/response. Not request, wait two hours, response!
If the work takes so long to do, then have the page request trigger the work, and then not wait for it. Put the long-running code into a Windows service, and have the service listen to an MSMQ queue (or use WCF with an MSMQ endpoint). Have the page send requests for work to this queue. The service will read a request, maybe start up a new thread to process it, then write a response to another queue, file, or whatever.
The same page, or a different, "progress" page can poll the response queue or file for responses, and update the user, assuming the user still cares after two hours.
For something that takes this long, I would figure out a way to kick it off via AJAX and then periodically check on it's status. The background process should update some status variable on a regular basis and store it's data in the cache or session when complete. When it completes and the browser detects this (via AJAX), have the browser do a real postback (or get by changing location.href), pick up the saved data, and generate the page.
I have a process that can take a few minutes so I spin off a separate thread and send the result via ftp. If an error occures in the process I send myself an error message including the stack trace. You may want to consider sending the results via email or some other place then the browser and use a thread as well.