Airflow - Slow Web UI - airflow

I have installed Airflow 2.4.2 on my local machine for learning purposes. When I open Airflow Web UI, the response is very slow. Sometimes, it errors out with the response "ERR_BAD_RESPONSE". Other times, when it loads, it takes about 10 seconds to load the page whenever a link on the web UI is clicked.
https://share.cleanshot.com/GSxb5h57
I tried it with localhost:8080, 127.0.0.1:8080 and myhostname:8080, and all of them resulted in the same slow response.
Is there some customisation/tweaking needed to improve the Web UI response time?
Thanks

Related

Com+ Performance Counters in 2008

I have a COM+ application which is called by a series of ASP Pages. I found this previous post which suggested the following were the ideal set of metrics to monitor:
Errors During Script Runtime
Errors From ASP Preprocessor
Requests Executing
Requests Queued
Sessions Total
Errors From Script Compilers
Debugging Requests
Request Execution Time
Request Wait Time
Requests/Sec
Requests Total
Requests succeeded
Requests Failed Total
Template Cache Hit Rate
Process (inetinfo) Private Bytes
Is this still prevalent in IIS 8.5?
Also, they made mention of a tool called PAL and shared a link on codeplex, however, PAL has been moved off of codeplex, and is now available on Git Here. Any insights from someone who may have recently had to track Perfmon stats for a COM+ application are welcomed.
Thanks!
Dustin

ASP.Net timeout processing file after upload

We have an ASP.Net web application on IIS7 that is used to upload Excel files and then load them into a SQL database by running jobs on the SQL server. The app will wait until the job completes then show the user a message. Due to some larger files being used the app is throwing the error below.
Network Error (tcp_error)
A communication error occurred: "" The Web Server may be down, too
busy, or experiencing other problems preventing it from responding to
requests. You may wish to try again at a later time. For assistance,
contact your network support team.
The app uses an asp:View to progress from various steps. I have tried to bump the session timeout and httpRuntime executionTimeout values to account for how long the job takes to run but it does not appear to have any effect. I know the job completes but the app isn't showing that feedback to the user. I think the error is thrown as the app hits the logic to display the user the view showing all the final messages.
I can only guess that a) there is another setting I'm not aware of for timeout, b) another config file setting is overruling my web config setting for the app, or c) the asp:View is counting all the various steps as one long process and not reseting the "clock" as each step is completed.
As I said, the file upload fine, and the job completes fine, the app just can't advance to that last step where it shows the user the view upon the end. Any ideas on what I can look for to fix this issue? My only other option would be to rewrite the app to not wait for the job to finish and handle notifying the user some other way.
Update 1
After further testing it appears the error is from the ASP.Net custom code we created that does a SQL bulk copy and not the running of the SQL job. The current test runs around 220 seconds testing locally but causes a timeout on a test server.
Update 2
After more research I'm inclinded to think user pevgeniev is correct and this is just a limiting factor of the browser. The only thing that prevents me from marking this as answered is I don't know why file uploads don't appear to have the same issue.
If you're getting this error in the browser, than the timeout is on the client side, and there isn't much you could do server side. As you've suggested, you could rewrite the app, so that it polls for the result from the client, rather than expecting to finish the task in a single request.

Automating a step-by-step process from an MVC application

We use SOLR for autocomplete search in our web application. We have a nightly process that adds a bunch of data to our database, and once that process finishes, we need to import that data from the database into our SOLR server.
Currently that import process is done manually in the following sequence:
Check if ondeck SOLR server is idle
Tell ondeck server to do a full import of the data, server status is "busy"
Poll ondeck server every so often and when it finishes, the status reverts to idle.
Swap the ondeck server (with the new information) with the live SOLR server.
Run a stored procedure on our database.
We have an administrative tools page on our web app. What I want to do is put a button on that page that kicks off that process (via AJAX call to our web application) and tells the user that the process has been kicked off. The entire process takes about an hour to do so I can't have the AJAX call run for an hour and then get back to the page telling the user that the process is finished.
I need it done asynchronously. The process has to be kicked off, once it gets kicked off, it informs the user that it's begun, and then the application has continue waiting on each step of the process and then call the next step until it finishes. What does the code in MVC 4 look like to make this happen? I've been looking at the tutorials on MSDN on async and await but I'm a little confused.
I may suggest you to take a look on Websokets (or long poling if you have to support old browsers). There is a really nice library called SignalR which will handle all complex staff for you. It use websockets if it is supported by client browser or automatically switch to long poling if it's not. It has good documentation and a lot of expamles on the Internet.

iis startup delay with aspx pages

Environment: Windows Server 2003; IIS 6, ASP.NET 2.0.50727
I'm going crazy with a brand new web server that we set up (note that this problem doesn't happen on our other web servers which have the same configuration). When loading and asp.net app the first time, the page hangs for over a full minute before showing the page in the browser. After it loads the first page, everything runs very quickly.
Note 1: You will probably say that the application is being compiled for the first time. But I've ruled that out. I put trace messages EVERYWHERE in the app and all the trace messages run within a second of requesting the page. Thus, the app compiles and runs immediately. But when the app is finished rendering the page and my last trace message is printed, nothing happens. IIS is doing something behind the scenes for a full minute before transferring the finished page along http to the user's browser.
Note 2: We found that after hitting the app the first time and things run fine, if we wait an hour then we get the delay again. Thus, IIS has something in its cache that it clears out after an hour and causes our site to stall again.
Note 3: Between each test we stop/start IIS to force it to hang upon loading the app.
Note 4: We watched the Task Manager to see if IIS was spiking and taking up a lot of resources processing something. But that wasn't it. We did see a very quick spike to 50% immediately before the browser showed the page, but for the previous 60 seconds there was only 1% usage on the server.
Note 5: On another test I created a HelloWorld.html page and this does not cause IIS to hang. Thus, it has something to do with calling the ASP.NET library the very first time it sends a rendered page across http. Also, since the app has already been compiled and runs instantly, it's just the part of asp.net that sends the rendered page to the user's browser that causes the delay.
Any ideas? We are a a loss here. All of our other web servers are setup the same way and work fine, but this is a new install. So there must be a configuration setting that was missed or maybe something needs to be installed?
Thanks,
Brian
If you have access to the servers, then make sure that app pool recycling is actually logged to the event logs
cscript adsutil.vbs get w3svc/AppPools/DefaultAppPool/LogEventOnRecycle
you can set it to log everything with
cscript adsutil.vbs Set w3svc/AppPools/DefaultAppPool/LogEventOnRecycle 255
See more here
Then check if there were any recycles.
App initialization, creation the worker process, threads, load the app domain and all the references dll's can take some time, that's normal, but that 1 minute delay is something else probably.
Try to precompile the app on the server and see if that helps
aspnet_compiler -m /LM/W3SVC/[site id ]/Root/[your appname]
If you want to dig deeper, you can check the event trace ETW.
logman query providers
Save the IIS /ASP.NET related Guids to a file like iisproviders.txt
logman start ExampleTrace -pf iisproviders.txt -ets -rt
reproduce
LogParser "SELECT * FROM ExampleTrace" -i:ETW
logman stop ExampleTrace -ets
You can find more hereTroubleshooting appdomain restarts and other issues with ETW tracing
I would also check the w3wp.exe with procexp if it has a TCP connection time out or with Procmon for other clues.
If you have experience with windbg, then you can make a request to the app then quickly attach the debugger to the process
windbg -p [process id of the app pool]
.loadby sos mscorwks
g
and take it from there. If there are exceptions, process crash, etc you should be able to catch it...
Once we had a weird server issue like this and a .NET reinstall solved the problem, still not sure what was the culprit.
Could be some aspnet.config settings on this box that are different from others. Have you tried copying over their config files to this server? There appears to be certificate options along with registry modifications that you can do to remove some lag time during the initial load of a page (precompiling aside)
See here and here
One thing you might want to check on is if there are any database access going on on your page load. That might be blocking the creation of the page during initial page load. Then when the query is cached (either by the db engine or another cache mechanism like memcached), subsequent page loads work as normal.
As per your last comment,
I could stop/start IIS multiple times and the app always ran instantly. I thought it was fixed for good. But now I just tried again (it has been sitting idle for the past couple of hours) and now it is back to hanging on the first request.
This could mean that the cache has expired and thus needs to hit the database once again, causing the delay in page load.

ASP.Net connection keep alive for long running server-side task

I have an ASP.Net C# web application, running on IIS, that I'm supporting which involves generation of word documents. Some of these word documents take a very long time (i.e. upwards of 20-30 minutes) to generate. What I notice while testing on my dev server is that the server closes the connection long before the process completes, the server-side ASP.Net code itself enters a loop and updates the status of a boolean value when the word doc generation completes.
My workaround for this is to keep the connection alive by implementing a dynamically animated wait screen ( using jquery and ajax) on the client-side that's updated by a repeated asynchronous AJAX call to the server that checks on the status of the operation from a server-side web method. I'm asking about that piece in another question.
Is the solution I'm looking at implementing the best approach to this problem? Are there more efficient or common methods for keeping the connection alive during a long running server-side operation? Any help or insight is appreciated, thanks.
UPDATE:
I tried Brian's suggestion, unfortunately I still get the same error from Chrome that no data is being sent from the server and the entirety of the error is as follows:
No data received Unable to load the webpage because the server sent no
data.
Here are some suggestions: Reload this webpage later.
Error 324 (net::ERR_EMPTY_RESPONSE): The server closed the connection
without sending any data.
I'll try again by setting the connection timeout in the advanced website settings as described and increasing the connection idle setting.
Ideally, you'd use a socket to notify the client when the process completes. Look at socket.io or native web-socket implementations on how to do this.
You can control Idle time within IIS 7. This is done by going to IIS management; select application pools; then right click on the pool your using for your website. Click the "Advance settings" here you will be able to control idle time out and some other settings for your website. Hope this is what your looking for.

Resources