Is there a way to measure this?
Certainly, for get requests, no available headers are being sent consistently from clients.
One idea I got is to get that from query string, but is that possible? Something like (pseudo-code follows)
http://server/default.aspx?t=(new Date().getTime())
Another one that would work is to have users hit a very small page that appends a query string as above, but wanted to avoid a redirection if possible.
(Overall goal is to gather per-request such statistics. The server processing time and server to client are more doable, under some assumptions.)
Thanks in advance.
I've done this through an AJAX request after the initial page load where you have control over the request from the very beginning. Pass the UNIX time in the query string and then when it reaches the server take the difference. I'm not familiar with iis7 so you'd have to make sure that timezone's are accounted for. This number could be very erratic since it's basically just calculating latency and DNS lookups which is different for every client.
Does the request start from an initial page that you have control over it ? in that case you can send the server time with the initial response, and then increment that time second-by-second with a javascript code while the user is on the page. this way you can have a server-synchronized time on the page and when a request goes to be sent from that page you can send that synced time with that request, then all you need is calculate difference on the server.
Related
I have a web app that has some fairly hefty data processing on the backend. A current example workflow is:
User POSTS a form
Server receives form, starts processing
2-4 minutes pass
The server responds
The reason i'm asking this is that initially a web proxy on the user side was killing idle POSTs after 2 minutes. The more I think about it the more this seemed like a reasonable default.
This leaves the question, should I increase the timeout and not fix the problem? Or is this bad practice? It is currently at 2-4 minutes but could easily get longer. Should the application be responding with something rather than just leaving the connection open? If so other than completely redesigning the UI to be asynchronous submit/check back later what options are there?
Generally, if I were to submit a form and it would take that long, I would think something would have gone wrong and attempt to submit again. I think that you should collect the data and give the user some sort of success message. Then create another page that allows them to check on the status of the processing (if the user needs to get the results from that processing).
I want to grab some financial data from sites like http://www.fxstreet.com/rates-charts/currency-rates/
up to now I'm using liburl to grab the sourcecode and some regexp search to get the data, which I afterwards store in a file.
Yet there is a little problem:
On the page as I see it in the browser, the data is updated almost each second. When I open the source code however the data I'm looking for changes only every two minutes.
So my program only gets the data with a much lower time-resolution than possible.
I have two questions:
(i) How is it possible that a source-code which remains static over two minutes produces a table that changes every second? What is the mechanism?
(ii) How do I get the data with second time-resolution, i.e. how do I read out such a changing table thats not shown in the sourcecode.
thanks in advance,
David
You can use the network panel in FireBug to examine the HTTP requests being sent out (typically to fetch data) while the page is open. This particular page you've referenced appears to be sending POST requests to http://ttpush.fxstreet.com/http_push/, then receiving and parsing a JSON response.
try sending POST request to http://ttpush.fxstreet.com/http_push/connect, and see what you get
it will continuously load new data
EDIT:
you can use liburl or python, it doesn't really matter. Under HTTP, when you browse the web, you send GET or POST requests.
Go to the website, open the Developer Tools (Chrome)/firebug(firefox plugin) and you will see that after all the data is loaded, there's a request that doesn't close - it stays open.
When you have a website and you want to fetch data continuously, you can do it in a few techniques:
make separate requests (using ajax) every few seconds - this will open a connection for each request, and if you want frequent data updates - it's wasteful
use long polling or server polling - make 1 request that fetches the data. it stays open, and flushes data to the socket (to your browser) whenever it needs. the TCP connection remains open. When the connection times out - you can reopen it. It's more effective than the above normally - but the connection remains open.
use XMPP or some other protocol (not HTTP) - used mainly on chats, like facebook/msn i think., probably google's and some others.
the website you posted uses the second method - when it detects a POST request to that page, it keeps the connection open and dumps data continuously.
What you need to do is make a POST request to that page, you need to see which parameters (if any) are needed to be sent. It doesn't matter how you make the request, as long as you send the right parameters.
you need to read the response with a delimiter - probably every time they want to process data, they send \n or some other delimiter.
Hope this helps. If you see that you still can't get around this let me know and i'll get into more technical details
Is it possible to request the time on a client computer with ASP.NET without using javascript?
i don't believe so. javascript executes on the client and therefore has access to the client attributes. asp.net code behind files are procesed on the server and therefore do not have access to client data.
Agree with jmatthews.
What I do is on login to the system, I have javascript to write the client time to a hidden field, then when the page posts I use that to compute an offset vs the server time. Anytime in the application I need the client time I can just apply the offset to the current server time. Saves having to "ask" for it at a random point, if that helps.
The only information about the client that the server can see without JavaScript is what comes over the wire in the HTTP headers. Unfortunately, that doesn't include the current time.
About the closest you can get without script is to measure approximate elapsed time, with coarse granularity. You could do that by setting a cookie with a short expiration time, and then looking to see if the cookie has expired, in which case it will no longer be sent to the server.
Another trick for elapsed time would be to set a short cache expiration time on an object on the page; the object will only be requested again once the cache entry expires.
You can also estimate the current local time at the client by using a GeoIP database to find the country and therefore the time zone.
We have a performance measurement module that relies on HttpModule and HttpHandlers and works like this:
Request comes in to server for a page.
HttpModule_Begin writes start time in the cookie.
HttpModule_End writes end time in the cookie.
On client, when page load is finished, fire off a special request via AJAX
that is handled by HttpHandler_ProcessRequest
In HttpHandler_ProcessRequest subtract DateTime.Now from end time written by the previous HttpModule_End (which is stored in the cookie) and store it in the database.
One thing I forgot to mention: all data is shared via a cookie, so the HttpHandler, just pulls the data from the cookie without any session id's, etc...
All participants are in the same time zone.
This process works fine for most cases, but there are some instances that report network time in excess of an hour!
This indicates a very large passage of time between the the writing of end time and firing off the AJAX request or a large passage of time between firing off AJAX request and the time it gets to the app server.
I am trying to figure what could be causing this issues in the wild. Why does the operation not simply time out?
Extend your troubleshooting: append values to the IIS log (HttpContext.Current.Response.AppendToLog(string)).
The IIS log will show your entries and you can determine when those requests were actually recorded. You can append whatever you want and it shows up in the query field of the log (assuming you're using W3C format.)
If you're experiencing delays in requests, as the evidence suggests, you can verify it.
This will give you the IIS log as a comparative source against the recorded values in cookies.
Are you including some unique id that allows the AJAX request handler to figure out the which end time to use for the calculation? If not, perhaps enough of the page has been rendered on the client that the AJAX request is actually coming before the end time has been written to the database and it is using the wrong end time.
I have an Ajax request to a web service that typically takes 30-60 seconds to complete. In some cases it could take as long as a few minutes. During this time the user can continue working on other tasks, which means they will probably be on a different page when the task finishes.
Is there a way to tell that the original request has been completed? The only thing that comes to mind is to:
wrap the web service with a web service of my own
use my web service to set a flag somewhere
check for that flag in subsequent page requests
Any better ways to do it? I am using jQuery and ASP.Net, if it matters.
You could add another method to your web service that allows you to check the status of a previous request. Then you can use ajax to poll the web service every 30 seconds or so. You can store the request id or whatever in Session so your ajax call knows what request ID to poll no matter what page you're on.
I would say you'd have to poll once in a while to see if request has ended and show some notifications, like this site does with badges for example.
At first make your request return immediately with something like "Started processing...". Then use a different request to poll for the result. It is not good neither for the server nor the client's browser to have long open HTTP sessions. Moreover the user should be informed and educated that he is starting a request that could take some time to complete.
To display the result you could have a"notification area" in all of your web pages. Alternatively you could have a dedicated page for this and instruct the user to navigate there. As others have suggested you could use polling to get the result.
You could use frames on your site, and perform all your long AJAX requests in an invisible frame. Frames add a certain level of pain to development, but might be the answer to your problems.
The only other way I could think of doing it is to actually load the other pages via an AJAX request, such that there are no real page reloads - this would mean that the AJAX requests aren't interrupted, but may cause issues with breaking browser functionality (back/forward, bookmarking, etc).
Since web development is stateless (you can't set a trigger/event on a server to update the client), the viable strategy is to setup up a status function that you can intermittently call using a javascript timer to check whether your code has finished executing. When it finishes, you can update your view.