I'm seeing a behaviour in Firefox which seems to me unexpected. This is not particularly repeatable (unfortunately) but does crop up from time to time. Once it starts, it is repeatable on refreshing the page until a full refresh (ctrl-f5) is done. Last time, I managed to get a trace.
Basically, FF4.0.1 is requesting a resource (from an ASP.NET MVC 3 application running under IIS7):
GET http://www.notarealdomain.com/VersionedContent/Scripts/1.0.40.5653/jquery.all.js HTTP/1.1
Host: www.notarealdomain.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
Accept: */*
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive
Referer: http://www.notarealdomain.com/gf
If-Modified-Since: Sat, 01 Jan 2000 00:00:00 GMT
It then gets the following response from the server (via a proxy, but I can see in the IIS server logs that the request was made all the way to the server):
HTTP/1.1 304 Not Modified
Date: Mon, 09 May 2011 14:00:47 GMT
Cache-Control: public
Via: 1.1 corp-proxy (NetCache NetApp/6.0.3P2D5)
This seems reasonable - the client makes a conditional request (if-modified-since), and the server responds "ok - use your copy" (304 Not Modified).
The problem is that the client in this case then does not dish up the file - it behaves as if there was no content (ie if an image, it doesn't appear, if js it behaves as if the .js file is missing on the page, if .css then the page renders without the css styles, etc). This is apparent on both the web page itself, and also when using the excellent HttpWatch tool. HttpWatch clearly shows that the browser did have the item in cache, but didn't use it as the content source.
What am I missing here? The above conversation seems reasonable, so why does FF make a conditional request and then not use its cached copy when told to do so? If you then ctrl-F5 to force a full refresh, the behaviour goes away, and only returns sporadically.
We also have some anecdotal evidence that this occurs with FF3 and Chrome too, but we haven't confirmed this with any forensic data.
Has anyone else seen this, and does anyone know what might be going wrong, or what further steps might isolate the problem?
OK - I got to the bottom of this problem. As ever - the answer was staring me in the face!
The problem was not with the caching behaviour at all. Another part of the system was sporadically failing and writing out a zero length file - which was being cached by Firefox.
So when Firefox sent a conditional request to the server and received it's 304/Not Modified, it dutifully went off and used the corrupt zero length version of the file that it found in its cache.
All the signs were there for me to see - it just took a bit of time to get there :)
Thanks all, for your comments and suggestions.
Related
Background
I am building a custom HTTP parser in C++/CX using sockets. As such, I have full control over the entire HTTP request and response.
Request
GET /posts/html-android-app?referrer=rss HTTP/1.1
Host: mixturatech.com
Connection: close
Response
HTTP/1.1 200 OK
Date: Thu, 30 Apr 2015 04:44:59 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Access-Control-Allow-Origin: *
Cache-Control: public
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
6a2f
<!DOCTYPE html>
[trimmed document content]
</html>
0
Additional Data
If I navigate to the webpage with Chrome, WireShark captures the same data that I am seeing (with extraneous characters), yet Chrome manages to trim that content out. (I am looking at Chrome's data in the Network tab in Developer Tools.)
I do not see this problem on every site I retrieve, but the problem, if it exists, seems to be sitewide.
Questions
What is up with the 6a2f and 0 preceding and following the document?
Is this an encoding issue?
Is there some way that I can positively identify, without hardcoding boundaries for the document, such as it must start with < and end with >, where the actual content lies?
Will those characters, if they exist in a page, always be limited to length 4 and 1 respectively?
This is "chunked transfer encoding". Read http://greenbytes.de/tech/webdav/rfc7230.html#chunked.encoding.
HttpBatchResponseMessage responses never fully return on my machine, it only returns the headers but does not complete the request. I tryed the following OData sample, it works just fine on my colleagues machine:
http://aspnet.codeplex.com/SourceControl/latest#Samples/WebApi/OData/v3/ODataEFBatchSample/ReadMe.txt
(super simple to try, download, start, show network requests in the browser or fiddler)
I would really appreciate ideas for further investigations!
The response
from POST to http://localhost:15625/odata/$batch only returns the headers without any content
Cache-Control:no-cache
Content-Length:2114
Content-Type:multipart/mixed; boundary=batchresponse_1c6b4001-17b8-41bb-9579-970cdc358dba
DataServiceVersion:3.0
Date:Tue, 10 Mar 2015 17:03:40 GMT
Expires:-1
Pragma:no-cache
Server:Microsoft-IIS/8.0
X-AspNet-Version:4.0.30319
X-Powered-By:ASP.NET
X-SourceFiles:=?UTF-8?B?QzpcVXNlcnNca2phcnRhblxEb3dubG9hZHNcT0RhdGFFRkJhdGNoU2FtcGxlXE9EYXRhRUZCYXRjaFNhbXBsZVxvZGF0YVwkYmF0Y2g=?=
Figured out ...
The returned headers already tell the length of the response (Content-Length:2114). I tryed to inject some custom classes an figured out:
Works on my colleagues computer
The problem occures when returning a ODataBatchContent, worked as I used a StringContent instead (which doesn't do the job)
The ODataBatchContent serialized perfectly fine when serializing to a MemoryStream
I Tryed
Disabling any Firewall, Antivirus software, Software, etc.
Updating Visual Studio etc., my configuration is nearly identical to the one of from my colleague
Debugging ... a lot
Request
POST to /odata/$batch
Status Code:202 Accepted
Headers:
Accept:multipart/mixed
Accept-Encoding:gzip, deflate
Accept-Language:de-DE,de;q=0.8,en-US;q=0.6,en;q=0.4
Cache-Control:no-cache
Connection:keep-alive
Content-Length:1201
Content-Type:multipart/mixed;boundary=batch_b58d-91a6-2b46
DataServiceVersion:1.0
Host:localhost:15625
MaxDataServiceVersion:3.0
Origin:http://localhost:15625
Pragma:no-cache
Referer:http://localhost:15625/Index.html
User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/40.0.2214.115 Safari/537.36
Thanks.
According to Firebug, here are the response headers the first time the resource is retrieved:
Accept-Ranges bytes
Cache-Control public, max-age=86400
Content-Language en
Content-Length 232
Content-Location http://localhost/myapp/cacheTest.html
Content-Type text/html; charset=WINDOWS-1252
Date Wed, 05 Sep 2012 15:59:31 GMT
Last-Modified Tue, 01 May 2012 05:00:00 GMT
Server Restlet-Framework/2.0.3
Vary Accept-Charset, Accept-Encoding, Accept-Language, Accept
I click away and click back, and here are are the request headers sent to the server:
Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding gzip, deflate
Accept-Language en-us,en;q=0.5
Connection keep-alive
Host localhost
Referer http://localhost/myapp/cacheTest2.html
User-Agent Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0
And so, naturally, the server can't send a 304 like I want, and instead sends the entire resource again.
This was happening in Firefox 14, and I thought it might be a bug, so I upgraded. But it is still happening in Firefox 15. Chrome has no problem.
I have tried both with and without an "Expires" header, it makes no difference. Firefox just refuses to send an If-Modified-Since header.
Okay, I feel like a doofus but decided to put my pride aside, and rather than just deleting this question, tell what the solution was in case anyone else ever did the same thing...
Once upon a time, in order to test something, I had turned off caching in Firefox. I turned it back on, and now it is sending the header.
For me, the problem turned out to be that the Last-Modified date in the response I was sending wasn't exactly RFC 1123. Chrome didn't mind; it happily sent my malformed timestamp back in the If-Modified-Since header. Firefox, however, quietly ignored it.
I can see from your headers this wasn't the reason in your case, but I'm posting this answer anyhow, since it took a while for me to realise this was the issue, and maybe, some day, somebody else will have the same problem.
This is under Linux, FWIW (Mint 17, to be precise) but I expect both browsers would behave the same way under other OSes.
For me, it was that Firefox (ESR 60.4.0) wasn't sending "If-Modified-Since" nor "If-None-Match" headers for some resources (like CSS, JS) when I loaded site from address bar.
However, when asking for reload with "ctrl+r", it was sending both headers, but resources still were reloaded with "200 OK" even if they should have returned "304 Not modified"
After some tracking, I found out that it was due to apache 2.4.25 deflate module. If resources were compressed, they effectively would not be cached (that is, they would be reloaded on next access). When looking more into it, it turns out it is due to ETag handling when using deflate.
So the most reasonable kludge for me was using "FileETag None", and now I properly get "304" even for compressed documents when I do "ctrl-r".
Amazingly, even after that it still showed green "200 OK" indicating full retrieval od CSS and JS (and no "If-Modified-Since" in detailed "Request headers" panel) which drove me crazy, until I figured out that this column is sometimes FAKED (there is another column called "Transferred", and if it says "cached" instead of number of bytes, it means firefox actually got the value from internal cache, and not from network "200 OK" request it shows. Checking access_log on server side confirmed that there is no network activity then, so that part just bad UI)
another reason that can cause firefox to not cache requests is if the disk is full. at least on OSX.
this is extra puzzling because safari at that point still properly caches requests, and firefox since could at least cache the requests in memory.
clearing the cache and making some space on the disk helps.
Q:
I have a web application which published on a server .When i try the web application from another city. the performance is so bad and every thing is slow.
Should i make any enhancement to my code or this is related mostly to the network factors?
Any advices please.
Error/Status Code: 200
Start Offset: 0.194 s
Initial Connection: 193 ms
Time to First Byte: 286 ms
Content Download: 1286 ms
Bytes In (downloaded): 48.6 KB
Bytes Out (uploaded): 0.4 KB
Request Headers:
GET /sch/ScheduleForm.aspx HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1 PTST/25
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Connection: keep-alive
Response Headers:
HTTP/1.1 200 OK
Date: Wed, 21 Dec 2011 15:17:00 GMT
Server: Microsoft-IIS/6.0
MicrosoftOfficeWebServer: 5.0_Pub
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Set-Cookie: ASP.NET_SessionId=ane2ncmyyoqwckjmv4bijq45; path=/; HttpOnly
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 4938
First of all you must find if the delay is because of the network, or by the server it self, or by the Client Computer or by your page design.
Its rare to have bad performance only because you change city, maybe the other computer have slow connection or bad configuration, or bad isp and you see that slow.
Client Quick Check
The faster way to a quick check of the network responce is to ping your site, open a command promt window and just write "ping www.yoursite.com /t" and see the time. And if your server is on the same country must be under 50ms.
Network - Page design
Now second point of a global check. You can use this site
http://www.webpagetest.org/
to check the speed of your page globally, and get very interesting results, as the time response.
Server
There always the case that you have place your site on a multi use/shared computer, with thousand of sites and bad configuration, so on peak time the server performs badly. I have see this happens a lot of times.
Your Web Application
If the site is slow compare with the development computer then is ether because live have a huge database that you do not have check, ether because a hacker have found a back door and is attacking by making empty accounts as attack, or something similar. This is up to you to find if the delay is because of your calls inside your program.
And more reasons for that delay exist
One more note, use the tools that exist on Chrome, Mozilla, Opera and Safari to see the time response of your site when your page load.
Hope this helps.
Your best starting point would be to find out where the speed issue is coming from. Try Firebug/Chrome debugging tools to see if the time is being taken on the server (ie sending you the first byte of the website) or if it's just plain old loading time (eg images take a long time).
If it's on the server then you have potentially some architectural/coding issues, if it's just delivery time of content then you have network/content issues (compress content with GZip, optimise pngs with PNGOUT that sort of thing).
Good luck :)
That depends heavily how the geographical location of the two cities is. If the two cities are nearby you will most likely not notice any difference.
If the cities are on different continents you may notice for sure latency issues.
If you have latency issues you can not solve them solely by adjusting your code. You need to do something like e.g. geographical load balancing and hosting different servers at different locations.
Facebook e.g. has multiple data centres.. e.g. one at the West-Coast one on the East-Coast and i think they have one in Europe as well. Depending from which location you come from requests are forwarded to the nearest data centre.
I'm using Apache Abdera to POST atom multipart data to my server, and am having some odd problems that I can't pin down.
It looks like an issue with chunked transfer encoding, but I'm insufficiently experienced to be certain. The problem manifests as the server throwing an error indicating that the request I sent it contains only one mime part, not two as required. I attached Wireshark to the interface and captured the conversation, and it went like this:
POST /sss/col-uri/2ee98ea1-f9ad-4f01-9b1c-cfa3c4a6dc3c HTTP/1.1
Host: localhost
Expect: 100-continue
Transfer-Encoding: chunked
Content-Type: multipart/related; boundary="1306399868259";type="application/atom+xml;type=entry"
The server's response:
HTTP/1.1 100 Continue
My client continues:
198
--1306399868259
Content-Type: application/atom+xml;type=entry
Content-Disposition: attachment; name="atom"
<entry xmlns="http://www.w3.org/2005/Atom"><title xmlns="http://purl.org/dc/terms/">Richard Woz Ere</title><bibliographicCitation xmlns="http://purl.org/dc/terms/">this is my citation</bibliographicCitation><content type="application/zip" src="cid:48bd9436-e8b6-4f68-aa83-5c88eda52fd4" /></entry>
0
b0e9
--1306399868259
Content-Type: application/zip
Content-Disposition: attachment; name="payload"; filename="example.zip"
Content-ID: <48bd9436-e8b6-4f68-aa83-5c88eda52fd4>
Packaging: http://purl.org/net/sword/package/SimpleZip
And at this point the server responds with:
HTTP/1.1 400 Bad Request
Date: Thu, 26 May 2011 08:51:08 GMT
Server: Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8l DAV/2 mod_wsgi/3.3 Python/2.6.1
Connection: close
Transfer-Encoding: chunked
Content-Type: text/xml
Indicating the error (which is well understood). My server goes on to stream a pile of base64 encoded bits onto the output stream, but in the mean time the server is not listening, it has already decided that the request was erroneous.
Unfortunately, I'm not in charge of the HTTP layer - this is all handled by Abdera using Apache httpclient. My code that does this looks like this:
client.execute("POST", url.toString(), new SWORDMultipartRequestEntity(deposit), options);
Here, the SWORDMultipartRequestEntity is a copy of the standard Abdera MultipartRequestEntity class, with a few extra headers thrown in (see, for example, Packaging in the above snippet); the "deposit" argument is just an object holding the atom part and the inputstream.
When attaching a debugger I get to this line of code fine, and then it disappears into a rat hole and then I get this error back.
Any hints or tips? I've pretty much exhausted my angles of attack!
The only thing that stands out for me is that immediately after the atom:entry document, there is a newline with "0" on it alone, which appears to be chunked transfer encoding speak for "I'm finished". Not sure how it got there, or whether it really has any effect. Help much appreciated.
Cheers,
Richard
The lonely 0 may indeed be a problem. My uninformed guess is that it results from some call to flush(), which then writes the whole buffer as another HTTP chunk. Unfortunately at the point where flush is called, the buffer had already been flushed and its size is therefore zero. So the HttpChunkedOutputFilter (or however it is called) should be taught than an empty buffer does not need to be flushed.
[update:] You should set a breakpoint in the ChunkedOutputStream class, especially the flush method. I just looked at its code and it seems to be ok, but maybe I missed something.