I'm getting reports of sporadic instances where users in various browsers get a 302 page, but instead of automatically redirecting, the browser just displays the Object moved to here HTML sent by the browser. What could possibly cause this? If it were one browser on one machine, I'd blame a bad installation or something, but I've had a handful of reports, just in the past couple of days, from a number of machines and browsers, so I'm nervous that something is actually wrong with the HTTP Response, even though it couldn't be that wrong or the app wouldn't work almost everywhere else, as it does.
Anyway, here's the Response packet:
HTTP/1.1 302 Found
Cache-Control: private
Content-Type: text/html; charset=utf-8
Location: /nextpage
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
Set-Cookie: cookiename={...data...}; path=/
Date: Fri, 10 Oct 2014 14:31:44 GMT
Content-Length: 124
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
Am I missing anything here?
Related
While optimizing the caching-behaviour of our website, I noticed that a whole lot of if-none-match-requests are sent to our site. As far as I understand caching, this should not be the case as long as the cache is still valid.
One particular request generates the following response-header:
HTTP/1.1 200 OK
Cache-Control: public, max-age=25920000
Transfer-Encoding: chunked
Content-Type: application/javascript; charset=utf-8
Content-Encoding: gzip
Expires: Thu, 04 Feb 2016 17:20:09 GMT
Last-Modified: Mon, 01 Jan 2001 23:00:00 GMT
ETag: W/"0"
Vary: Accept-Encoding
Server: Microsoft-IIS/8.5
X-AspNet-Version: 4.0.30319
Date: Fri, 10 Apr 2015 16:20:09 GMT
As you can see, the cache should be valid for 300 days. The way I understand it, the browser should use its cache directly during that period. Only after this period is over, it should issue a request with the header if-none-match.
But browsers seem to ignore that and send this if-none-match -request each and every time the page is loaded just to receive a 304-response ("Not Modified").
What do I need to change to keep browsers from sending these useless requests?
Yes, while the cache is fresh browsers should use a local copy without revalidation. However, this is not guaranteed. For example, when users use the Refresh button browsers make requests to the origin server anyway.
There is a Cache-Control: immutable, max-age=… extension that tells browsers you really really mean they should use the cached resource without contacting the server.
A grid of EC2 web servers is running behind an ELB load balancer. The ELB is behind Amazon's CloudFront content delivery network. Content Delivery Networks are very new to me. My understanding is that CloudFront is supposed to speed up performance by caching static content at its "edges". But this isn't what's happening.
Consider my EC2 instances whose content should always have a lifetime of five minutes. For static content this usually means declaring the following in my web.config file:
<staticContent>
<clientCache cacheControlCustom="public" cacheControlMode="UseMaxAge" cacheControlMaxAge="00.00:05:00"/>
</staticContent>
...and for the dynamic stuff, it usually means executing the following commands against an HttpResponse object:
resp.Cache.SetCacheability(HttpCacheability.Public);
resp.Cache.SetMaxAge(TimeSpan.FromMinutes(5));
With that as background...
When my browser hits the ELB directly, everything works as expected. Firebug consistently shows that 304 (Not Modified) is returned for content that exists in the browser's cache, has passed its five minute expiration, but has not been changed on the server. Here are the response headers for a download of defs.js, for example:
HTTP/1.1 304 Not Modified
Accept-Ranges: bytes
Cache-Control: public,max-age=300
Date: Tue, 22 Apr 2014 13:54:16 GMT
Etag: "0152435d158cf1:0"
Last-Modified: Tue, 15 Apr 2014 17:36:18 GMT
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Connection: keep-alive
IIS correctly sees that the file hasn't been changed since April 15th and returns 304.
But looks what happens when the file is grabbed through CloudFront.
HTTP/1.1 200 OK
Content-Type: application/x-javascript
Content-Length: 205
Connection: keep-alive
Accept-Ranges: bytes
Cache-Control: public,max-age=300
Date: Tue, 22 Apr 2014 14:07:33 GMT
Etag: "0152435d158cf1:0"
Last-Modified: Tue, 15 Apr 2014 17:36:18 GMT
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Age: 16
X-Cache: Hit from cloudfront
Via: 1.1 0f140ef1be762325ad24a7167aa57e65.cloudfront.net (CloudFront)
X-Amz-Cf-Id: Evfdhs-pxFojnzkQWuG-Ubp6B2TC5xbunhavG8ivXURdp2fw_noXjw==
In this case CloudFront forces the browser to download the entire file again even though, as you can see:
(a) it knows the file hasn't been modified since April 15th (see Last-Modified header), and
(b) CloudFront does have a cached copy of the file on hand (see X-Cache header)
Perhaps you're wondering if my browser is sending a valid If-Modified-Since header. Indeed it is. Here are the request headers:
GET /code/shared/defs.js HTTP/1.1
Host: d2fn6fv5a0cu3b.cloudfront.net
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:28.0) Gecko/20100101 Firefox/28.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Referer: http://d2fn6fv5a0cu3b.cloudfront.net/
Connection: keep-alive
If-Modified-Since: Tue, 15 Apr 2014 17:36:18 GMT
If-None-Match: "0152435d158cf1:0"
Cache-Control: max-age=0
It's an odd situation. If I just sit in front of my browser and keep doing page Reloads (Cmd-R), maybe about half the time CloudFront will correctly return a 304 and the other half of the time it'll incorrectly return 200 along with all of the content. Waiting for the five minute expiration before interacting with the page yields primarily 200's and only a few 304's. This odd behavior applies to all of the files (.css, .js, .png, etc.) referenced on the HTML page as well as for the containing HTML page itself. I know my app is coded properly because as mentioned above, hitting the ELB directly without going through CloudFront results in the expected 304 result. Any ideas?
The answer was found in an obscure sentence written in a seemingly unrelated piece of Amazon documentation:
When you configure CloudFront to forward cookies to your origin [...] If-Modified-Since and If-None-Match conditional requests are not supported.
Strange, but indeed the reality of the situation is far worse; It's not that forwarding cookies to your origin servers disables conditional requests, but rather that is disables them sometimes -- to the point where the HTTP result code (304 vs 200) is virtually random.
It's important to note that you'll be bitten by this bizarre behavior even if you're not using cookies at all. It's still absolutely essential that the Forward Cookies drop-down be set to "None" as shown in the image below:
Switching the setting to "None" fixes the errant behavior described in my original post.
This solution presents you with another problem though. You're telling CloudFront to totally strip out all cookies prior to forwarding the request to your origin. But your origin server might need those cookies. Further, if you're using the ELB (load balancer) as your origin, a critical cookie that the ELB depends upon to maintain sticky sessions will be totally dropped. Not good.
The solution to the cookie-stripping problem will depend on how your site is organized. In my case, transmission of cookies (session-related or otherwise) is only necessary when posting AJAX data to myDomain.com/ajax/. Because all cookie-dependent URLs fall under the category of ajax/* , a new behavioral rule for that path had to be created and in that rule, and that rule only, the Forward Cookies drop-down is set to "All" instead of "None."
So there it is. Hope this helps someone.
Hello StackOverflow community!
I started to learn Node.js recently, and decided to implement a reverse HTTP proxy as a task. There were a couple of rough places, which I managed to get through on my own, but now I'm a bit of stuck, and need your help. I managed to handle redirects and relative urls, and with implementation of relative url support I faced the problem I'm going to describe.
You can find my code at http://pastebin.com/vZfEfk8r. It's not very big, but still doesn't fit nicely to this page.
So to the problems (there are 2 of them). I'm using http.request to forward client's request to the target server, then waiting for response and sending this response back to client. It works okay for some of the requests, but not for others. This is the first problem: on the web-site I'm using to test the proxy ( http://ixbt.com, cool russian web-site about the tech) I can always get the main page /index.html, but when browser starts to fetch other files referenced from that page (css, img, etc.), most of the requests are ending with ParseError ({"bytesParsed":0}).
While debugging (using Wireshark) I noticed that at some of the requests (if not all) fail with this error when the following HTTP negotiation between proxy and target server occurs:
Request:
GET articles/pics2/201206/coolermaster-computex2012_70x70.jpg HTTP/1.1
Host: www.ixbt.com
Connection: keep-alive
Response:
<html>
<head><title>400 Bad Request</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx</center>
</body>
</html>
Looks like server doesn't send the status code, and no headers. So the question is, can this be the reason of failure (ParseError)?
My another concern is that when I'm trying to get the same file as a standalone request, I have no problems. Just look:
Request:
GET /articles/pics2/201206/coolermaster-computex2012_70x70.jpg HTTP/1.1
Host: www.ixbt.com
Connection: keep-alive
Response:
HTTP/1.1 200 OK
Server: nginx
Date: Mon, 25 Jun 2012 17:09:51 GMT
Content-Type: image/jpeg
Content-Length: 3046
Last-Modified: Fri, 22 Jun 2012 00:06:27 GMT
Connection: keep-alive
Expires: Wed, 25 Jul 2012 17:09:51 GMT
Cache-Control: max-age=2592000
Accept-Ranges: bytes
... and here goes the body ...
So in the end of the day there may be some mistake in how I do the proxy requests. Maybe it's because I actually do lots of them, when the main page is loaded - it has many images, etc.?
I hope I was clear enough, but please ask about details if I missed something. And the full source code is available (again, at the http://pastebin.com/vZfEfk8r), so if somebody would try it, it would be just great. :)
Much thanks in advance!
P.S. As I said, I'm just learning, so if you'll see some bad practices in my code (even unrelated to the question), it would be nice know them.
UPDATE: As was mentioned in comment, I didn't proxied the original request's headers, which in theory could lead to problems with the following requests. I changed that, but, unfortunately, the behavior remained the same. Here's example of new request and response:
Request
GET css/main_fixed.css HTTP/1.1
Host: www.ixbt.com
connection: keep-alive
cache-control: no-cache
pragma: no-cache
user-agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.56 Safari/536.5
accept: text/css,*/*;q=0.1
accept-encoding: gzip,deflate,sdch
accept-language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4
accept-charset: windows-1251,utf-8;q=0.7,*;q=0.3
referer: http://www.ixbt.com/
Response
<html>
<head><title>400 Bad Request</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx</center>
</body>
</html>
I had to craft the 'referer' header by hand, since browser is sending it with reverse proxy url. Still behavior is the same, as you can see. Any other ideas?
Thanks to the valuable comments, I was able to find the answer to this problem. It was nothing related to Node or target web-servers, just a coding error.
The answer is that path component of url was wrong for the relative urls. It already can be visible from my logs in the question's body. I'll repeat them here to reiterate:
Wrong request:
GET articles/pics2/201206/coolermaster-computex2012_70x70.jpg HTTP/1.1
Right request:
GET /articles/pics2/201206/coolermaster-computex2012_70x70.jpg HTTP/1.1
See the difference? The leading slash. Turns out I missed it on my requests for relative urls, due to my own awkward client's url handling. But with a quick-and-dirty fix it's working now, well enough until I'll do a proper client's url handling.
Much thanks for comments, they were insightful!
If above solutions do not work, try removing content-length header. Content-length mismatch causes body parsers to cause this errors
I have a file proxy IHttpHandler to ensure authentication and to log requests. It works fine on the development server and IIS 6. In IIS 7, I have two problems:
Microsoft Office (Word, Excel...) sends WebDAV requests with OPTION and PROPFIND verbs. ASP.NET throws an exception since it doesn't support them. Is there any way to disable these verbs at the IIS level so that it never reaches ASP.NET? I'm guessing it would be returning a 405 Method Not Allowed error (http://en.wikipedia.org/wiki/List_of_HTTP_status_codes#4xx_Client_Error).
IIS 7 turns on chunked encoding. In that case the Content-Length header is not valid and apparently IIS 7 removes it: http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.4.4. However, it also removes the Content-Type header, causing the files to show up as text in the browser. So how can I stop IIS 7 from removing Content-Type, OR how do I turn off chunked encoding for this one page? Below are the response headers for you to compare.
Development server response:
HTTP/1.1 200 OK
Server: ASP.NET Development Server/9.0.0.0
Date: Thu, 23 Dec 2010 17:57:09 GMT
X-AspNet-Version: 2.0.50727
Content-Length: 68096
Content-Disposition: inline; filename=test.doc
Cache-Control: private
Last-Modified: Thu, 23 Dec 2010 09:14:18 GMT
Content-Type: application/msword
Connection: Close
IIS 7 response:
HTTP/1.1 200 OK
Cache-Control: private
Transfer-Encoding: chunked
Last-Modified: Thu, 23 Dec 2010 09:30:31 GMT
Server: Microsoft-IIS/7.5
Content-Disposition: inline; filename=test.doc
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Thu, 23 Dec 2010 17:57:59 GMT
My question on chunked encoding was inaccurate. I had made one small change on my development machine: I added Content-Length. On the development machine it didn't make a difference--it always worked. In IIS 7, adding Content-Length actually disabled chunked encoding and everything worked as expected.
For the WebDAV requests, IIS 7 doesn't send them through to ASP.NET so we're fine. The development server does, however. I saw a suggestion to add the DefaultHttpHandler to handle them, but on the development server that means the raw aspx page is served.
At random this output it occurring at the top of the page. Site is installed on a lot of servers issue only happens on one server.
HTTP/1.1 200 OK Date: Mon, 24 May 2010 04:18:30 GMT Server:
Microsoft-IIS/6.0 X-Powered-By: ASP.NET X-AspNet-Version: 2.0.50727
Cache-Control: private Content-Type: text/html; charset=utf-8
Content-Length: 39611
Use a diagnostics tool such as Fiddler to work out exactly what server its happening on, and what page is being requested, what is being responded with, etc. Then replay that request to another server. Should be able to sort out the problem from there.