I am trying to make a REST call using chrome DHC and Advanced rest clients. For example:
A HEAD call to http://www.google.co.in
using advanced REST client gives:
Redirect To:https://www.google.co.in/ with status: 302 Show explanation HTTP/1.1 302 Found
Redirection information has not been cached.
Location: https://www.google.co.in/
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Date: Fri, 29 Nov 2013 06:57:46 GMT
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic
Content-Length: 222
Proxy-Connection: Keep-Alive
Connection: Keep-Alive
Status
200 OK Show explanation Loading time: 764
Request headers
Accept: application/json
Accept-Language: en-US
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36
X-Chrome-Variations: <some value>
Accept-Encoding: gzip,deflate,sdch
Cookie: <some value>
Response headers
status: 200 OK
version: HTTP/1.1
alternate-protocol: 443:quic
cache-control: private, max-age=0
content-encoding: gzip
content-length: 34821
content-type: text/html; charset=UTF-8
date: Fri, 29 Nov 2013 06:57:46 GMT
expires: -1
server: gws
x-frame-options: SAMEORIGIN
x-xss-protection: 1; mode=block
I am more interested on details of request header as "Accept-Charset" is not recognized. If i give anything as header it is atleast displayed in request header. Now some questions:
why this is not recognized? is this client problem or server problem? do i need to use any other client. I am not aware.
If it is recognized then I should be able to validate this value in server code, which in my case is running in jetty?
Thanks,
Akhi
I found that its a client problem, specific to Chrome. I dont know why with chrome this header is completely ignored.
When i tried to send http request using CURL command, i was able to retrieve this header. Now i can go ahead with validation of "Accept-Charset" header.
Related
I'm building a little application that loads fresh data from the OpenStreetMap project. For my app, it's very important to request the freshest data possible, so I don't want to be served cached data. Now, since OpenStreetMap has a lot of data requests to handle, they have some kind of caching on their side too. Simply sending a GET request isn't enough, because this will likely serve a pre-cached tile:
x-served-by: cache-ams21054-AMS
x-cache: HIT
x-cache-hits: 1
x-timer: S1652643221.620976,VS0,VE1
This led me to inspect the requests sent by Firefox on the official website. This is the header that is sent on a normal refresh:
Host: tile.openstreetmap.org
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0
Accept: image/avif,image/webp,*/*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://www.openstreetmap.org/
Connection: keep-alive
Sec-Fetch-Dest: image
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: same-site
And this is the request sent on a force-refresh (ctrl+f5):
GET /14/8033/6198.png HTTP/2
Host: tile.openstreetmap.org
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0
Accept: image/avif,image/webp,*/*
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://www.openstreetmap.org/
Connection: keep-alive
Sec-Fetch-Dest: image
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: same-site
Pragma: no-cache
Cache-Control: no-cache
The only two additional header fields are Cache-Control: no-cache and Pragma: no-cache.
This led me to implement the same in my own application. I added the same header fields to my requests. However, this did not fix my problem and I was still getting served cached content (x-cache: HIT) in the response. After some googling and experimenting, I tried adding max-age=0 to Cache-Control and this did stop the servers from sending me cached content.
Here's my question: how come that my app needs the max-age=0 value, while a browser can get non-cached content reliably with just the no-cache values?
What other fields control that behavior, if all the fields I have are these:
User-Agent: MyApp-v1
Cache-Control: no-cache, max-age=0
Pragma: no-cache
For reference, here are the response headers for a standard refresh and a force-refresh:
Refresh:
HTTP/2 200 OK
server: Apache/2.4.41 (Ubuntu)
strict-transport-security: max-age=31536000; includeSubDomains; preload
expect-ct: max-age=0
etag: "afa1fdc623f5ea600b40ebd27c27f97a"
cache-control: max-age=33407, stale-while-revalidate=604800, stale-if-error=604800
expires: Thu, 12 May 2022 16:47:46 GMT
access-control-allow-origin: *
x-tilerender: odin.openstreetmap.org
content-type: image/png
accept-ranges: bytes
date: Sun, 15 May 2022 20:01:23 GMT
via: 1.1 varnish
age: 304223
x-served-by: cache-ams21051-AMS
x-cache: HIT
x-cache-hits: 1
x-timer: S1652644883.024001,VS0,VE1
content-length: 8290
X-Firefox-Spdy: h2
Forced refresh:
HTTP/2 200 OK
server: Apache/2.4.41 (Ubuntu)
strict-transport-security: max-age=31536000; includeSubDomains; preload
expect-ct: max-age=0
etag: "1e2555621fd6e9da0c35d0ab5440d2e5"
cache-control: max-age=8170, stale-while-revalidate=604800, stale-if-error=604800
expires: Sun, 15 May 2022 22:03:15 GMT
access-control-allow-origin: *
x-tilerender: nidhogg.openstreetmap.org
content-type: image/png
accept-ranges: bytes
date: Sun, 15 May 2022 19:47:05 GMT
via: 1.1 varnish
age: 0
x-served-by: cache-ams21083-AMS
x-cache: MISS
x-cache-hits: 0
x-timer: S1652644026.595670,VS0,VE97
content-length: 14499
X-Firefox-Spdy: h2
A server can emit data however it wishes, including from its own caches. The Cache-Control: no-cache mostly pertain to the browser-side caches.
For a browser to do a 'fresh' request, the most important this is mainly just not sending an ETag or If-Modified-Since header.
There's no standard header/instruction to tell a server to get a 'fresher' version of an item and not use its own internal caches. It's possible that for the API you're using they do have a mechanism to request uncached responses, but it's not standard.
I've been trying to think of the proper name for the top part of an HTTP message, which includes the request line and the headers. "Headers" doesn't quite cut it because of the request line and "request" doesn't work because that would include the body as well.
Can somebody please enlighten me? :(
Can be HTTP request headers in the following format:
GET /tutorials/other/top-20-mysql-best-practices/ HTTP/1.1
Host: net.tutsplus.com
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 (.NET CLR 3.5.30729)
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Cookie: PHPSESSID=r2t5uvjq435r4q7ib3vtdjq120
Pragma: no-cache
Cache-Control: no-cache
Could also be HTTP response header in the following format:
HTTP/1.x 200 OK
Transfer-Encoding: chunked
Date: Sat, 28 Nov 2009 04:36:25 GMT
Server: LiteSpeed
Connection: close
X-Powered-By: W3 Total Cache/0.8
Pragma: public
Expires: Sat, 28 Nov 2009 05:36:25 GMT
Etag: "pub1259380237;gz"
Cache-Control: max-age=3600, public
Content-Type: text/html; charset=UTF-8
Last-Modified: Sat, 28 Nov 2009 03:50:37 GMT
X-Pingback: http://net.tutsplus.com/xmlrpc.php
Content-Encoding: gzip
Vary: Accept-Encoding, Cookie, User-Agent
Here Mate, take a look. I thing that there no other valid answers. TonyGW is correct.
Is an screenshot from "Head First Servlets and JSP 2nd Edition"
I want to use ETag to cache the version of a request and return 304 not modifed response to the client so the client can use last cached page.
So my url is like this which returns a json response
"http://server/WEB_GWT/prmCall?prmName=PRM_SIS_PROG_REG_STATUS"
In my servlet handling this request, I am always putting ETag information to store its value to be the version of url param PRM_SIS_PROG_REG_STATUS.
So response header returning to client is
HTTP/1.1 200 OK
Date: Sat, 07 Dec 2013 16:07:49 GMT
Server: IBM_HTTP_Server
ETag: "5"
Last-Modified: Sat, 07 Dec 2013 16:07:49 GMT
Content-Length: 356
Keep-Alive: timeout=10, max=99
Connection: Keep-Alive
Content-Type: application/json
Content-Language: tr-TR
In my next request, I am expecting this request header to include "If-None-Match" header to return the version of the request but I cannot get this header param. Any idea why I cannot get my ETag back.
My next request header is
GET /OZU_GWT/prmCall?prmName=PRM_SIS_PROG_REG_STATUS HTTP/1.1
Host: 10.100.199.103
Connection: keep-alive
Cache-Control: max-age=0
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36
Accept: */*
Referer: http://10.100.199.103/OZU_GWT/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4
Cookie: JSESSIONID=0000kvocMBmjoWPbpKt_VAsDUMv:-1
Inoder to cache your request you need to include "Cache-Control" directive and specify the way in which the response should be cached and for what period.
HTTP/1.1 200 OK
Date: Sat, 07 Dec 2013 16:07:49 GMT
Server: IBM_HTTP_Server
ETag: "5"
Cache-Control : public, max-age=86400
Content-Length: 356
Keep-Alive: timeout=10, max=99
Connection: Keep-Alive
Content-Type: application/json
Content-Language: tr-TR
Here the Cache-Control header says that the content can be stored by "public" caching servers and the duration after which it needs to revalidate the content is 86400 seconds. And so when you refresh the page again "If-None-Match" and "If-Modified-Since" conditional headers will kick in and use the cached data.
After some investigation,I found out SmartGWT framework requests are sent to server with bypassCache:true flag which was sending my xhr request without any cache header. I managed to fixed it by overriding following method in DataSource class.
#Override
protected Object transformRequest(DSRequest dsRequest) {
dsRequest.setBypassCache(false);
I've read all the caching documentation for HTTP, and this are the headers for my file:
Edit: I am now using amazon S3 to see if it helps and still the same problem! Here go the request and the response> What am I missing? Even thought the response asks for the browser to cache the file, it doesn't! I've checked the Google Chrome cache and the file is not there.
Request URL:https://s3-sa-east-1.amazonaws.com/combo-combat-release/WebPlayer.unity3d
Request Method:GET
Status Code:200 OK
**Request Headers**
GET /combo-combat-release/WebPlayer.unity3d HTTP/1.1
Host: s3-sa-east-1.amazonaws.com
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.52 Safari/537.17
Accept: */*
Accept-Encoding: gzip,deflate,sdch
Accept-Language: es,en-US;q=0.8,en;q=0.6
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
**Response Headers**
HTTP/1.1 200 OK
x-amz-id-2: xu/RKgR4Xt6G3jQ1qkdO5rzd4TlngDYYERZ+bbGlalitJFMDjAgiLAoP4GXhmkeo
x-amz-request-id: 6A9389D16407D28D
Date: Wed, 16 Jan 2013 22:42:26 GMT
Cache-Control: max-age=2592000, public, must-revalidate
Last-Modified: Wed, 16 Jan 2013 22:29:55 GMT
ETag: "0f8d22257da9fcae61f21fd30b7a1fd4"
Accept-Ranges: bytes
Content-Type: application/octet-stream
Content-Length: 39222189
Server: AmazonS3
But the browser doesn't cache the file. Each time, the request is sent and all the data transferred from scratch (200 OK instead of 304). :(
File size is 37.4 MB
Any tip on what could be wrong?
Thanks,
Manuel
Try HTTP 1.1 instead of 1.0.
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html states:
Note that HTTP/1.0 caches might not implement Cache-Control and
might only implement Pragma: no-cache
I have an asp.net mvc application which is authenticating a username/password and if correct setting a cookie in the browser and doing a 302 to another page.
For some reason though the browser will ignore the set-cookie and thus never stores the cookie. The weird thing is:
The exact same code on my local environment works perfectly even though the same code run in my test environment in a different tab fails
It works perfectly in some browsers but not others
It will sometimes work in a browser that previously failed to work
I have opened up fiddler and have been comparing the requests/responses and everything looks identical. Can anyone give any suggestions of whats going on?
Not working
Login response:
HTTP/1.1 302 Found
Cache-Control: private
Content-Type: text/html; charset=utf-8
Location: https://foo.test/bar
Server: Microsoft-IIS/7.0
X-AspNetMvc-Version: 2.0
X-AspNet-Version: 2.0.50727
Set-Cookie: .ASPXAUTH=7C02633DAD998CB9CD25CC413FF34506DBF9095B78FC69FD03F83C4F7A091BF45469D389510F5ADD286AB6131EEC14609199C9CAD6B82E2BAFB61DE382BC34A65B72FEE5A9DD53820250E339FB6B863974C91F25CD2BE53646296C6E72F6C18F53C4BE7F9977CE9DB58647D9190093A167DCCBC698D5D4803739D0ECDA4621E744FF886EF7E0E1D3B0ED4A12FB08E34D521F20AA5C9549C66BD3171C68313E70E0ACCB851FA7A7D1509EF30345998A80DF0577F38A8C85E141C4F17803205CDDE05DD2C9; expires=Wed, 22-Dec-2010 16:45:06 GMT; path=/
X-Powered-By: ASP.NET
Date: Sun, 01 Jan 2012 15:36:01 GMT
Content-Length: 220
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
Next request (note no new cookie):
GET https://foo.test/bar HTTP/1.1
Host: foo.test
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,fr;q=0.7,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: https://foo.test/login
Cookie: ASP.NET_SessionId=oskgcf45t5oqvo453kmftw45;
Working
Login response:
HTTP/1.1 302 Found
Cache-Control: private
Content-Type: text/html; charset=utf-8
Location: https://foo.local/bar
Server: Microsoft-IIS/7.5
X-AspNetMvc-Version: 2.0
X-AspNet-Version: 2.0.50727
Set-Cookie: .ASPXAUTH=CE23F217A77AC4D7DE76D16EDC27B21257973DECEF8F85D2FD3E345D051C81BE42F35978884BF7E508BF298824BADE1461C56966F5C6A2BF96F2E99F038068CBC068755494A3CD36BB2283040378982B7F96C76E1E2DCF1F6F481AB9C1399D7CFADF30B3049BDE7E94215DF58D091364974C69399AF92B0E03A10C3BF25907DC187060E681D4867E24DBB39F2D26659FDCDCD661DF8DFD88ABD7E4D931207614611013CA68065F7805D055E1FFF72B91C07C5576D4581FB9E4A04029E51E0A78ADD6B894; expires=Wed, 22-Dec-2010 16:49:34 GMT; path=/
X-Powered-By: ASP.NET
Date: Wed, 22 Dec 2010 16:19:34 GMT
Content-Length: 221
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
Next request (note the new cookie):
GET https://local.toptable.com/ism/confirm?c=2&o=True&v=1313&t=12-00&d=21-12-2010&l=True HTTP/1.1
Host: foo.local
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.4) Gecko/20100611 Firefox/3.6.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,fr;q=0.7,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: https://foo.local/login
Cookie: ASP.NET_SessionId=5aefag4544y4pvqht1k3k455; .ASPXAUTH=CE23F217A77AC4D7DE76D16EDC27B21257973DECEF8F85D2FD3E345D051C81BE42F35978884BF7E508BF298824BADE1461C56966F5C6A2BF96F2E99F038068CBC068755494A3CD36BB2283040378982B7F96C76E1E2DCF1F6F481AB9C1399D7CFADF30B3049BDE7E94215DF58D091364974C69399AF92B0E03A10C3BF25907DC187060E681D4867E24DBB39F2D26659FDCDCD661DF8DFD88ABD7E4D931207614611013CA68065F7805D055E1FFF72B91C07C5576D4581FB9E4A04029E51E0A78ADD6B894
In the "Not Working" login response, the server's Date: header has the value Sun, 01 Jan 2012 15:36:01 GMT (the future!), and your cookie expiration is set for Wed, 22-Dec-2010 16:45:06 GMT which will cause the browser to immediately expire the cookie, thus not storing it.