By referring the below forum
Removing/Hiding/Disabling excessive HTTP response headers in Azure/IIS7 without UrlScan
I could easily hide the http headers like server information but still i'm getting below informtaion.
Is there a possibility to hide Allow and Public methods in the headers? Please share
HTTP/1.1 200 OK
**Allow: OPTIONS, TRACE, GET, HEAD, POST**
Date: Thu, 09 Jan 2014 09:37:00 GMT
**Public: OPTIONS, TRACE, GET, HEAD, POST**
Content-Length: 0
Connection: keep-alive
Like I said in my comments, you don't want to remove those headers from a response to an OPTIONS request, because that request asks the server what other methods can be called on the given resource, in this case /Main.aspx.
The response contains the allowed methods in the Allow and Public headers. If you remove those headers, the response becomes meaningless.
If you want to disable the OPTIONS request altogether, see Disable HTTP OPTIONS, TRACE, HEAD, COPY and UNLOCK methods in IIS.
Headers should be hidden anyway... Not quite sure what's happening here. Have you refreshed the main reference file.
Related
I've an HTTP client sending many POST requests to a server. The server responds to all requests with 201 Created and a response body. For my purposes, the response header is enough, as I'm only interested in the Location header. I'd like to avoid that the server produces a response body in order to significantly decrease network traffic.
According to RFC 7231, ...
[...] if one or more resources has been created on the origin server as a
result of successfully processing a POST request, the origin server
SHOULD send a 201 (Created) response containing a Location header [...]
..., thus, I assume, the server COULD also respond e.g. with 204 No Content, omiting the body.
Therefore my question: Is it possible to construct a POST request which makes the server respond with 204 No Content or to omit the response body in another way?
Update 1: The server side is a Spring Data REST project and I'm free to configure it. I know that I could set RepositoryRestConfiguration#setReturnBodyOnCreate to false, but that would be overdone as it affects all incoming requests. Therefore, I'd prefer to make the decision on the client side.
There's no real lever you can pull from the client side to control if the server will respond with a body or not, unless the service you work with has a specific feature that allows this.
A header that a server might use is Prefer: return=minimal but if the service doesn't explicitly document support for this, chances are low that this will work.
Really the only think you can do one the client is to:
Kill the TCP connection as soon as you got the response headers
Kill the HTTP/2 stream when you recieved the headers.
This is a pretty 'drastic' thing but clients do use this mechanism for some cases and it does work. However, if the POST response body was somewhat small there's a chance that it's not really making a ton of difference because the response might already have been sent.
There is no way to do it client side only as it is not natively implemented in Spring REST server.
Anyway, any client demand can be transformed as an extra custom header or a query parameter in the request.
A way could be to override default response handlers and detect custom header (implements Prefer: return=minimal as suggested before for instance) and/or query param presence to trigger an empty response with a 204 status. This post may help you to figure it out.
Can you try changing your client such that you
a) Query the server with HTTP HEAD requests instead of POST requests
b) Analyze the response headers. There is no response body for HEAD requests as the purpose of HEAD requests is very similar to your requirement
c) Perform the necessary POST requests only when required
I understand that you may have difficulties at the client end to apply these changes. But, in the longer run, I believe this would be worth it.
Based on Evert's and Bertrand's answers plus a bit of googling, I finally implemented the following interceptor in the Spring Data REST server:
#Configuration
class RepositoryConfiguration {
#Bean
public MappedInterceptor preferReturnMinimalMappedInterceptor() {
return new MappedInterceptor(new String[]{"/**"}, new HandlerInterceptor() {
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
if ("return=minimal".equals(request.getHeader("prefer"))) {
response.setContentLength(0);
response.addHeader("Preference-Applied", "return=minimal"");
}
return true;
}
});
}
}
It produces the following communication, which is good enough for my purposes:
> POST /versions HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.59.0
> Accept: */*
> Content-Type: application/json
> Prefer: return=minimal
> Content-Length: 123
>
> [123 bytes data]
...
< HTTP/1.1 201
< Preference-Applied: return=minimal
< ETag: "0"
< Last-Modified: Fri, 30 Nov 2018 12:37:57 GMT
< Location: http://localhost:8080/versions/1
< Content-Type: application/hal+json;charset=UTF-8
< Content-Length: 0
< Date: Fri, 30 Nov 2018 12:37:57 GMT
I would like to share the bounty evenly, but this is not possible. It goes to Bertrand, as he came with an answer which guided me to the very implementation. Thanks for your help.
I have some js hosted on AWS. I want to cache it to not to pay extra for 304 GET request, but I'm puzzled why two headers are different.
Request Method:GET
Status Code:304 Not Modified
Request header of helper.js
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Cache-Control:max-age=0
Connection:keep-alive
If-Modified-Since:Tue, 20 Aug 2013 13:08:13 GMT
and response header
Age:4348
Cache-Control:max-age=604800
Connection:keep-alive
Why they are different? Does it mean that Cache-Control is wrong? I used Chrome console to get the headers.
I don't think that Cache-Control is wrong and it seems that your content is already cached. From the request headers, I understand that the first request was done at Tue, 20 Aug 2013 13:08:13 GMT as the browser indicates the server "hey, has the content changed since that time?". In return, the server responses with 304 Not Modified header, indicating that the content has not been changed and it should be cached 604800 seconds more until revalidating it. Remember that the caching is done on server side. So, you may want to look at your server defitinitons on js files. Usually, in the deployment environment, I instruct my webserver to send cache header for *.js *.png etc. After configuring the web server for sending cache headers, it is the browser's work to take care of the rest. In that case, your browser works as expected.
You can look at RFC2616 for 304 response. You may also want to look at this decent caching tutorial. It should clear some ideas.
The problem is with Chrome. If you press Refresh button it invalidates the cache, but if you press Enter in address bar it gets the resources from cache.
When I visit chesseng.herokuapp.com I get a response header that looks like
Cache-Control:private
Connection:keep-alive
Content-Encoding:gzip
Content-Type:text/css
Date:Tue, 16 Oct 2012 06:37:53 GMT
Last-Modified:Tue, 16 Oct 2012 03:13:38 GMT
Status:200 OK
transfer-encoding:chunked
Vary:Accept-Encoding
X-Rack-Cache:miss
and then I refresh the page and get
Cache-Control:private
Connection:keep-alive
Date:Tue, 16 Oct 2012 06:20:49 GMT
Status:304 Not Modified
X-Rack-Cache:miss
so it seems like caching is working. If that works for caching then what is the point of Expires and Cache-Control:max-age. To add to confusion, when I test the page at https://developers.google.com/speed/pagespeed/insights/ it tells me to "Leverage browser caching".
Cache-Control: private
Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache, such as a proxy server.
From RFC2616 section 14.9.1
To answer your question about why caching is working, even though the web-server didn't include the headers:
Expires: [a date]
Cache-Control: max-age=[seconds]
The server kindly asked any intermediate proxies to not cache the contents (i.e. the item should only be cached in a private cache, i.e. only on your own local machine):
Cache-Control: private
But the server forgot to include any sort of caching hints:
they forgot to include Expires (so the browser knows to use the cached copy until that date)
they forgot to include Max-Age (so the browser knows how long the cached item is good for)
they forgot to include E-Tag (so the browser can do a conditional request)
But they did include a Last-Modified date in the response:
Last-Modified: Tue, 16 Oct 2012 03:13:38 GMT
Because the browser knows the date the file was modified, it can perform a conditional request. It will ask the server for the file, but instruct the server to only send the file if it has been modified since 2012/10/16 3:13:38:
GET / HTTP/1.1
If-Modified-Since: Tue, 16 Oct 2012 03:13:38 GMT
The server receives the request, realizes that the client has the most recent version already. Rather than sending the client 200 OK, followed by the contents of the page, it instead tells you that your cached version is good:
304 Not Modified
Your browser did have to suffer the round-trip delay of sending a request to the server, and waiting for the response, but it did save having to re-download the static content.
Why Max-Age? Why Expires?
Because Last-Modified sucks.
Not everything on the server has a date associated with it. If I'm building a page on the fly, there is no date associated with it - it's now. But I'm perfectly willing to let the user cache the homepage for 15 seconds:
200 OK
Cache-Control: max-age=15
If the user hammers F5, they'll keep getting the cached version for 15 seconds. If it's a corporate proxy, then all 67,198 users hitting the same page in the same 15-second window will all get the same contents - all served from close cache. Performance win for everyone.
The virtue of adding Cache-Control: max-age is that the browser doesn't even have to perform a "conditional" request.
if you specified only Last-Modified, the browser has to perform a If-Modified-Since request, and watch for a 304 Not Modified response
if you specified max-age, the browser won't even have to suffer the network round-trip; the content will come right out of the caches.
The difference between "Cache-Control: max-age" and "Expires"
Expires is a legacy (c. 1998) equivalent of the modern Cache-Control: max-age header:
Expires: you specify a date (yuck)
max-age: you specify seconds (goodness)
And if both are specified, then the browser uses max-age:
200 OK
Cache-Control: max-age=60
Expires: 20180403T192837
Any web-site written after 1998 should not use Expires anymore, and instead use max-age.
What is ETag?
ETag is similar to Last-Modified, except that it doesn't have to be a date - it just has to be a something.
If I'm pulling a list of products out of a database, the server can send the last rowversion as an ETag, rather than a date:
200 OK
ETag: "247986"
My ETag can be the SHA1 hash of a static resource (e.g. image, js, css, font), or of the cached rendered page (i.e. this is what the Mozilla MDN wiki does; they hash the final markup):
200 OK
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
And exactly like in the case of a conditional request based on Last-Modified:
GET / HTTP/1.1
If-Modified-Since: Tue, 16 Oct 2012 03:13:38 GMT
304 Not Modified
I can perform a conditional request based on the ETag:
GET / HTTP/1.1
If-None-Match: "33a64df551425fcc55e4d42a148795d9f25f89d4"
304 Not Modified
An ETag is superior to Last-Modified because it works for things besides files, or things that have a notion of date. It just is
RFC 2616, section 14.9.1:
Indicates that all or part of the response message is intended for a single user and MUST NOT be cached by a shared cache...A private (non-shared) cache MAY cache the response.
Browsers could use this information. Of course, the current "user" may mean many things: OS user, a browser user (e.g. Chrome's profiles), etc. It's not specified.
For me, a more concrete example of Cache-Control: private is that proxy servers (which typically have many users) won't cache it. It is meant for the end user, and no one else.
FYI, the RFC makes clear that this does not provide security. It is about showing the correct content, not securing content.
This usage of the word private only controls where the response may be cached, and cannot ensure the privacy of the message content.
The Expires entity-header field gives the date/time after which the response is considered stale.The Cache-control:maxage field gives the age value (in seconds) bigger than which response is consider stale.
Althought above header field give a mechanism to client to decide whether to send request to the server. In some condition, the client send a request to sever and the age value of response is bigger then the maxage value ,dose it means server needs to send the resource to client? Maybe the resource never changed.
In order to resolve this problem, HTTP1.1 gives last-modifided head. The server gives the last modified date of the response to client. When the client need this resource, it will send If-Modified-Since head field to server. If this date is before the modified date of the resouce, the server will sends the resource to client and gives 200 code.Otherwise,it will returns 304 code to client and this means client can use the resource it cached.
I need to test some client application code I've written to test its' handling of various status codes returned in an HTTP response from a web server.
I have Fiddler 2 (Web Debugging Proxy) installed and I believe there's a way to modify responses using this application, but I'm struggling to find out how. This would be the most convenient way, as it would allow me to leave both client and server code unmodified.
Can anyone assist as I'd like to intercept the HTTP response being sent from server to client and modify the status code before it reaches the client?
Any advice would be much appreciated.
Ok, so I assume that you're already able to monitor your client/server traffic. What you want to do is set a breakpoint on the response then fiddle with it before sending it on to the client.
Here are a couple of different ways to do that:
Rules > Automatic Breakpoints > After Responses
In the quickexec box (the black box at the bottom) type "bpafter yourpage.svc". Now Fiddler will stop at a breakpoint before all requests to any URL that contains "yourpage.svc". Type "bpafter" with no parameters to clear the breakpoint.
Programmatically tamper with the response using FiddlerScript. The best documentation for FiddlerScript is on the official site: http://www.fiddler2.com/Fiddler/dev/
Once you've got a response stopped at the breakpoint, just double click it to open it in the inspectors. You've got a couple of options now:
Right next to the green Run to Completion button (which you click to send the response) there's a dropdown that lets you choose some default response types.
Or, on the Headers inspector, change the response code & message in the textbox at the top.
Or, click the "Raw" inspector and mess with the raw response to do arbitrary things to it. Also a good way to see what your client does when it gets a malformed response, which you'll probably test accidentally :)
Another alternative is to use Fiddler's AutoResponder tab (on the right-hand panel). This allows you to catch a request to any URI that matches a string and serve a "canned" response from a file. The file can contain both headers and payload. The advantage of this approach is that you don't have to write FiddlerScript and you don't have to handle each request manually via a breakpoint.
You would set the rule up in Fiddler like shown below (ensure you enable unmatched requests passthrough otherwise all other http requests will fail).
In this example, any request whose URI includes "fooBar" will get the canned response. The format of the file will vary depending on your APIs (you can use your browser to intercept a "real" response and base it on that) but mine looked like the following:
HTTP/1.1 409 Conflict
Server: Apache-Coyote/1.1
X-Powered-By: Servlet 2.5; JBoss-5.0/JBossWeb-2.1
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, DELETE, PUT, PATCH, OPTIONS
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization
Access-Control-Max-Age: 86400
Content-Type: application/vnd.api+json
Content-Length: 149
Date: Tue, 28 Mar 2017 10:03:29 GMT
{"errors":[{"code":"OutOfStock","detail":"Item not in stock","source":{"lineId":{"type":"Order line Number","id":"1"}},"meta":{"availableStock":0}}]}
I found that it needed a carriage return at the end of the last line (i.e. after the json), and that the Content-Length header had to match the number of characters in the json, otherwise the webapp would hang. Your mileage may vary.
Create a FiddlerScript rule. Here's what I used in order to generate a local copy of a website that was intentionally using 403 on every page to thwart HTTrack/WGET.
https://gist.github.com/JamoCA/22db8d68a9a2fb20cb04a85360185333
/* 20180615 Fiddler rule to ignore all 403 HTTP Status errors so WGET or HTTrack can generate local copy of remote website */
SCENARIO: Changing the user agent or setting a delay isn't enough and the entire remote server is configured to respond w/403.
CONFIGURE: Add below rule to FiddlerScript OnBeforeReponse() section. Configure HTTrack/WGET/CRON to use proxy 127.0.0.01:8888 */
static function OnBeforeResponse(oSession: Session) {
if (oSession.HostnameIs("TARGETHOSTNAME_FILTER.com") && oSession.responseCode == 403) {
oSession.responseCode = 200;
oSession.oResponse.headers.HTTPResponseCode = 200;
oSession.oResponse.headers.HTTPResponseStatus = "200 OK";
}
}
I would like to create a cacheable HTTP response for a POST request.
My actual implementation responds the following for the POST request:
HTTP/1.1 201 Created
Expires: Sat, 03 Oct 2020 15:33:00 GMT
Cache-Control: private,max-age=315360000,no-transform
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 9
ETag: 2120507660800737950
Last-Modified: Wed, 06 Oct 2010 15:33:00 GMT
.........
But it looks like that the browsers (Safari, Firefox tested) are not caching the response.
In the HTTP RFC the corresponding part says:
Responses to this method are not cacheable unless the response includes appropriate Cache-Control or Expires header fields. However, the 303 (See Other) response can be used to direct the user agent to retrieve a cacheable resource.
So I think it should be cached. I know I could set a session variable and set a cookie and do a 303 redirect, but I want to cache the response of the POST request.
Is there any way to do this?
P.S.: I've started with a simple 200 OK, so it does not work.
I'd also note that caching is always optional (it's a MAY in the HTTP/1.1 RFC). Since under most circumstances, a successful POST invalidates a cache entry, it's probably simply the case that the browser caches you're looking at just don't implement caching POST responses (since this would be pretty uncommon--usually this is accomplished by formatting things as a GET, which it sounds like you've done).
Short answer: POST caching rarely makes sense. A cache may serve GET requests to a URL which is the same as that of a previous POST, whose response came with a Content-Location header containing the POST's request URI.
From rfc-7231 (http-bis, superseding rfc-2616):
Responses to POST requests are only cacheable when they include
explicit freshness information (see Section 4.2.1 of [RFC7234]).
However, POST caching is not widely implemented. For cases where an
origin server wishes the client to be able to cache the result of a
POST in a way that can be reused by a later GET, the origin server
MAY send a 200 (OK) response containing the result and a
Content-Location header field that has the same value as the POST's
effective request URI (Section 3.1.4.2).
See also Mark Nottinghams Blog:
POSTs don't deal in representations of identified state, 99 times out
of 100. However, there is one case where it does; when the server goes
out of its way to say that this POST response is a representation of
its URI, by setting a Content-Location header that's the same as the
request URI. When that happens, the POST response is just like a GET
response to the same URI; it can be cached and reused -- but only for
future GET requests.
The rfc also describes a PRG sequence which has a similar effect, allowing the response cycle to a POST to fill the cache for a subsequent GET - which is probably more widely implemented.
Can you try to change the Cache-Control to public instead of private and see if it's working?