I've an HTTP client sending many POST requests to a server. The server responds to all requests with 201 Created and a response body. For my purposes, the response header is enough, as I'm only interested in the Location header. I'd like to avoid that the server produces a response body in order to significantly decrease network traffic.
According to RFC 7231, ...
[...] if one or more resources has been created on the origin server as a
result of successfully processing a POST request, the origin server
SHOULD send a 201 (Created) response containing a Location header [...]
..., thus, I assume, the server COULD also respond e.g. with 204 No Content, omiting the body.
Therefore my question: Is it possible to construct a POST request which makes the server respond with 204 No Content or to omit the response body in another way?
Update 1: The server side is a Spring Data REST project and I'm free to configure it. I know that I could set RepositoryRestConfiguration#setReturnBodyOnCreate to false, but that would be overdone as it affects all incoming requests. Therefore, I'd prefer to make the decision on the client side.
There's no real lever you can pull from the client side to control if the server will respond with a body or not, unless the service you work with has a specific feature that allows this.
A header that a server might use is Prefer: return=minimal but if the service doesn't explicitly document support for this, chances are low that this will work.
Really the only think you can do one the client is to:
Kill the TCP connection as soon as you got the response headers
Kill the HTTP/2 stream when you recieved the headers.
This is a pretty 'drastic' thing but clients do use this mechanism for some cases and it does work. However, if the POST response body was somewhat small there's a chance that it's not really making a ton of difference because the response might already have been sent.
There is no way to do it client side only as it is not natively implemented in Spring REST server.
Anyway, any client demand can be transformed as an extra custom header or a query parameter in the request.
A way could be to override default response handlers and detect custom header (implements Prefer: return=minimal as suggested before for instance) and/or query param presence to trigger an empty response with a 204 status. This post may help you to figure it out.
Can you try changing your client such that you
a) Query the server with HTTP HEAD requests instead of POST requests
b) Analyze the response headers. There is no response body for HEAD requests as the purpose of HEAD requests is very similar to your requirement
c) Perform the necessary POST requests only when required
I understand that you may have difficulties at the client end to apply these changes. But, in the longer run, I believe this would be worth it.
Based on Evert's and Bertrand's answers plus a bit of googling, I finally implemented the following interceptor in the Spring Data REST server:
#Configuration
class RepositoryConfiguration {
#Bean
public MappedInterceptor preferReturnMinimalMappedInterceptor() {
return new MappedInterceptor(new String[]{"/**"}, new HandlerInterceptor() {
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
if ("return=minimal".equals(request.getHeader("prefer"))) {
response.setContentLength(0);
response.addHeader("Preference-Applied", "return=minimal"");
}
return true;
}
});
}
}
It produces the following communication, which is good enough for my purposes:
> POST /versions HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.59.0
> Accept: */*
> Content-Type: application/json
> Prefer: return=minimal
> Content-Length: 123
>
> [123 bytes data]
...
< HTTP/1.1 201
< Preference-Applied: return=minimal
< ETag: "0"
< Last-Modified: Fri, 30 Nov 2018 12:37:57 GMT
< Location: http://localhost:8080/versions/1
< Content-Type: application/hal+json;charset=UTF-8
< Content-Length: 0
< Date: Fri, 30 Nov 2018 12:37:57 GMT
I would like to share the bounty evenly, but this is not possible. It goes to Bertrand, as he came with an answer which guided me to the very implementation. Thanks for your help.
Related
I'm trying to send AT commands to my ESP32* module and am not getting any response back.
I need to perform a POST request that contains the username and password and other requests later on. I am not structuring these correctly and there is not a lot of good documentation for this.
NOTE: because I cannot share my complete url due to privacy I will use something with the same length ********connected.com:443
Send login information to ********connected.com/login (POST) body{"email":"myemail.ca", "password":"xxxxx"}
once I get the token I will make other requests.
get information regarding user profile ********connected.com/getRoutine ( GET) query param username="bob"
I really want to understand how these requests are structured so if someone can explain it to me elegantly that would be great!
Here is what I have tried..
AT
OK
AT+CIPSTART="TCP","********connected.com",443
CONNECT
OK
AT+CIPSEND=48
> "GET ********connected.com:443/getUsersOnline"
OK
>
Recv 48 bytes
SEND OK
CLOSED
REQUESTED POST REQUEST I HAVE USED
AT+CIPSEND=177 “POST \r Host: ********connected.com\r\n Accept: application/json\r\n Content-Length: 224r\n Content-Type: application/jsonr\n { "email":"myemail.com", "password":"myPassword" } “
There are actually several parts of your system that might be the cause of the malfunctioning:
The AT commands sent (it is not clear how you check for server responses. Responses could proviede clues about what's wrong)
The server side app seems to be a custom implementation that might have bugs as well
The POST request might be malformed
Let's focus on the last one.
POST are described in RFC 7231, and though it is an obscure description without examples, it makes one thing clear: there's not actually a well defined standard... because it is strictly application dependant!
I also quote the relevant part of this brilliant answer to an old SO question:
When receiving a POST request, you should always expect a "payload", or, in HTTP terms: a message body. The message body in itself is pretty useless, as there is no standard.
For this reason, all we can do is to build a POST request as accurate as possible and then to debug the system as a whole thing making sure that the request matches what expected by the server side application.
In order to do this, let's check another external link I found: POST request examples. We found this request:
POST /test HTTP/1.1
Host: foo.example
Content-Type: application/x-www-form-urlencoded
Content-Length: 27
field1=value1&field2=value2
Now let's compare this example to your request:
POST
Host: ********connected.com
Accept: application/json
Content-Length: 224
Content-Type: application/jsonr
{ "email":"myemail.com", "password":"myPassword" }
You are saying to the server that you want to pass a resource to an unspecified application (no path), and that this resource is 224 bytes long (wrong! Message body is shorter).
For these reasons, at least these things can be improved:
POST /path/invic18app.php HTTP/1.1 //The path to the application and HTTP version are missing
Content-Length: 48 //This must be the length of the message body, without the header. I also would write it as the last option, just before message body
Write TWO empty lines before message body, otherwise the server will interpret it as further (wrong) options
I hope this helps you, even if it is a tentative answer (I cannot try this request myself). But, again, you definitely need to sniff packets a TCP levels, in order to avoid debugging the server if you are not sure that data is actually received! If you cannot install Wireshark, also tcpdump will be ok.
By referring the below forum
Removing/Hiding/Disabling excessive HTTP response headers in Azure/IIS7 without UrlScan
I could easily hide the http headers like server information but still i'm getting below informtaion.
Is there a possibility to hide Allow and Public methods in the headers? Please share
HTTP/1.1 200 OK
**Allow: OPTIONS, TRACE, GET, HEAD, POST**
Date: Thu, 09 Jan 2014 09:37:00 GMT
**Public: OPTIONS, TRACE, GET, HEAD, POST**
Content-Length: 0
Connection: keep-alive
Like I said in my comments, you don't want to remove those headers from a response to an OPTIONS request, because that request asks the server what other methods can be called on the given resource, in this case /Main.aspx.
The response contains the allowed methods in the Allow and Public headers. If you remove those headers, the response becomes meaningless.
If you want to disable the OPTIONS request altogether, see Disable HTTP OPTIONS, TRACE, HEAD, COPY and UNLOCK methods in IIS.
Headers should be hidden anyway... Not quite sure what's happening here. Have you refreshed the main reference file.
I have a REST API that allows modification of resources using HTTP POST. It's possible that a client may submit a POST request that results in no modification of the resource. I'm thinking about using the 304 response generally used for conditional responses to indicate that the request had no effect. I haven't been able to find any examples of this being done, so I figured I'd ask here and see if anyone else is doing this or has an opinion about it.
After some consideration, I've decided to stick to a normal 200 response with the unchanged resource entity. My initial intent was to provide a concise way to indicate to the client that the resource was not modified. As I thought more about it I realized that in order to do anything useful with the 304 response, they would have to already have a cached version and in that case it would be trivial to compare the version of the cached copy with the version returned in a 200 response.
I have a REST API that allows modification of resources using HTTP POST. It's possible that a client may submit a POST request that results in no modification of the resource.
HTTP POST in the RESTful approach implies a creation of a resource, not a modification. For modification you should use HTTP PUT.
Solution of your problem is HTTP Status 200 OK when something was modified and HTTP Status 204 No Content when there was no modification. According to:
The common use case is to return 204 as a result of a PUT request, updating a resource, without changing the current content of the page displayed to the user. If the resource is created, 201 Created is returned instead. If the page should be changed to the newly updated page, the 200 should be used instead.
-- MDN web docs
For example:
-- Request
POST /people HTTP/1.1
Content-Type: application/json
{
"name": "John"
}
-- Response
HTTP/1.1 201 Created
Location: /people/1
-- Request
PUT /people/1 HTTP/1.1
Content-Type: application/json
{
"name": "John"
}
-- Response
HTTP/1.1 204 No Content
-- Request
PUT /people/1 HTTP/1.1
Content-Type: application/json
{
"name": "Robert"
}
-- Response
HTTP/1.1 200 OK
Content-Type: application/json
{
"name": "Robert"
}
I need to test some client application code I've written to test its' handling of various status codes returned in an HTTP response from a web server.
I have Fiddler 2 (Web Debugging Proxy) installed and I believe there's a way to modify responses using this application, but I'm struggling to find out how. This would be the most convenient way, as it would allow me to leave both client and server code unmodified.
Can anyone assist as I'd like to intercept the HTTP response being sent from server to client and modify the status code before it reaches the client?
Any advice would be much appreciated.
Ok, so I assume that you're already able to monitor your client/server traffic. What you want to do is set a breakpoint on the response then fiddle with it before sending it on to the client.
Here are a couple of different ways to do that:
Rules > Automatic Breakpoints > After Responses
In the quickexec box (the black box at the bottom) type "bpafter yourpage.svc". Now Fiddler will stop at a breakpoint before all requests to any URL that contains "yourpage.svc". Type "bpafter" with no parameters to clear the breakpoint.
Programmatically tamper with the response using FiddlerScript. The best documentation for FiddlerScript is on the official site: http://www.fiddler2.com/Fiddler/dev/
Once you've got a response stopped at the breakpoint, just double click it to open it in the inspectors. You've got a couple of options now:
Right next to the green Run to Completion button (which you click to send the response) there's a dropdown that lets you choose some default response types.
Or, on the Headers inspector, change the response code & message in the textbox at the top.
Or, click the "Raw" inspector and mess with the raw response to do arbitrary things to it. Also a good way to see what your client does when it gets a malformed response, which you'll probably test accidentally :)
Another alternative is to use Fiddler's AutoResponder tab (on the right-hand panel). This allows you to catch a request to any URI that matches a string and serve a "canned" response from a file. The file can contain both headers and payload. The advantage of this approach is that you don't have to write FiddlerScript and you don't have to handle each request manually via a breakpoint.
You would set the rule up in Fiddler like shown below (ensure you enable unmatched requests passthrough otherwise all other http requests will fail).
In this example, any request whose URI includes "fooBar" will get the canned response. The format of the file will vary depending on your APIs (you can use your browser to intercept a "real" response and base it on that) but mine looked like the following:
HTTP/1.1 409 Conflict
Server: Apache-Coyote/1.1
X-Powered-By: Servlet 2.5; JBoss-5.0/JBossWeb-2.1
Access-Control-Allow-Origin: *
Access-Control-Allow-Methods: GET, POST, DELETE, PUT, PATCH, OPTIONS
Access-Control-Allow-Headers: Origin, X-Requested-With, Content-Type, Accept, Authorization
Access-Control-Max-Age: 86400
Content-Type: application/vnd.api+json
Content-Length: 149
Date: Tue, 28 Mar 2017 10:03:29 GMT
{"errors":[{"code":"OutOfStock","detail":"Item not in stock","source":{"lineId":{"type":"Order line Number","id":"1"}},"meta":{"availableStock":0}}]}
I found that it needed a carriage return at the end of the last line (i.e. after the json), and that the Content-Length header had to match the number of characters in the json, otherwise the webapp would hang. Your mileage may vary.
Create a FiddlerScript rule. Here's what I used in order to generate a local copy of a website that was intentionally using 403 on every page to thwart HTTrack/WGET.
https://gist.github.com/JamoCA/22db8d68a9a2fb20cb04a85360185333
/* 20180615 Fiddler rule to ignore all 403 HTTP Status errors so WGET or HTTrack can generate local copy of remote website */
SCENARIO: Changing the user agent or setting a delay isn't enough and the entire remote server is configured to respond w/403.
CONFIGURE: Add below rule to FiddlerScript OnBeforeReponse() section. Configure HTTrack/WGET/CRON to use proxy 127.0.0.01:8888 */
static function OnBeforeResponse(oSession: Session) {
if (oSession.HostnameIs("TARGETHOSTNAME_FILTER.com") && oSession.responseCode == 403) {
oSession.responseCode = 200;
oSession.oResponse.headers.HTTPResponseCode = 200;
oSession.oResponse.headers.HTTPResponseStatus = "200 OK";
}
}
I would like to create a cacheable HTTP response for a POST request.
My actual implementation responds the following for the POST request:
HTTP/1.1 201 Created
Expires: Sat, 03 Oct 2020 15:33:00 GMT
Cache-Control: private,max-age=315360000,no-transform
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Content-Length: 9
ETag: 2120507660800737950
Last-Modified: Wed, 06 Oct 2010 15:33:00 GMT
.........
But it looks like that the browsers (Safari, Firefox tested) are not caching the response.
In the HTTP RFC the corresponding part says:
Responses to this method are not cacheable unless the response includes appropriate Cache-Control or Expires header fields. However, the 303 (See Other) response can be used to direct the user agent to retrieve a cacheable resource.
So I think it should be cached. I know I could set a session variable and set a cookie and do a 303 redirect, but I want to cache the response of the POST request.
Is there any way to do this?
P.S.: I've started with a simple 200 OK, so it does not work.
I'd also note that caching is always optional (it's a MAY in the HTTP/1.1 RFC). Since under most circumstances, a successful POST invalidates a cache entry, it's probably simply the case that the browser caches you're looking at just don't implement caching POST responses (since this would be pretty uncommon--usually this is accomplished by formatting things as a GET, which it sounds like you've done).
Short answer: POST caching rarely makes sense. A cache may serve GET requests to a URL which is the same as that of a previous POST, whose response came with a Content-Location header containing the POST's request URI.
From rfc-7231 (http-bis, superseding rfc-2616):
Responses to POST requests are only cacheable when they include
explicit freshness information (see Section 4.2.1 of [RFC7234]).
However, POST caching is not widely implemented. For cases where an
origin server wishes the client to be able to cache the result of a
POST in a way that can be reused by a later GET, the origin server
MAY send a 200 (OK) response containing the result and a
Content-Location header field that has the same value as the POST's
effective request URI (Section 3.1.4.2).
See also Mark Nottinghams Blog:
POSTs don't deal in representations of identified state, 99 times out
of 100. However, there is one case where it does; when the server goes
out of its way to say that this POST response is a representation of
its URI, by setting a Content-Location header that's the same as the
request URI. When that happens, the POST response is just like a GET
response to the same URI; it can be cached and reused -- but only for
future GET requests.
The rfc also describes a PRG sequence which has a similar effect, allowing the response cycle to a POST to fill the cache for a subsequent GET - which is probably more widely implemented.
Can you try to change the Cache-Control to public instead of private and see if it's working?