Why 204 are not cached by OkHttp? - http

Looking at the code in CacheInterceptor I see that response with the code 204 are not cached. Yet I believe 204 are cacheable as discussed here
We use 204 as a response to GET don't indicate an empty response and just recently noticed those are not cached.

There are a number of reasons why.
Why does the implementation behave like this?
Technically, 204s are skipped because the intercept method will only cache responses if promisesBody returns true:
if (response.promisesBody() && CacheStrategy.isCacheable(response, networkRequest)) {
// Offer this request to the cache.
As you might expect, promisesBody returns false for 204 responses, unless they have a Content-length header, or chunked transfer encoding:
/**
* Returns true if the response headers and status indicate that this response has a (possibly
* 0-length) body. See RFC 7231.
*/
...
val responseCode = code
if ((responseCode < HTTP_CONTINUE || responseCode >= 200) &&
responseCode != HTTP_NO_CONTENT &&
responseCode != HTTP_NOT_MODIFIED) {
return true
}
// If the Content-Length or Transfer-Encoding headers disagree with the response code, the
// response is malformed. For best compatibility, we honor the headers.
if (headersContentLength() != -1L ||
"chunked".equals(header("Transfer-Encoding"), ignoreCase = true)) {
return true
}
return false
Why was this design used?
To justify this decision, it's possible that OkHttp's general use case for a cache is to save re-sending large payloads, rather than keeping round-trips to the absolute minimum. In addition, 204 is valid as a response to a GET, but more often used as a response to a POST; it's possible that the design reflects knowledge of this heuristic.
Why hasn't this changed?
Looking through relevant issues, there is one related to changing the default behaviour for certain status codes: OkHttp cache-by-default status codes out of date:
The spec wants these:
200, 203, 204, 206, 300, 301, 404, 405, 410, 414, and 501
which resulted in a test:
assertCached(true, 204);
However, due to the configuration of the mock server, the response used to test is:
HTTP/1.1 204 OK
...
Content-Length: 0
with exactly the unnecessary Content-Length that its own detection requires, and which a valid response should not include.
tl;dr
It's a bug, which nobody who's run into this case has reported.

A 204 is returned when there is no content - "The server successfully processed the request and is not returning any content."
Therefore there is no data to cache if using the code to spec. If you are returning content/data than a 200 would be more appropriate.

Related

Cache-Control & Etag not honored on Google Chrome

I've implemented the HTTP Caching guidelines described by Google in a URL Shortening API that I've developed.
Here's how I'm sending the response:
const urlResponseCacheControlMaxAge = 172800 // 2 days
type urlResponse struct {
LongURL string `json:"longUrl"`
ShortURL string `json:"shortUrl"`
}
func (u urlResponse) Hash() string {
parts := strings.Split(u.ShortURL, "/")
return parts[len(parts)-1]
}
func sendURLResponse(w http.ResponseWriter, req *http.Request, urlResponse *urlResponse) {
if eTag, ok := req.Header["ETag"]; ok && urlResponse.Hash() == eTag[0] {
w.WriteHeader(http.StatusNotModified)
io.WriteString(w, "")
return
}
cacheControl := fmt.Sprintf(
"max-age:%d, public",
urlResponseCacheControlMaxAge,
)
w.Header().Set("Cache-Control", cacheControl)
w.Header().Set("Content-Type", "application/json;charset=utf-8")
w.Header().Set("ETag", urlResponse.Hash())
w.WriteHeader(http.StatusOK)
encoder := json.NewEncoder(w)
err := encoder.Encode(urlResponse)
if err != nil {
SendError(w, NewError(
URLResponseEncoding,
"Error encoding response",
map[string]string{"error": err.Error()},
))
return
}
}
Basically, when the browser sends a request to the API (using GET), I return an ETag and Cache-Control header in the response; the Cache Control header sets a max age of two days.
What I expect to happen is that in subsequent requests, the browser uses the cached response. After 2 days have elapsed, the browser should send the ETag in the request header to check if the response has changed.
However, what I'm observing is that each time I click on the submit button, the browser resends the request. On Google Chrome Developer Console, I've unchecked 'Disable Caching' and yet it still sends requests each time.
Whatsmore is that the browser is not sending the ETag back with the request headers.
Is there something that I'm missing that's causing the cache to not work as expected?
cacheControl := fmt.Sprintf(
"max-age:%d, public",
The Cache-Control header must contain the caching time with max-age=... not max-age:... as you use (= vs :). The value you've tried to set in the wrong way will be simply ignored.
Whatsmore is that the browser is not sending the ETag back with the request headers.
if eTag, ok := req.Header["ETag"]; ok && urlResponse.Hash() == eTag[0] {
w.WriteHeader(http.StatusNotModified)
The browser will not send etag back within an ETag header. This header is only used to set the etag. The browser will instead ask the server to provide the resource if it was modified compared and will do this by putting the received etag into a If-None-Match: ... header - meaning: please return the resource if it does not match the given etag.
For ETags to be used and the browser to send the If-Modified-Since and If-None-Match headers, the cache control must be set to no-cache.
Either by the server using the Cache-Control: no-cache header or by the browser through the Request.cache option.

HTTP status codes when errors in the request body

Can someone please clarify what status codes should I expect from following situations? For instance, I am sending POST request with such body format:
{
"id": 321,
"username": "tombrown",
"email": "tombrown#gmail.com",
"password": "qwerty123",
"activated": true
}
So the questions are:
1) Should the server return 400 if I specify data of wrong type, for instance, "id": “threetwoone” instead of int, "activated": “yes” instead of boolean etc. Or server should rather return 422 here?
2) The “id” value should be int, but actually it is long int, e.g. 9223372036854774700.
3) Some fields are missing in the body, e.g. I try to send:
{
"id": 321,
"username": "tombrown",
"activated": true
}
Should these examples cause 400, 422 or some other options? What reaction should be correct?
If the JSON is syntactically invalid, return 400. If JSON is syntactically valid but its content is invalid, return 422 to indicate that the request entity cannot be processed by the server.
See the following quote from the RFC 4918 (for your situation, just read JSON when it says XML):
11.2. 422 Unprocessable Entity
The 422 (Unprocessable Entity) status code means the server
understands the content type of the request entity (hence a
415 (Unsupported Media Type) status code is inappropriate), and the
syntax of the request entity is correct (thus a 400 (Bad Request)
status code is inappropriate) but was unable to process the contained
instructions. For example, this error condition may occur if an XML
request body contains well-formed (i.e., syntactically correct), but
semantically erroneous, XML instructions.

Apache CXF WebClient multiple requests with www-authenticate header

I got simple JAX-RS resource and I'm using Apache CXF WebClient as a client. I'm using HTTP basic authentication. When it fails on server, typical 401 UNAUTHORIZED response is sent along with WWW-Authenticate header.
The strange behavior happens with WebClient when this (WWW-Auhenticate) header is received. The WebClient (internally) repeats the same request multiple times (20 times) and than fails.
WebClient webClient = WebClientFactory.newClient("http://myserver/auth");
try {
webClient.get(SimpleResponse.class);
// inside GET, 20 HTTP GET requests are invoked
} catch (ServerWebApplicationException ex) {
// data are present when WWW-authenticate header is not sent from server
// if header is present, unmarshalling fails
AuthError err = ex.toErrorObject(webClient, AuthError.class);
}
I found the same problem in CXF 3.1.
In my case for all async http rest request if response came 401/407, then thread is going in infinite loop and printing WWW-Authenticate is not set in response.
What I analysed the code I found that :
In case of Asynchronous call Control flow from HttpConduit.handleRetransmits-> processRetransmit-> AsyncHTTPConduit.authorizationRetransmit
which return true and in HttpConduit the code is
int maxRetransmits = getMaxRetransmits();
updateCookiesBeforeRetransmit();
int nretransmits = 0;
while ((maxRetransmits < 0 || nretransmits < maxRetransmits) && processRetransmit()) {
nretransmits++;
}
If maxRetransmits = -1 and processRetransmit() return true then thread going in infinite loop.
So to overcome this issue we pass maxRetransmitValue as 0 in HttpConduit.getClient().
Hope it will others.
This has been fixed in the latest versions of CXF:
https://issues.apache.org/jira/browse/CXF-4815

How to return error pages with body using HttpListener/HttpListenerResponse

I'm in the process of creating a REST API using HttpListener in .NET (C#). This all works out great, except for one slight issue.
I'm trying to return responses with Status Codes other than OK (200), for instance ResourceNotFound (404).
When I set the StatusCode of the HttpListenerResponse to something other than 200, and create a response body (using HttpListenerResponse.OutputStream), it seems to be resetting the status code to 200. I'm not able to send a response with StatusCode 404 and a message body. However, this should be possible according to the HTTP specs. I'm checking the requests and responses with Fiddler, but I'm not able to get what I'm looking for.
I've had the same problem and found the source of the problem :
If you write the body in the OutputStream before set the StatusCode (or any other property), the response will be sent before the modification is applied !
So, you have to proceed in this order :
public void Send(HttpListenerContext context, byte[] body)
{
// First, set a random status code and other stuffs
context.Response.StatusCode = (int)HttpStatusCode.Unauthorized;
context.Response.ContentType = "text/plain";
// Write to the stream IN LAST (will send request)
context.Response.OutputStream.Write(body, 0, body.Length);
}

Caching of Web Service not working when request has query string

I'm trying to implement the client-side caching of web service calls, and based on information from the web, I was able to do it according to the SetCachingPolicy() function as shown in code 1 below.
I was able to successfully get it working with a web method, RetrieveX, but not with method RetrieveY. I noticed that RetrieveX has no parameters and RetrieveY has one string parameter, and on inspection under Fiddler, the difference seems to be that the HTTP GET request of the web service call from RetrieveY has a query string for the parameter.
All HTTP GET web service calls so far without a query string is doing the caching properly, but not this call that has a query string in it.
Examination under Fiddler indicates that RetrieveX has the following caching information in output 1, and RetrieveY has the information in output 2.
Is this a limitation of this caching method or can I do something to get the client side caching of RetrieveY working?
Code 1: SetCachingPolicy
private void SetCachingPolicy()
{
HttpCachePolicy cache = HttpContext.Current.Response.Cache;
cache.SetCacheability(HttpCacheability.Private);
cache.SetExpires(DateTime.Now.AddSeconds((double)30));
FieldInfo maxAgeField = cache.GetType().GetField(
"_maxAge", BindingFlags.Instance | BindingFlags.NonPublic);
maxAgeField.SetValue(cache, new TimeSpan(0, 0, 30));
}
Code 2: RetrieveX
[System.Web.Services.WebMethod]
[System.Web.Script.Services.ScriptMethod(UseHttpGet = true)]
public string[] RetrieveX()
{
SetCachingPolicy();
// Implementation details here.
return array;
}
Code 3: RetrieveY
[System.Web.Services.WebMethod]
[System.Web.Script.Services.ScriptMethod(UseHttpGet = true)]
public string[] RetrieveY(string arg1)
{
SetCachingPolicy();
// Implementation details here.
return array;
}
Output 1: RetrieveX caching info
HTTP/200 responses are cacheable by default, unless Expires, Pragma, or Cache-Control headers are present and forbid caching.
HTTP/1.0 Expires Header is present: Wed, 12 Sep 2012 03:16:50 GMT
HTTP/1.1 Cache-Control Header is present: private, max-age=30
private: This response MUST NOT be cached by a shared cache.
max-age: This resource will expire in .5 minutes. [30 sec]
Output 2: RetrieveY caching info
HTTP/200 responses are cacheable by default, unless Expires, Pragma, or Cache-Control headers are present and forbid caching.
HTTP/1.0 Expires Header is present: -1
Legacy Pragma Header is present: no-cache
HTTP/1.1 Cache-Control Header is present: no-cache
no-cache: This response MUST NOT be reused without successful revalidation with the origin server.
I ran into this issue as well, I thought I'd share what worked for me. The underlying issue is that VaryByParams is not being set on the response. If you add this to your SetCachingPolicy() method RetrieveY should begin working as expected:
cache.VaryByParams["*"] = true

Resources