Some hours my web site's server has too much load.
Which HTTP status code should I send to the Googlebot that visits my website?
Is "269 Call Back Later" this suitable for this case, or 503 Service Unavailable or do you have any more suggestions?
503 means the service is temporarily unavailable so it is appropriate to use while the server is overloaded. http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
The Wikipedia article defines 269 as the initial response for a request that must be processed asynchronously. 269 means the request added something to the server's "queue" of things to do and the server will have a response available as soon as possible, whereas 503 means the server will not "remember" the request.
I've never heard of 269, though, and the first Google result for it is this question, followed by the Wikipedia article.
Another option is a 429 - Too Many Requests response.
Defined in RFC6585 - https://www.rfc-editor.org/rfc/rfc6585#section-4
The spec does not define how the origin server identifies the user, nor how it counts requests.
For example, an origin server that is limiting request rates can do so based upon counts of requests on a per-resource basis, across the entire server, or even among a set of servers.
Likewise, it might identify the user by its authentication credentials, or a stateful cookie.
Also see the Retry-After header in the response.
Related
Does/should server code run when the server return status code 304?
I understand that the server should not return anything (client should use the cache), but I cant find any info on whether the server will executed the code in an api endpoint for example.
RFC 2616 section 10.3.5 describes the 304 Not Modified response:
If the client has performed a conditional GET request and access is
allowed, but the document has not been modified, the server SHOULD
respond with this status code. The 304 response MUST NOT contain a message-body
The server will send Date, ETag and/or Content-Location (200 only), or Expires, Cache-Control, and/or Vary if the respond may vary.
What is a Conditional GET?
An cliet application, browser, or proxy with a retained Last-Modified or Etag value will issue a Conditional GET as an initial header only. This allows the client to determine if the resource has been updated.
How does the client know if the resource changed?
Well it depends on how the server is configured.
The Origin Server May:
Ignore caching, serve every request new.
Similar to ignoring, you may develop the application to change query strings. This prevents caching at Proxy Servers and invalidates the client cache.
If configured to do so, issue a Last-Modified or Etag value. Often done for static content. Proxy Servers and Client Caches use to invalidate their version.
A Web application could issue a Last-Modified far
into the future then change the URL to invalidate stale content. This requires the application be developed with this feature in mind.
Resources may also be issued version numbers. This allows them to preserve Proxy Caches but invalidate Client Caches.
Does server code run when the server return status code 304?
Almost Never. Disregarding that it is technically possible for an incorrectly configured application to respond with a 304 Not Modified code instead of a 200 code.
With a ETag and/or Content-Location value, a server (nginx for example) can confirm nothing has changed without issued a call to the application. This also neatly handles resources with version numbers the same way.
For query strings (image.jpg?version=12), the client cache will invalidate the content. A Proxy Server will also invalidate, and the query will be requested fresh.
I understand that the server should not return anything (client should use the cache), but I cant find any info on whether the server will executed the code in an api endpoint for example.
I'm a fan of nginx, here is a good resource on how caching applies to it.
In short, as much as you can do to support various caches between your client and the application the more requests you can support per day.
I have a health check endpoint at /status that returns the following status codes and response bodies:
Healthy - 200 OK
Degraded - ?
Unhealthy - 503 Service Unnavailable
What should the HTTP status code be for a degraded response be? A 'degraded' check is used for checks that did succeed but are slow or unstable. What HTTP status code makes the most sense?
The most suitable HTTP status code for a "Degraded" status response from a health endpoint is nothing other than 200 OK.
I say this because I can't find any better code in the official Hypertext Transfer Protocol (HTTP) Status Code Registry maintained by IANA, pointed to by [RFC7231] HTTP/1.1: Semantics and Content. Unofficial codes should be avoided, because they only make your API more difficult to understand.
You should design your APIs so that they become easy to use. Resource names, HTTP verbs, status codes, etc. should be more or less self-explanatory, so that people who already know "the REST language" can immediately understand how to use your API without having to decipher vague names or unusual status codes. Which brings me to the next part of my answer...
Other comments on your design
The most natural way to interpret a 5xx response to any request is that the operation in question failed.
So a 503 Service Unavailable response to a GET /status request means that the status checking operation itself failed. Such a response would only be useful if we can be certain that /status is a health endoint, as pointed out in the API Health Check draft referred to in Nkosi's answer:
A health endpoint is only meaningful in the context of the component
it indicates the health of. It has no other meaning or purpose. As
such, its health is a conduit to the health of the component.
Clients SHOULD assume that the HTTP response code returned by the
health endpoint is applicable to the entire component (e.g. a larger
API or a microservice).
But with a URL path of just /status, it is not completely obvious that this really is a health endpoint. From looking at the URL, we only know that it returns information about the status of something, but we can't really be sure what that "something" is.
Since you're also telling us that yes, it is in fact a health endpoint, I must suggest that you change the name to health. I would also suggest placing it under some base path, e.g. /things/health, to make it more clear which component it indicates the health of.
If, on the other hand, /status was actually a resource of it own, i.e. something that represents the status of some other component/thing (like its name currently suggests), then 200 OK is the only reasonable status for successful invocations, even if the thing that it indicates the status of is "Unhealthy". In that case, a 5xx would mean that no status could be obtained, and details in the response payload would be assumed to be related to a failure in the /status service itself.
So be careful with how you name things and what status codes you use!
Consider returning a custom code within the 2xx Success range that is not already taken within the known/common status codes. Similar to some of the unofficial codes not supported by any standard.
For example 218 This is fine (Apache Web Server)
Used as a catch-all error condition for allowing response bodies to flow through Apache when ProxyErrorOverride is enabled. When ProxyErrorOverride is enabled in Apache, response bodies that contain a status code of 4xx or 5xx are automatically discarded by Apache in favor of a generic response or a custom response specified by the ErrorDocument directive
After doing some research I came across a draft
Health Check Response Format for HTTP APIs: draft-inadarei-api-health-check-03
Where they also made similar suggestions
In case of the “warn” status, endpoints MUST return HTTP status in the 2xx-3xx range, and additional information SHOULD be provided, utilizing optional fields of the response.
where the warn status in the draft is healthy, with some concerns, which I believe aligns closely to your desired model.
While not definitive, I believe it provides some ideas to help with the eventual design.
I would be wary of splitting hairs like this on a healthcheck on the upstream server side. The service providing the healthcheck should be lightly (and concurrently) testing all its upstream dependencies based on its own set of policies or rules - request timeouts, connection failures and so on. In reality the healthcheck either works or it doesn't and the application shouldn't really need to be keeping track of the results of the healthcheck (other than capturing metrics about what happened). IMHO a stateful healthcheck is a recipe for disaster.
I typically use the following interface for application healthchecks:
204 - No Content, everything is working within tolerences
500 - Something failed, and here's some details in the response about what went wrong
Where it gets tricky depends on your architecture. You may have a VIP or reverse proxy that is interpreting this response and deciding if a given node is healthy or not, in which case it's going to either route the request to a healthy node or return the 503 Service Unavailable. This decision is going to made on some policy basis - x healthcheck requests failed over a y time period across z upstream services.
If you use a mesh then everyone can feed data back to the service registry to keep the health state up to date and it can be based on actual service calls rather than a healthcheck.
The client is perfectly placed to make a decision based on the health of services it depends on as they can keep track of the various responses from the service. Circuit breakers are an excellent way to handle that and can do it continuously on actual requests rather than just on the healthcheck. Circuit breaker libraries (such as resilience4j) will do this for you at the cost of setting up some policies about how many failed/slow requests constitute a bad service. Service Registrys like the netflix eureka can help with the discovery and ongoing monitoring.
Assuming you are referring to the status code of a liveness/healthcheck endpoint of a service - to distinguish from 200 OK a 203 likely seems applicable and in line with:
https://datatracker.ietf.org/doc/draft-inadarei-api-health-check/
https://www.rfc-editor.org/rfc/rfc7234#section-5.5 despite being deprecated Warning: 199-header MAY carry details
align max-age with livenessProbe.periodSeconds
HTTP/1.1 203 Non-Authoritative Information
Warning: 199 - "FooBar Warning Details"
Content-Type: application/health+json
Cache-Control: max-age=10
Connection: close
{"status": "warn"}
RFC 2616 says 502 means: "The server, while acting as a gateway or proxy, received an invalid response from the upstream server it accessed in attempting to fulfill the request."
What exactly counts as an invalid response? Does this include something like a 500 from the upstream server?
Imagine that you have implemented and API that aggregates responses from a couple of third party servers. E.g. you have an application that if asked for info about lol#lol.com user will return all the public data about this user from FB, TW and G+.
More or less your API acts as a proxy server. Now if one of these requests fails you have a choice what will be returned to the client. If you want to be explicit about how your application works you can return 502 Bad Gateway along with message: Sorry, request to FB failed for whatever reason and that's all. But you can also hide the implementation details and return a e.g. 409 Conflict response along with details: Have no info about the user's FB activity.
In this particular example 207 Multi Status would be also very useful.
I'm implementing a /_status/ endpoint which does some sanity checks on data in our database.
For example, we are collecting measurements and the status should go "bad" if the latest measurement is over an hour old.
I would like to point Pingdom at this URL to leverage their alerting infrastructure and tell us when something's wrong.
On a "good" status I will serve an HTML page with an HTTP 200 OK status. But what would an appropriate HTTP status code be for "bad"? Or would it be more correct not to convey this information via status code, but via HTML content instead?
Thanks!
Well... this is an old question, but I ended up here, so I thought I'd give my two cents here:
It seems pretty clear that a 2xx should be returned if all is OK
If health is not OK, I think it should return a 5xx result (4xx talks about the client being at fault in the request; 2xx and 3xx are all successful to some degree).
I think that a 5xx is correct because this is a special request that is answering about the state of the whole service. Also, because most Load Balancers offer liveliness checks based on response codes and not all offer a way to parse a more complex payload (other than perhaps a RegExp Match which can make the check brittle).
I agree with #Julien that a 500 (specifically) doesn't seem appropriate, and we've decided on 503 Service Unavailable.
503 seems to fit for a couple of reasons:
It's a 5xx family result code which indicates that something is going on on the server side.
It has a temporary nature to it indicating that it may recover.
We just had a similar discussion in our group. We decided for our purposes that the HTTP response codes should be reporting on your server's success or failure to honor the request. For a GET, this would mean whether or not you can respond with the requested resource. In this case, the requested resource is a health report, so as long as you're returning that successfully, it should be a 200 response.
We're returning JSON for our health check, with a top-level "isHealthy" field set to true or false. Our load balancer and other monitors will parse the JSON and use this field to determine if the system is healthy or not.
If you don't want to parse JSON in your monitors, you could try putting a custom response header to indicate binary health of the system, e.g., System-Health: true or System-Health: false. You might have better luck getting monitors which can check that.
If you really want to use a response code, I would recommend an additional endpoint called something like "health" which returns a "204 No Content" when healthy, and a "404 Not Found" when not healthy. In this case, the resource defined by the URL is, symbolically, the health of your system, and so if it's healthy, you can return a successful response. If it's unhealthy, then it's health can't be found, hence the 404.
If your data is 'bad' because there is a service failure (even if that is a backend job failing) then a HTTP 500 seems like a valid response. It indicates that something, somewhere is broken.
It isn't very specific, you're shrugging your shoulders and saying:
The 500 (Internal Server Error) status code indicates that the server
encountered an unexpected condition that prevented it from fulfilling
the request.
ietf rfc7231
If you ask for health and the server state is not healthy, I'm partial to 409 Conflict which "Indicates that the request could not be processed because of conflict in the current state of the resource" .
Some people might object that if you can respond then the request can be processed, but I disagree. Every error message is a response. The server defines resource semantics. If you ask for the good news resource and the server responds "here is bad news", it didn't give you what it defines to have offered at that resource.
In practice, it's much easier to say 2**="up" 4**="down" and pipe request counts into an availability metric and have a load balancer remove the server from its pool based on the response code. Coming up with ways to argue that "hey, we told you something, so 200 OK" just seems like missing the forrest for the trees to me.
I cannot find any RFC or Standard of HTTP client behavior in case it gets HTTP response with an error 4xx. I know the 401, 407 are the examples when the HTTP headers are parsed, but...
I have the concrete problem for OPTIONS method (HTTP1.1). The server responses 401 Unauthorized, so client tries to authenticate and re-sends the request with an authentication. After that the response has the error 404 Not Found and HTTP header is filled with Set-Cookie HTTP Header. The client use Apache Java HTTPClient/HTTPComponents, which ignores HTTP headers in case of an error in the response.
Should this HTTP Header be accepted by the client? I believe it should not be, but I cannot find the supportive quotation in the RFC.
RFC 2616 does not specify that any headers should be ignored, not for 404 responses and not for 4xx responses in general either.
RFC 6265 allows clients to ignore Set-Cookie headers, but does not specify situations where that might happen; a single example is given, that does not cover your case:
the user agent might wish to block responses to "third-party" requests
from setting cookies
In your case, since your server seems to use HTTP basic access authentication, it does not seem to concern the Set-Cookie header. In HTTP basic authentication, the Authorization header is sent by the client with every request, so there should be no need to keep state in a cookie.
It is not clear from your question if you have a very specific HTTP server that you're talking to, or if you are implementing a general HTTP client that is supposed to work with whatever server you throw it at. If you have such a specific case that the HTTP server you work with sends state with 404 responses, and you're required to honor that state in order to communicate with the server, and you have no control over the server, then it does not matter what the standard says; you will honor the state sent, or you will not be able to talk to the server.
If, on the other hand, you're implementing a general client and need it to work regardless of the remote server, then your best bet is to stick to RFC 1958:
Be strict when sending and tolerant when receiving.
Implementations must follow specifications precisely when sending to
the network, and tolerate faulty input from the network. When in
doubt, discard faulty input silently, without returning an error
message unless this is required by the specification.
Which, to me, would mean that you should honor the full response received, regardless of the status code, unless you have an objective reason making it impossible for you to do so. I don't see a reason to ignore the state, even if it violates the standard (or in this case, your personal perception of the standard, since it does not say anything about accepting or ignoring the state).
Update: RFC 2617 (HTTP Authentication) states:
A client SHOULD assume that all paths at or deeper than the depth of
the last symbolic element in the path field of the Request-URI also
are within the protection space specified by the Basic realm value of
the current challenge. A client MAY preemptively send the
corresponding Authorization header with requests for resources in
that space without receipt of another challenge from the server.
It is highly inconsistent if the server expects HTTP authentication for one URL, but does not honor it for URLs beneath it, requiring a separate cookie-based authentication for them. If anything should be changed in your server implementation, it should be to harmonize the authentication scheme for all resources.