I'm designing an RESTful API where some calls are public over HTTP, and some require an API key and encryption over HTTPS. I'm deliberating on what response code should be sent if an HTTP request is sent to one of the private resources. So far the only one that jumps out at me is 412 - Precondition Failed, but the standard indicates that the precondition is imposed by the requester not the server.
Is there an appropriate response code for this condition or do I just need to give in and do 400?
I cannot say if this is broadly accepted by HTTP clients, but speaking strictly RFC, the server should respond with:
HTTP/1.1 426 Upgrade Required
Upgrade: TLS/1.0, HTTP/1.1
Connection: Upgrade
Source:
https://www.rfc-editor.org/rfc/rfc2817#section-4.2
The most secure way to force HTTP client to use HTTPS is HTTP Strict Transport Security.
Previously a common suggestion was to drop the connection, but this practice has been removed in favor of HSTS (OWASP website).
The appropriate error code to return would be similar to 403.4 - SSL required.
Although not explicitly documented in the RFC for HTTP 1.1, this behavior does match the requirements outlined there:
The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.
Adding your own subcode (as with the SSL example) might be helpful in some cases, but since this subcode would not be meaningful to third parties, I would recommend against it.
So, your final error message would be something like "403 - Private Resource". Note that, even in the case of a missing API key, "401 - Unauthorized" should not be used, unless your API key can actually be transmitted in a WWW-Authenticate header field.
Returning a 403 with reason phrase "HTTPS Required" seems like a practical option and what I use.
see https://en.wikipedia.org/wiki/HTTP_403
Redirecting a REST Api is not a good idea especially as you may have no idea as to how or what is consuming your service.
Just send a redirect to the corresponding https: URI.
UPDATE
The is a wrong answer - see comments below
Related
I'm trying to determine what a client should do with headers on receiving a 303 (See Other) from the server. Specifically, what should be done with the Authorization header that was sent on the initial request?
Here's the problem: the client makes a request to myserver.com (HTTP request method is not relevant here) and the server at myserver.com responds with a 303 and the Location header contains otherserver.com/some_resource/. Tools like Postman and curl will follow the redirect by passing all the same headers in the subsequent request to otherserver.com. I haven't found a way to make these tools drop the headers.
In the case I've described, sending the Authorization header to otherserver.com seems like a security risk: otherserver.com now knows my token and possibly what host it can be used on so now the token is compromised. This can also cause errors, depending on how the destination host is configured. In the case where the redirect is to another resources on the same host (ie, myserver.com) then the Authorization header will (probably) need to be sent, and because it's the same host nothing is compromised.
Effectively, in different situations it seems that the correct behaviour is different. The relevant section in the RFC does not address this issue. In developing my own API, I've written documentation telling API clients to drop the Authorization header on redirect to otherserver.com. However, based on mucking around with curl and Postman, it's not clear to me either (a) what the default behaviour is for a typical HTTP client library or (b) whether HTTP client libraries permit easy modification of the HTTP headers before following a 303 redirect. As a result, it's possible my suggestion isn't practical. I also know of no way for the server to instruct the client as to what it should do with headers on following the 303 redirect.
What should a HTTP client do with the headers when it follows a 303 redirect? Who is responsible for deciding whether to use the same headers on the redirect, the HTTP client or server?
You can argue that when sending the 303 with otherserver.com's Location, myserver.com trusted otherserver.com to handle your token. It could have sent the token in the background as well. From the client's perspective, the client trusts myserver.com to handle the token, store and verify it securely, etc. If myserver.com decides to send it on to otherserver.com, should the client override? In this case it can of course, but in general I don't think it should.
As an attacker does not control the response headers from myserver.com which is a legit resource, I think in general it is secure to send the token by default to the other server it specifies, maybe unless you have some good reason not to (say an explicit policy on the client).
How does an HTTP client definitively determine whether a login was successful when using Basic Authentication? I've always thought that it was simple - look at the response code:
200 = Successful
401 = Unsuccessful
This is not correct. If you read RFC 2617 it says:
If the origin server does not wish to accept the credentials sent with
a request, it SHOULD return a 401 (Unauthorized) response. The
response MUST include a WWW-Authenticate header field containing at
least one (possibly new) challenge applicable to the requested
resource.
The word SHOULD is tricky, and in fact I've seen devices that do not return a 401 on login failure. This would suggest:
WWW-Authenticate does not exist = Successful
WWW-Authenticate exists = Unsuccessful
This is also incorrect. I'm playing with a TP-LINK router that provides the following headers:
- Server : Router Webserver
- Connection : close
- Content-Type : text/html
- WWW-Authenticate : Basic realm="TP-LINK Wireless N Router WR841N"
The response code is 200.
Here we have a 200 with a WWW-Authenticate!!!!!
Given all of this, what is the DEFINITIVE way to determine whether a BA login was successful?
RFC 7235 obsoletes RFC 2617 and one of the corrections it brings is to point out that, SHOULD is to be interpreted as per RFC 2119. (This was perhaps omitted because it is pointed out in RFC 2616, which goes hand-in-hand with 2617):
SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.
So, it's not just a matter of "well, you should do that, but we understand if you don't" as should means colloquially, but rather "you must do this, unless you have an extremely good reason that you can clearly state".
If a server has a very good reason for not sending a 401, it's most likely because it is sending some other error code (e.g. 404 to pretend the resource doesn't exist unless you've successfully authenticated). The wisdom of sending anything other than 401 is perhaps questionable, but the only reason to send 200 is because you want to indicate that the response is successful. Really, while the should does allow something other than 401, it really doesn't allow 200.
Including WWW-Authenticate on the other hand, definitely doesn't mean the authentication wasn't successful. It's allowed at any time, and with other schemes apart from Basic can be necessary on successful requests (to allow a nonce-count to increment, for example).
In short you've got three possible states: Success, Authentication Error, Some Other Error. The should allows for the last of those. If you get a 200 then you were successful (or well, the server is behaving wrong, but that always applies anyway).
The 200 status is what counts. That the server keeps including WWW-Authenticate simply is a bug.
And yes, I tested this, I happen to have the same router over here.
Which HTTP status code would you return to indicate an unsupported browser? I've looked through the 4xx (client error) codes but none seem to fit.
403 Forbidden is the most appropriate.
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.4 says:
The server understood the request, but is refusing to fulfill it.
400 Bad Request is not a good fit because it implies the request itself is malformed, which is probably not true.
If you are building a website, however, it is not a good practice to forbid specific web browsers in this way. Try to build a website that is compliant with all the browsers used by your audience. If you are building an HTTP API, then that's a different story, and you should use a secure mechanism for authorizing clients. If that's what you need, consider OAuth.
EDIT July 2015: The newer RFC 7231 elaborates its explanation of 403, making it more clear that it is okay to use even when it is not a credential authorization issue.
https://www.rfc-editor.org/rfc/rfc7231#section-6.5.3 (bold added by me):
The 403 (Forbidden) status code indicates that the server understood the request but refuses to authorize it. A server that wishes to make public why the request has been forbidden can describe that reason in the response payload (if any).
If authentication credentials were provided in the request, the server considers them insufficient to grant access. The client SHOULD NOT automatically repeat the request with the same credentials. The client MAY repeat the request with new or different credentials. However, a request might be forbidden for reasons unrelated to the credentials.
400 (bad/malformed request) is the only one that fits... somewhat
I cannot find any RFC or Standard of HTTP client behavior in case it gets HTTP response with an error 4xx. I know the 401, 407 are the examples when the HTTP headers are parsed, but...
I have the concrete problem for OPTIONS method (HTTP1.1). The server responses 401 Unauthorized, so client tries to authenticate and re-sends the request with an authentication. After that the response has the error 404 Not Found and HTTP header is filled with Set-Cookie HTTP Header. The client use Apache Java HTTPClient/HTTPComponents, which ignores HTTP headers in case of an error in the response.
Should this HTTP Header be accepted by the client? I believe it should not be, but I cannot find the supportive quotation in the RFC.
RFC 2616 does not specify that any headers should be ignored, not for 404 responses and not for 4xx responses in general either.
RFC 6265 allows clients to ignore Set-Cookie headers, but does not specify situations where that might happen; a single example is given, that does not cover your case:
the user agent might wish to block responses to "third-party" requests
from setting cookies
In your case, since your server seems to use HTTP basic access authentication, it does not seem to concern the Set-Cookie header. In HTTP basic authentication, the Authorization header is sent by the client with every request, so there should be no need to keep state in a cookie.
It is not clear from your question if you have a very specific HTTP server that you're talking to, or if you are implementing a general HTTP client that is supposed to work with whatever server you throw it at. If you have such a specific case that the HTTP server you work with sends state with 404 responses, and you're required to honor that state in order to communicate with the server, and you have no control over the server, then it does not matter what the standard says; you will honor the state sent, or you will not be able to talk to the server.
If, on the other hand, you're implementing a general client and need it to work regardless of the remote server, then your best bet is to stick to RFC 1958:
Be strict when sending and tolerant when receiving.
Implementations must follow specifications precisely when sending to
the network, and tolerate faulty input from the network. When in
doubt, discard faulty input silently, without returning an error
message unless this is required by the specification.
Which, to me, would mean that you should honor the full response received, regardless of the status code, unless you have an objective reason making it impossible for you to do so. I don't see a reason to ignore the state, even if it violates the standard (or in this case, your personal perception of the standard, since it does not say anything about accepting or ignoring the state).
Update: RFC 2617 (HTTP Authentication) states:
A client SHOULD assume that all paths at or deeper than the depth of
the last symbolic element in the path field of the Request-URI also
are within the protection space specified by the Basic realm value of
the current challenge. A client MAY preemptively send the
corresponding Authorization header with requests for resources in
that space without receipt of another challenge from the server.
It is highly inconsistent if the server expects HTTP authentication for one URL, but does not honor it for URLs beneath it, requiring a separate cookie-based authentication for them. If anything should be changed in your server implementation, it should be to harmonize the authentication scheme for all resources.
The HTTP OPTIONS method is supposedly used to determine what other methods the server supports on a given resource. Given that, I have two questions:
What does this response look like? I have seen examples with CSV lists in Public, Allow, and even Access-Control-Allow-Methods headers. Are they all needed? What's the difference? RFC 2616 doesn't seem to be very helpful here.
Would it be appropriate to use this to list the actions that a resource supports in a non-REST-API environment? For example, if my ConversionController supports the action convert, would a response like this make sense:
Request:
OPTIONS /conversion HTTP/1.1
Response:
HTTP/1.1 200 OK
...
Allow: CONVERT
...
RFC 2616 defines "Allow" (http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.7). "Public" is not in use anymore. "Access-Control-Allow-Methods" is defined in the CORS specification (see http://www.w3.org/TR/cors/).
What is an HTTP OPTIONS request?
It is a request from the client to know what HTTP methods the server will allow, like GET, POST, etc.
Request
The request might look like this when asking about the options for a particular resource:
OPTIONS /index.html HTTP/1.1
or like this when asking about the server in general:
OPTIONS * HTTP/1.1
Response
The response would contain an Allow header with the allowed methods:
Allow: OPTIONS, GET, HEAD, POST
Why is the server receiving an HTTP OPTIONS request?
Some REST APIs need it (but if you are defining the API, you'd know that)
Browsers send it to servers as "preflighted" requests to see if the server understands CORS
Attackers send it to get more information about the API
How to respond to an HTTP OPTIONS request?
You could respond with an Allowed header and even document your API in the body.
You could respond with additional CORS defined Access-Control-Request-* headers.
You could respond with 405 Method Not Allowed or 501 Not Implemented.
How do I stop getting HTTP OPTIONS requests?
If it's coming from a browser then update your API so that it isn't doing anything "dangerous" (like PUT or DELETE, or POST with application/json). Only perform simple requests.
See also
RFC 2616 Section 9: Method definitions
MDN Web docs: OPTIONS
MDN Web docs: Cross-Origin Resource Sharing (CORS)
CORS - What is the motivation behind introducing preflight requests?
How to exploit HTTP Methods
In response to the title: "How to respond to an HTTP OPTIONS request?" To answer that, I'd want to know why you want to respond to an OPTIONS request? Who/what is sending you an OPTIONS request, and why? Many public servers respond with some form of "error" or "not allowed" (500, 501, 405). So, unless you're in a specific situation where your clients will be reasonably sending OPTIONS requests and expecting useful/meaningful information back (e.g., WebDAV, CORS), you probably want to respond with: "don't do that."
In terms of your question about the "OPTIONS /conversion HTTP/1.1" request: unless you know that there's some client of your server, a client which would send an OPTIONS request to "/conversion" and expect a response with "Allow: CONVERT," the answer is no: it wouldn't make sense to respond like that. I think that most implementations that do support OPTIONS and respond with "Allow," respond with standard HTTP methods.
Here's a great article on the topic.
Summary: OPTIONS is immediately problematic because it doesn't support caching. Alternatives: server-wide metadata: try well-known URI's. Resource-specific: try using a Link header on its responses, or a link in the representation format for that resource.
Lastly, if what you're after is a service description, have a look at WADL or RSDL.
EDIT:
dotnetguy makes a good point in the comment below: OPTIONS is undeniably valuable in certain contexts (e.g., CORS); I certainly didn't mean to suggest otherwise.