I'm successfully setting up an HTTP Digest Authorization between the web browser and the server. But some of the resources on the same page to the same host are failing because the browser isn't sending the authorization for them.
For example,
Page https://myhost/A/B/C/D/E/ is loaded, browser sends Authorization header.
Page contains IMG ref to https://myhost/A/B/C/D/E/F.JPG, browser sends Authorization header.
Page also contains IMG ref to https://myhost/A/B.JPG, but for some reason browser does not send Authorization header. Server returns 401 Unauthorized but browser does not retry with authorization or pop up a username/password field, it simply displays the "broken image" icon.
I have looked a bit at how HTTP Authorization and I don't see anything mentioned regarding the scope of a request. Nevertheless, because I am explicitly sending back a 401 if the browser doesn't send Authorization for a request, I would expect it should work.
How can I fix this problem?
HTML authorization is governed by RFC 2617, which in section 1.2 says:
The realm value (case-sensitive), in combination with the canonical
root URL (the absoluteURI for the server whose abs_path is empty; see
section 5.1.2 of [2]) of the server being accessed, defines the
protection space.”. Later in the same section it says: “The protection
space determines the domain over which credentials can be
automatically applied. If a prior request has been authorized, the
same credentials MAY be reused for all other requests within that
protection space for a period of time determined by the authentication
scheme, parameters, and/or user preference.
So as long as the two URLS are in the same "protection space" the browser is supposed to resend the same credentials. However in this case the problem is that they are not. If authorization occurs in the https://myhost/A/B/C/D/E/ space, then the browser may not see a need to send authorization for https://myhost/A/B.JPG.
Section 2 mentions:
A client SHOULD assume that all paths at or deeper than the depth of
the last symbolic element in the path field of the Request-URI also
are within the protection space specified by the Basic realm value of
the current challenge. A client MAY preemptively send the
corresponding Authorization header with requests for resources in that
space without receipt of another challenge from the server.
So the solution is to make sure that the WWW-Authenticate header sent by the server sends a domain=/ entry, so that everything under that will be considered in the same protection space.
Related
As a preface I'd like to claim I have some understanding of how the HTTP-supported authentication is supposed to work according to RFC 7235.
I am specifically interested to know how a client is supposed to know, after authenticating, which URIs on the server it is expected to provide same authorization (through the Authorization header) bearer for? Furthermore, is there any provision by HTTP to assist client in determining which Authorization headers it (the client) may have available (through whatever means it acquires them -- "login" form/dialog etc), would go with which realm(s)?
A realm doesn't seem to be specified in the form of an URI or even a regular expression of an URI, it's a value the interpretation of which appears to be left to the HTTP client application. Of note, a "Protection Space (Realm)" is defined, quoting:
A protection space is defined by the canonical root URI (the scheme and authority components of the effective request URI (see Section 5.5 of RFC7230) of the server being accessed, in combination with the realm value if present.
The above is all well and good, but it doesn't facilitate client mapping realms to URIs that may require authorization.
Say my Web server returns a response with status code 401 and the WWW-Authenticate: Bearer realm="The hangout" header line, for a request with a given URI, let's say /foobar. That makes it obvious that subsequent requests to the particular URI must include Authorization header line corresponding to solved challenge (as the client authenticates the user). No problem there. But what about e.g. requests with URI(s) that have the same pathname - those starting with /foobar/ -- is there an implication here that the same Authorization value is expected for these as well? What about entirely unrelated URI pathnames [on the same server]?
It would seem beneficial for the kind of authorization negotiation HTTP already does, to somehow relate or facilitate said relation of realms to URIs. But maybe I am missing something very obvious. Does HTTP do something along of what I am describing? Does it facilitate it in any way, at least, beyond leaving it entirely to the application? How would one realistically let the client determine which authorization bearer to send for which requests? Must it always get a 401 and a challenge response first, before knowing for sure requests to the particular URI and only said URI, must include related authorization bearer?
HTTP is a stateless protocol that deals with a request-response pair. The protocol does not deal with any information that would describe the concept of a "page", "site", "application", etc. Even though it deals with hypermedia, the protocol itself doesn't go beyond the concrete request. This means that you won't get any information from the protocol itself about any other paths under the same domain that are in the same authentication realm. This is left to the documentation of APIs or websites.
It seems to me that your research is centered on one type of authentication process that we call basic auth, know they are some other ways to authenticate a user and that they might suits your needs better as basic auth is kinda old as you can see on that RFC you linked.
To my understanding, the principle behind basic auth is to have a simple process based on challenges. When your client asks for a resource and that resource is protected by authentication, the server responds with a challenge : 401 Unauthorized with a header WWW-authenticate: Basic realm="some realm". The client then know the resource is restricted and depending on the realm, knows if it can have access (or asks the user for credentials for that realm), and try to access with a basic auth header : Authorization: Basic viFWGrwehryfviehtNRBRGWrwGERThRGE. You then repeat that process every time you need a resource.
HTTP and basic auth don't implement any sort of deeper and more complex system for authentication like you're searching for. It's one of the simplest system as its name implies and has not a lot more to offer. I'd even add that it's one of the riskier way to authenticate a system (even using SSL cert and cert pinning) as the client must send credentials for every single authenticated resource request.
In case you want to search other ways to authenticate requests, here are some :
OAuth (2.0) (most secured and complex)
Bearer (JWT or session tokens)
API keys
MDN says, when the credentials like cookies, authorisation header or TLS client certificates has to be exchanged between sites Access-Control-Allow-Crendentials has to be set to true.
Consider two sites A - https://example1.xyz.com and another one is B- https://example2.xyz.com. Now I have to make a http Get request from A to B. When I request B from A I am getting,
"No 'Access-Control-Allow-Origin' header is present on the requested
resource. Origin 'http://example1.xyz.com' is therefore not allowed
access."
So, I'm adding the following response headers in B
response.setHeader("Access-Control-Allow-Origin", request.getHeader("origin"));
This resolves the same origin error and I'm able to request to B. When and why should I set
response.setHeader("Access-Control-Allow-Credentials", "true");
When I googled to resolve this same-origin error, most of them recommended using both headers. I'm not clear about using the second one Access-Control-Allow-Credentials.
When should I use both?
Why should I set Access-Control-Allow-Origin to origin obtained from request header rather than wildcard *?
Please quote me an example to understand it better.
Allow-Credentials would be needed if you want the request to also be able to send cookies. If you needed to authorize the incoming request, based off a session ID cookie would be a common reason.
Setting a wildcard allows any site to make requests to your endpoint. Setting allow to origin is common if the request matches a whitelist you've defined. Some browsers will cache the allow response, and if you requested the same content from another domain as well, this could cause the request to be denied.
Setting Access-Control-Allow-Credentials: true actually has two effects:
Causes the browser to actually allow your frontend JavaScript code to access the response if credentials are included
Causes any Set-Cookie response header to actually have the effect of setting a cookie (the Set-Cookie response header is otherwise ignored)
Those effects combine with the effect that setting XMLHttpRequest.withCredentials or credentials: 'include' (Fetch API) have of causing credentials (HTTP cookies, TLS client certificates, and authentication entries) to actually be included as part of the request.
https://fetch.spec.whatwg.org/#example-cors-with-credentials has a good example.
Why should I set Access-Control-Allow-Origin to origin obtained from request header rather than wildcard *?
You shouldn’t unless you’re very certain what you’re doing.
It’s actually safe to do if:
The resource for which you’re setting the response headers that way is a public site or API endpoint intended to be accessible by everyone, and
You’re just not setting cookies that could enable an attacker to get access to sensitive information or confidential data.
For example, if your server code is just setting cookies just for the purpose of saving application state or session state as a convenience to your users, then there’s no risk in taking the value of the Origin request header and reflecting/echoing it back in the Access-Control-Allow-Origin value while also sending the Access-Control-Allow-Credentials: true response header.
On the other hand, if the cookies you’re setting expose sensitive information or confidential data, then unless you’re really certain you have things otherwise locked down (somehow…) you really want to avoid reflecting the Origin back in the Access-Control-Allow-Origin value (without checking it on the server side) while also sending Access-Control-Allow-Credentials: true.
If you do that, you’re potentially exposing sensitive information or confidential data in way that could allow malicious attackers to get to it. For an explanation of the risks, read the following:
https://web-in-security.blogspot.jp/2017/07/cors-misconfigurations-on-large-scale.html
http://blog.portswigger.net/2016/10/exploiting-cors-misconfigurations-for.html
And if the resource you’re sending the CORS headers for is not a public site or API endpoint intended to be accessible by everyone but is instead inside an intranet or otherwise behind some IP-address-restricted firewall, then you definitely really want to avoid combining Access-Control-Allow-Origin-reflects-Origin and Access-Control-Allow-Credentials: true. (In the intranet case you almost always want to only be allowing specific hardcoded/whitelisted origins.)
After the user requests a protected resource X the server responds
with code 401.
The browser prompts the user to inser user-name and
password and automatically re-send the request to the server with
those authentication information
My question is : is this process repeated over and over for each protected resource ?
Look at RFC 2617. There is stated for basic-athentication :
Upon receipt of an unauthorized request for a URI within the
protection space, the origin server MAY respond with a challenge ...
and also
A client SHOULD assume that all paths at or deeper than the depth of
the last symbolic element in the path field of the Request-URI also
are within the protection space specified by the Basic realm value of
the current challenge. A client MAY preemptively send the
corresponding Authorization header with requests for resources in
that space without receipt of another challenge from the server.
Similarly, when a client sends a request to a proxy, it may reuse a
userid and password in the Proxy-Authorization header field without
receiving another challenge from the proxy server.
So, from the server side this may occur at any request the the server deems unauthenticated. If resource Y does not share the prefix that had been yuthenticated with resource X then the server will re-request authentication.
For avoiding this the authentication scheme e.g. could request authentication for a common prefix of the related resources , such that authentication for prefix of resource X also covers resource Y as a prefix. This will allow the client to send the authentication header and cause the server to detect the call as already being authenticated.
Once the user input the password, the browser will remember it.
each time the client request the resource at the same website, the browser will send the authentication header automatically.
When I ask user for HTTP Basic Auth at some URL, browser sends Authorization header only for this and some other URLs.
Testcase script written in PHP:
http://testauth.veadev.tk/
There are three URLs to ask for credentials (you can use any random).
Logout link (drops current credential after pressing "Cancel" button in browser auth form, not working in IE).
Links to root URL and some test deeper URLs.
Questions:
Why browser not sending Authorization header at / URL if HTTP/1.0 401 Unauthorized was sent at /system/dev?
To repeat: open clean http://testauth.veadev.tk/, click Auth2, enter any credentials, you'll be forwarded to / after that. You'll see Auth: null which means no credentials header was sent by browser.
Why does browser send Authorization header at / if HTTP/1.0 401 Unauthorized was sent at /dev?
To repeat: open clean http://testauth.veadev.tk/, click Auth1, enter any credentials, you'll be forwarded to / after that. You'll see something like Auth: string 'Basic dHQ6dHQ=' (length=14) which means credentials header was sent by browser.
If you repeat first case and then click Auth1 you'll have credentials at Root and all other pages. Why?
If you click Auth3 (/some/deep/and/long/url) and you'll have credentials at Page3 (/some/deep/and/long/3) and nowhere else. Why?
To clear credential state between tests either restart your browser or click Logout, Cancel in Auth form and Root to return back (Firefox, Google Chrome).
What are the rules of sending Authorization header?
RFC 2617, section 2 states:
A client SHOULD assume that all paths at or deeper than the depth of
the last symbolic element in the path field of the Request-URI also
are within the protection space specified by the Basic realm value of
the current challenge. A client MAY preemptively send the
corresponding Authorization header with requests for resources in that
space without receipt of another challenge from the server.
If you are using Digest Challenge, section 3.2 states that you may specify a domain in the WWW-Authenticate header to indicate what the protection space will be. I would try setting that to something like domain=/. I am not sure if this will work with Basic authorization, but it wouldn't hurt to try it; if not, Digest authorization is not much more difficult to work with and is a bit more secure.
I cannot find any RFC or Standard of HTTP client behavior in case it gets HTTP response with an error 4xx. I know the 401, 407 are the examples when the HTTP headers are parsed, but...
I have the concrete problem for OPTIONS method (HTTP1.1). The server responses 401 Unauthorized, so client tries to authenticate and re-sends the request with an authentication. After that the response has the error 404 Not Found and HTTP header is filled with Set-Cookie HTTP Header. The client use Apache Java HTTPClient/HTTPComponents, which ignores HTTP headers in case of an error in the response.
Should this HTTP Header be accepted by the client? I believe it should not be, but I cannot find the supportive quotation in the RFC.
RFC 2616 does not specify that any headers should be ignored, not for 404 responses and not for 4xx responses in general either.
RFC 6265 allows clients to ignore Set-Cookie headers, but does not specify situations where that might happen; a single example is given, that does not cover your case:
the user agent might wish to block responses to "third-party" requests
from setting cookies
In your case, since your server seems to use HTTP basic access authentication, it does not seem to concern the Set-Cookie header. In HTTP basic authentication, the Authorization header is sent by the client with every request, so there should be no need to keep state in a cookie.
It is not clear from your question if you have a very specific HTTP server that you're talking to, or if you are implementing a general HTTP client that is supposed to work with whatever server you throw it at. If you have such a specific case that the HTTP server you work with sends state with 404 responses, and you're required to honor that state in order to communicate with the server, and you have no control over the server, then it does not matter what the standard says; you will honor the state sent, or you will not be able to talk to the server.
If, on the other hand, you're implementing a general client and need it to work regardless of the remote server, then your best bet is to stick to RFC 1958:
Be strict when sending and tolerant when receiving.
Implementations must follow specifications precisely when sending to
the network, and tolerate faulty input from the network. When in
doubt, discard faulty input silently, without returning an error
message unless this is required by the specification.
Which, to me, would mean that you should honor the full response received, regardless of the status code, unless you have an objective reason making it impossible for you to do so. I don't see a reason to ignore the state, even if it violates the standard (or in this case, your personal perception of the standard, since it does not say anything about accepting or ignoring the state).
Update: RFC 2617 (HTTP Authentication) states:
A client SHOULD assume that all paths at or deeper than the depth of
the last symbolic element in the path field of the Request-URI also
are within the protection space specified by the Basic realm value of
the current challenge. A client MAY preemptively send the
corresponding Authorization header with requests for resources in
that space without receipt of another challenge from the server.
It is highly inconsistent if the server expects HTTP authentication for one URL, but does not honor it for URLs beneath it, requiring a separate cookie-based authentication for them. If anything should be changed in your server implementation, it should be to harmonize the authentication scheme for all resources.