What is the correct format for sending an HTTP 403 forbidden message?
I'm writing a proxy in c for a homework project that has a content filtering system built in. When my proxy detects that a server's response has certain keywords that are contained in the content blacklist, I would like to send a HTTP 403 Forbidden message.
Currently, I am sending the message as: "HTTP/1.1 403 Forbidden\r\n\r\n" (without the quotes) as per this standard: https://www.rfc-editor.org/rfc/rfc7231#section-6.5.3
When I send this message, the browser doesn't display an error and looks like it's still trying to load the page.
Are there any required header fields for this http message that I missed? Also, is this the correct usage for the 403 error? I couldn't find anything else that would be more fitting, so I chose 403 because the client won't automatically re-request the data.
Thanks in advance for any help!
For those struggling with this issue as I did, you need to make sure to close the socket or set Connection: Close as Sami noted in the comments. I assumed that you could keep it open so they could send another request with http persistent connections, but they will need to open a new connection.
As for the html displayed, you can send a body with the response (make sure you set Content-Length) that contains the html you want displayed.
Finally, here are two references, one to the HTTP response spec, and the other to the Amazon Restful response spec:
https://www.rfc-editor.org/rfc/rfc7231#section-6.5.3
https://developer.amazon.com/docs/amazon-drive/ad-restful-api-response-codes.html
It's my understanding that caching is one of the main utilities of a proxy server. I'm currently trying to develop a simple one and I would like to know exactly how caching works.
Intuitively I think that it's basically an association between a request and a response. For example: for the following request: "GET google.com" you have the following response: "HTTP/1.0 200 OK..."
That way, whenever the proxy server receives a request for that URL he can reply with the cached response (I'm not really worried right now about when to serve the cached response and when to actually send the request to the real destination).
What I don't understand is how to establish the association between a request and a response since the HTTP response doesn't have any field saying "hey this is the response you get when you request the X URL" (or does it?).
Should I get this information by analyzing the underlying protocols? If so, how?
Your cache proxy server is already putted into play when a request arrives. Therefore you have the requested resource URL. Then you look in your cache and try to find the cached resource for the requested resource URL, if you cannot find the resource in your cache (or the cache is outdated), you fetch the data from the source. Keep in Mind, that you have to invalidate the cached resource if you receive a PUT, POST or DELETE request.
I have a TXT file, stored in a subfolder on my web site, and our Windows application reads this file through HTTP.
Our application just sends a request with minimum headers: HTTP verb, filename, protocol on first line and Host key to point the server. Web server (which is a shared hosting) returns 500 to this request, where it returns 200 to request populated by browser.
Using Fiddler, I added User-Agent header info to request and voila, it returned 200. Regardless of what the value is.
Now, we'll add this user-agent info into our application, but it takes time to update all clients. Hosting staff say they did not make any changes on server settings.
Is there any solution that I can apply using web.config?
Regards,
I have a client that issues an HTTP PUT to upload a file to a SharePoint server. This server requires authentication (NTLM or Negotiate). The client includes an "Expect: 100-Continue" header in the PUT request.
When I PUT the file to SharePoint2010 (IIS 7.5), the server immediately responds with a 401 error as soon as it receives the header. This allow the client to begin the authentication process without unnecessarily uploading the entire file.
When I PUT the file to SharePoint2013 (IIS 8.0), the server immediately responds with a 100-Continue. However, once the client has uploaded the file in the body of the request, it replies with a 401. This means that the client has uploaded the entire file just to receive an authentication error. The server should have been able to respond with the 401 prior to the client uploading all of the data, as IIS 7.5 does.
Is this change in behavior inherent in IIS8? Is there a configuration option that controls this? Is there something I can tweak in the header to change the server's behavior?
You must remove the expect: 100-Continue from your HTTP Header
This link will be usefull if you use .NET :
Removing the expect http header
When a user is not logged in and tries to access a page that requires login, what is the correct HTTP status code for a redirect to the login page?
I am asking because none of the 3xx response codes set out by the W3C seem to fit the requirements:
10.3.1 300 Multiple Choices
The requested resource corresponds to
any one of a set of representations,
each with its own specific location,
and agent- driven negotiation
information (section 12) is being
provided so that the user (or user
agent) can select a preferred
representation and redirect its
request to that location.
Unless it was a HEAD request, the
response SHOULD include an entity
containing a list of resource
characteristics and location(s) from
which the user or user agent can
choose the one most appropriate. The
entity format is specified by the
media type given in the Content- Type
header field. Depending upon the
format and the capabilities of
the user agent, selection of the most
appropriate choice MAY be performed
automatically. However, this
specification does not define any
standard for such automatic selection.
If the server has a preferred choice
of representation, it SHOULD include
the specific URI for that
representation in the Location field;
user agents MAY use the Location field
value for automatic redirection. This
response is cacheable unless indicated
otherwise.
10.3.2 301 Moved Permanently
The requested resource has been
assigned a new permanent URI and any
future references to this resource
SHOULD use one of the returned URIs.
Clients with link editing capabilities
ought to automatically re-link
references to the Request-URI to one
or more of the new references returned
by the server, where possible. This
response is cacheable unless indicated
otherwise.
The new permanent URI SHOULD be given
by the Location field in the response.
Unless the request method was HEAD,
the entity of the response SHOULD
contain a short hypertext note with a
hyperlink to the new URI(s).
If the 301 status code is received in
response to a request other than GET
or HEAD, the user agent MUST NOT
automatically redirect the request
unless it can be confirmed by the
user, since this might change the
conditions under which the request was
issued.
Note: When automatically redirecting a POST request after
receiving a 301 status code, some existing HTTP/1.0 user agents
will erroneously change it into a GET request.
10.3.3 302 Found
The requested resource resides
temporarily under a different URI.
Since the redirection might be altered
on occasion, the client SHOULD
continue to use the Request-URI for
future requests. This response is only
cacheable if indicated by a
Cache-Control or Expires header field.
The temporary URI SHOULD be given by
the Location field in the response.
Unless the request method was HEAD,
the entity of the response SHOULD
contain a short hypertext note with a
hyperlink to the new URI(s).
If the 302 status code is received in
response to a request other than GET
or HEAD, the user agent MUST NOT
automatically redirect the request
unless it can be confirmed by the
user, since this might change the
conditions under which the request was
issued.
Note: RFC 1945 and RFC 2068 specify that the client is not allowed
to change the method on the redirected request. However, most
existing user agent implementations treat 302 as if it
were a 303
response, performing a GET on the Location field-value regardless
of the original request method. The status codes 303 and 307 have
been added for servers that wish to make unambiguously clear which
kind of reaction is expected of the client.
10.3.4 303 See Other
The response to the request can be
found under a different URI and SHOULD
be retrieved using a GET method on
that resource. This method exists
primarily to allow the output of a
POST-activated script to redirect the
user agent to a selected resource. The
new URI is not a substitute reference
for the originally requested resource.
The 303 response MUST NOT be cached,
but the response to the second
(redirected) request might be
cacheable.
The different URI SHOULD be given by
the Location field in the response.
Unless the request method was HEAD,
the entity of the response SHOULD
contain a short hypertext note with a
hyperlink to the new URI(s).
Note: Many pre-HTTP/1.1 user agents do not understand the 303
status. When interoperability with such clients is a concern, the
302 status code may be used instead, since most user agents react
to a 302 response as described here for 303.
10.3.5 304 Not Modified
If the client has performed a
conditional GET request and access is
allowed, but the document has not been
modified, the server SHOULD respond
with this status code. The 304
response MUST NOT contain a
message-body, and thus is always
terminated by the first empty line
after the header fields.
The response MUST include the
following header fields:
- Date, unless its omission is required by section 14.18.1 If a
clockless origin server obeys these
rules, and proxies and clients add
their own Date to any response
received without one (as already
specified by [RFC 2068], section
14.19), caches will operate correctly.
- ETag and/or Content-Location, if the header would have been sent
in a 200 response to the same request
- Expires, Cache-Control, and/or Vary, if the field-value might
differ from that sent in any previous response for the same
variant If the conditional GET used a strong cache validator (see
section 13.3.3), the response SHOULD
NOT include other entity-headers.
Otherwise (i.e., the conditional GET
used a weak validator), the response
MUST NOT include other entity-headers;
this prevents inconsistencies between
cached entity-bodies and updated
headers.
If a 304 response indicates an entity
not currently cached, then the cache
MUST disregard the response and repeat
the request without the conditional.
If a cache uses a received 304
response to update a cache entry, the
cache MUST update the entry to reflect
any new field values given in the
response.
10.3.6 305 Use Proxy
The requested resource MUST be
accessed through the proxy given by
the Location field. The Location field
gives the URI of the proxy. The
recipient is expected to repeat this
single request via the proxy. 305
responses MUST only be generated by
origin servers.
Note: RFC 2068 was not clear that 305 was intended to redirect a
single request, and to be generated by origin servers only. Not
observing these limitations has significant security consequences.
10.3.7 306 (Unused)
The 306 status code was used in a
previous version of the specification,
is no longer used, and the code is
reserved.
10.3.8 307 Temporary Redirect
The requested resource resides
temporarily under a different URI.
Since the redirection MAY be altered
on occasion, the client SHOULD
continue to use the Request-URI for
future requests. This response is only
cacheable if indicated by a
Cache-Control or Expires header field.
The temporary URI SHOULD be given by
the Location field in the response.
Unless the request method was HEAD,
the entity of the response SHOULD
contain a short hypertext note with a
hyperlink to the new URI(s) , since
many pre-HTTP/1.1 user agents do not
understand the 307 status. Therefore,
the note SHOULD contain the
information necessary for a user to
repeat the original request on the new
URI.
If the 307 status code is received in
response to a request other than GET
or HEAD, the user agent MUST NOT
automatically redirect the request
unless it can be confirmed by the
user, since this might change the
conditions under which the request was
issued.
I'm using 302 for now, until I find the correct answer.
Update & conclusion:
HTTP 302 is better since its known to have best compatibility with clients/browsers.
I'd say 303 see other 302 Found:
The requested resource resides temporarily under a different URI. Since the redirection might be altered on occasion, the client SHOULD continue to use the Request-URI for future requests. This response is only cacheable if indicated by a Cache-Control or Expires header field.
fits a login page most closely in my opinion. I initially considered 303 see other which would work just as well. After some thought, I'd say 302 Found is more fitting because the requested resource was found, there just is another page to go through before it can be accessed. The response doesn't get cached by default which is fine as well.
This is a misuse of HTTP redirection mechanism. If user is not authorized then your app must return 401 Unauthorized. In case that the user is authorized but does not have an access to the requested resource then 403 Forbidden must be returned.
You should do the redirect on client side, e.g. by javascript. status code for redirection because required authorization does not exist. Using 30x for this does not conform to HTTP.
How to Think About HTTP Status Codes by Mark Nottingham
401 Unauthorized triggers HTTP’s request authentication mechanism.
401 Unauthorized status code requires presence of WWW-Authenticate header that supports various authentication types:
WWW-Authenticate: <type> realm=<realm>
Bearer, OAuth, Basic, Digest, Cookie, etc
Hypertext Transfer Protocol (HTTP) Authentication Scheme
Registry
Cookie-based HTTP Authentication - DRAFT
I think the appropriate solution is the HTTP 401 (Not Authorized) header.
http://en.wikipedia.org/wiki/HTTP_codes#4xx_Client_Error
The purpose of this header is exactly this. But, instead of redirecting to a login page, the correct process would be something like:
User not logged try to access a login-restricted page.
system identifies user is not logged
system returns HTTP 401 header, AND display the login form in the same response (not a redirect).
This is a good practice, like providing a useful 404 page, with sitemap links, and a search form for example.
See you.
Lots of compelling and contradictory responses here! How to choose?
Finalists:
302 Found
Indicates that the resource requested has been
temporarily moved to the URL given by the Location header. A browser
redirects to this page but search engines don't update their links to
the resource.
The specification requires the method (and the body) not be
altered when the redirection is performed, but not all user-agents
conform. It
is therefore recommended to set the 302 code only as a response for
GET or HEAD methods and to use 307 Temporary Redirect instead, as the
method change is explicitly prohibited in that case.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/302
Vague, possibly buggy, and maybe outdated, but could work, especially for GET requests.
307 Temporary Redirect
The resource requested has been temporarily moved to the URL given by
the Location header.
The only difference between 307 and 302 is that 307 guarantees that
the method and the body will not be changed when the redirected
request is made. With 302, some old clients were incorrectly changing
the method to GET: the behavior with non-GET methods and 302 is then
unpredictable on the Web, whereas the behavior with 307 is
predictable. For GET requests, their behavior is identical.
https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/307
My original take was that if your application supports HTTP 1.1+, this would be the newer, clearer way to make a temporary redirect.
401 Unauthorized
indicates that the client request has not been completed because it
lacks valid authentication credentials for the requested resource.
This status code is sent with an HTTP WWW-Authenticate response header
that contains information on how the client can request for the
resource again after prompting the user for authentication
credentials.
This status code is similar to the 403 Forbidden status code, except
that in situations resulting in this status code, user authentication
can allow access to the resource.
I agree this was probably intended to be the correct response code to say, "Not until you sign in." But almost every webapp uses HTML forms with Cookies to sign-in and this HTTP code doesn't allow "Cookie" as an authentication scheme:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/WWW-Authenticate
and
https://www.iana.org/assignments/http-authschemes/http-authschemes.xhtml
A few people have pointed out that someone drafted a "Cookie" scheme in 2009: WWW-Authenticate Cookie
https://datatracker.ietf.org/doc/html/draft-broyer-http-cookie-auth-00
Not sure why that didn't fly. Maybe because that's outside the realm of HTTP?
Without "Cookie" as an option, I think this is not suitable for HTML-and-Cookie based authentication.
Survey
In the absence of an undeniable single "correct" answer, I looked at what a few industry leaders were doing.
Google: 302 Redirect to Sign In Page
Request URL: https://mail.google.com/mail/u/0/?tab=cm Request Method:
GET Status Code: 302
Response:
content-type: text/html; charset=UTF-8 location:
https://accounts.google.com/ServiceLogin?service=mail&passive=1209600&osid=1&continue=https://mail.google.com/mail/u/0/?tab%3Dcm&followup=https://mail.google.com/mail/u/0/?tab%3Dcm&emr=1
Microsoft: 302 Redirect to Sign In page
Request URL:
https://outlook.live.com/mail/0/?authRedirect=true&state=0 Request
Method: GET Status Code: 302
Response: Location:
https://outlook.live.com/owa/0/?state=1&redirectTo=aHR0cHM6Ly9vdXRsb29rLmxpdmUuY29tL21haWwvMC8
Facebook: 302 Redirect to Sign In Page
Request URL: https://www.facebook.com/friends Request Method: GET
Status Code: 302
Response: location:
https://www.facebook.com/login.php?next=https%3A%2F%2Fwww.facebook.com%2Ffriends
Twitter: 200 with interstitial sign-in page
Request URL: https://twitter.com/messages/ Request Method: GET Status
Code: 200 (from service worker)
Me: 307: Temporary Redirect
I've used this for over a decade and never had a problem. Not suggesting that others adopt it, just saying it works as advertised as a 302 alternative in every major browser. I could be convinced to switch to using 302 as a result of researching this answer.
Conclusion
302 is the de-facto standard for redirects, unless you require a POST to be redirected to another POST, which I didn't try.
Really, I think a POST should either succeed or fail outright. POST causes a state change on the server (such as adding a product to inventory) and you really want to know if the change was processed and how many times. Redirecting would raise the question of whether your change was accepted or not before the redirect. Once you decide to NEVER redirect a POST, then 302 and 307 are equivalent. The "bugs" in 302 were in unrealistic use cases. Therefore, 307 should never have been created. Also 302 is more popular, so likely to be best supported.
The nice thing about using a single code for all temporary redirects is that you don't have to worry about the reason for the redirect.
200 with JavaScript throwing up an interstitial sign-in page is another option, probably more suited to single-page webapps. You asked for the page, you got the page, but you're going to be forced to sign in before you can see the contents.