What is the correct practice to handle a POST request on that requires authentication?
Consider a client that going to POST some data to a location and does not know if that location requires authentication. Assuming the POST request has endless data and does not set a Content-Length header the server should read everything and then give a response (since giving back a 404 in the middle of the content would not correspond to the way how HTTP works if I understand everything correctly). But it simply can't because the content is endless. On the other hand the client could try an initial POST with no content and specify Content-Length: 0 and then it would know if any authentication is needed. But if that is some kind of a single use location then with this fake request it is gone and needs to be regenerated.
Related
I'm currently trying to replicate a https request with the following headers.
The authentication is based on a Session Cookie.
However, there is also a header field called :authority,
which I'm not able to replicate in Postman.
Also, couldn't I find any solution for that problem
or what exactly the :authority header means on the Internet.
This header seems to be required, as I'm getting a 400 error code back
as soon as I send my replicated request.
I've proxied the request sent from Postman and can confirm that the only difference at the first glance is the missing :authority header.
Thanks ^^
Alright, taking a closer look at the Raw Request, I noticed that the actual header is called Host and is mapped in the UI to :autority. Now its working. cheers
I was thinking about asking on Software Recommendations, but then I've found out that it may be a too strange request and it needs some clarification first.
My points are:
Each response contains an etag
which is a hash of the content
and which is globally unique (with sufficient probability)
The content is (mostly) dynamic and may change anytime (expires and max-age headers are useless here).
The content is partly user-dependent, as given by the permissions (which itself change sometimes).
Basically, the proxy should contain a cache mapping the etag to the response content. The etag gets obtained from the server and in the most common case, the server does not deal with the response content at all.
It should go like follows: The proxy always sends a request to the server and then either
1 the server returns only the etag and the proxy makes a lookup based on it and
1.1 on cache hit,
it reads the response data from cache
and sends a response to the client
1.2 on cache miss,
it asks the server again and then
the server returns the response with content and etag,
the proxy stores it in its cache
and sends a response to the client
2 or the server returns the response with content and etag,
the proxy stores the data in its cache
and sends a response to the client
For simplicity, I left out the handling of the if-none-match header, which is rather obvious.
My reason for this is that the most common case 1.1 can be implemented very efficiently in the server (using its cache mapping requests to etags; the content isn't cached in the server), so that most requests can be handled without the server dealing with the response content. This should be better than first getting the content from a side cache and then serving it.
In case 1.2, there are two requests to the server, which sounds bad, but is no worse than the server asking a side cache and getting a miss.
Q1: I wonder, how to map the first request to HTTP. In case 1, it's like a HEAD request. In case 2, it's like GET. The decision between the two is up to the server: If it can serve the etag without computing the content, then it's case 1, otherwise, it's case 2.
Q2: Is there a reverse proxy doing something like this? I've read about nginx, HAProxy and Varnish and it doesn't seem to be the case. This leads me to Q3: Is this a bad idea? Why?
Q4: If not, then which existing proxy is easiest to adapt?
An Example
A GET request like /catalog/123/item/456 from user U1 was served with some content C1 and etag: 777777. The proxy stored C1 under the key 777777.
Now the same request comes from user U2. The proxy forwards it, the server returns just etag: 777777 and the proxy is lucky, finds C1 in its cache (case 1.1) and sends it to U2. In this example, neither the clients not the proxy knew the expected result.
The interesting part is how could the server know the etag without computing the answer. For example, it can have a rule stating that requests of this form return the same result for all users, assuming that the given user is allowed to see it. So when the request from U1 came, it computed C1 and stored the etag under the key /catalog/123/item/456. When the same request came from U2, it just verified that U2 is permitted to see the result.
Q1: It is a GET request. The server can answer with an "304 not modified" without body.
Q2: openresty (nginx with some additional modules) can do it, but you will need to implement some logic yourself (see more detailed description below).
Q3: This sounds like a reasonable idea given the information in your question. Just some food for thought:
You could also split the page in user-specific and generic parts which can be cached independently.
You shouldn't expect the cache to keep the calculated responses forever. So, if the server returns a 304 not modified with etag: 777777 (as per your example), but the cache doesn't know about it, you should have an option to force re-building the answer, e.g. with another request with a custom header X-Force-Recalculate: true.
Not exactly part of your question, but: Make sure to set a proper Vary header to prevent caching issues.
If this is only about permissions, you could maybe also work with permission infos in a signed cookie. The cache could derive the permission from the cookie without asking the server, and the cookie is tamper proof due to the signature.
Q4: I would use openresty for this, specifically the lua-resty-redis module. Put the cached content into a redis key-value-store with the etag as key. You'd need to code the lookup logic in Lua, but it shouldn't be more than a couple of lines.
Im wondering if I can detect whether some webpage was reached by a link (button, link, redirect, ...) or by typping in the URL bar.
Im using Clojure for web programming, and I wish I could block the access to some page when the request came from a "typed url".
Right now I am encoding the urls so the user can't have access to the "real" url.
Thanks in advance.
Short answer is no. Your aproach is correct.
And with a bit of skills one can also send arbitrary HTTP requests, GET or POST, modifying cookies, headers and the body of the request, so if this is about security, your approach to use encrypted data seems the only one possible to ensure that the URL is not tampered or manually typed.
The HTTP Referer header can be also faked, the same as a url parameter. There is no more security on using a http header or a url encoded parameter.
Courtesy of Wikipedia:
The HTTP referer (originally a misspelling of referrer[1]) is an HTTP header field that identifies the address of the webpage (i.e. the URI or IRI) that linked to the resource being requested. By checking the referrer, the new webpage can see where the request originated.
I assume you can access the HTTP header fields. If the user were to type in the URL into the address bar, there would be no HTTP referer.
Is this possible.. for example, imagine I respond to a request with a 302 (or 303), and I inform the browser to do a request to a given location.. is there a header that I can send with the HTTP 302, so that the subsequent request from the browser would include that header?
I know I could do this with the location header, as in redirect and specify the information in the url as a query string.. but I'm wondering if there is a better way.. it seems that it should be a legit scenario..
'Content has moved, go here .. oh and you'll want to take this with you to give to the redirect location'
I'm guessing a big fat no!
Thanks in advance.
Edit
The reason for this is in respect to PRG patterns, where you have a GET url and POST url, given that you post data and it isn't acceptable, the server redirects you to the GET, and does some 'magic' in order to 'send data' to that GET, using most often session state to store a variable.
However this can breakdown in scenarios where many of these PRG requests are happening, granted this isn't a common scenario and generally nobody need worry about this.. but if you do- you'll need a way to identify the requests, this can be done with query string parameters send in the 302.. so that a specific entry can be put in session state according to that request.
The question was regarding trying to remove the 'request key' from the url, and making it more implicit.. cookies 'appear' to work, but they only make the window for screw ups smaller.
It would be great to say when you go the 'location' i've specified, send these parameters.
Edit
Just to note, I'm not trying to get the browser to send arbitrary headers to the location, but if there is ANY headers designed to hint the context of the request (like the querystring parameters could).
A redirect response itself doesn't contain any data. You can redirect using a URL with query parameters, but the new "location" will need to know how to consume those parameters.
No, that’s not possible. You cannot force the client to something. You just can say “this is not the right location, but try that location instead”. But it’s not guaranteed that the client will send the same request or another request to that new location. And telling the client to add a specific header field in that subsequent request to the new location is also not possible.
I am setting up a back end API in a script of mine that contacts one of my sites by sending XML to my web server in the form of POST data. This script will be used by many and I want to limit the bandwidth waste for people that accidentally turn the feature on without a proper access key.
I will be denying requests that do not have the correct access key by maybe generating a 403 access code.
Lets say the POST data is ~500kb of data. Does the server receive all 500kb of data when this attempt is made regardless of the status code?
How about if I made the url contain the key mydomain/api/123456789 and generate 403 status on all bad access keys.
Does the POST data still get sent/received regardless or is it negotiated before the data is finally sent.
Thanks in advance!
Generally speaking, the entire request will be sent, including post data. There is often no way for the application layer to return a response like a 403 until it has received the entire request.
In reality, it will depend on the language/framework used and how closely it is linked to the HTTP server. Section 8.2.2 of RFC2616 HTTP/1.1 specification has this to say
An HTTP/1.1 (or later) client sending
a message-body SHOULD monitor the
network connection for an error status
while it is transmitting the request.
If the client sees an error status, it
SHOULD immediately cease transmitting
the body. If the body is being sent
using a "chunked" encoding (section
3.6), a zero length chunk and empty trailer MAY be used to prematurely
mark the end of the message. If the
body was preceded by a Content-Length
header, the client MUST close the
connection.
So, if you can find a language environemnt closely linked with the HTTP server (for example, mod_perl), you could do this in a way which does comply with standards.
An alternative approach you could take is to make an initial, smaller request to obtain a URL to use for the larger POST. The application can then deny providing the URL to clients without an appropriate key.
Here is great book about RESTful Web Services, where it's explained how HTTP works: http://oreilly.com/catalog/9780596529260
You can consider any request as envelope, where on top of it it's written address (URL), some properties (HTTP Headers) and inside it there's some data (if request is initiated by post method). So as you might guess you can't receive envelope partially.
Oh I forgot, it's when you are using HTTP Post with standard HTTP header "application/x-www-form-urlencoded" but if you are uploading files (correspondingly using ""multipart/form-data") Django gives you control over streamed chunks of files using Middleware classes: http://docs.djangoproject.com/en/dev/topics/http/middleware/