I have a web server which contains an API to upload files to Amazon's S3 storage. Since I do not want to waste resources on streaming the files through my server, when an upload request comes in, I generate a pre-signed URL for the client and then redirect that client to this URL using HTTP 307 - Temporary redirect.
In practice, the flow looks like this:
Client issues a PUT request to my server, requesting a file upload
My server inspects the request and generates a pre-signed URL for S3
My server responds to client with 307 redirection to the pre-signed URL
Client repeats the PUT request to the pre-signed URL
Upload commences
The challenge
My server uses the Authorization header for... well, authorisation. Incidentally, Amazon also accepts this header for authorisation, although the values expected by both parties are completely different.
The problem is, that since my upload API requires this header to be present during file upload request, when my server issues the 307 redirect back to the client, the client takes all the headers in the original request and sends them along to the pre-signed S3 URL, which causes the request to be rejected by Amazon due to authorisation error.
The question
Can I somehow instruct the client (via HTTP response header) to not include the Authorization header when following the redirection?
Current solution
Right now we "fixed" this by returning the pre-signed URL to the client in the response body. The client then manually issues a new PUT request to that URL without the Authorization header. This works fine. I would like to know if there is a way to achieve this behaviour without this extra manual work.
What is the client? In the above statements, when I mention the "client", right now it could be either a modern web browser or a native iOS or Android app. On iOS, we use Alamofire for HTTP communication. I am unsure of what library or components are used on Android.
Note: I have seen this question and its answers, but it does not contain the answers I seek.
Related
I'm trying to determine what a client should do with headers on receiving a 303 (See Other) from the server. Specifically, what should be done with the Authorization header that was sent on the initial request?
Here's the problem: the client makes a request to myserver.com (HTTP request method is not relevant here) and the server at myserver.com responds with a 303 and the Location header contains otherserver.com/some_resource/. Tools like Postman and curl will follow the redirect by passing all the same headers in the subsequent request to otherserver.com. I haven't found a way to make these tools drop the headers.
In the case I've described, sending the Authorization header to otherserver.com seems like a security risk: otherserver.com now knows my token and possibly what host it can be used on so now the token is compromised. This can also cause errors, depending on how the destination host is configured. In the case where the redirect is to another resources on the same host (ie, myserver.com) then the Authorization header will (probably) need to be sent, and because it's the same host nothing is compromised.
Effectively, in different situations it seems that the correct behaviour is different. The relevant section in the RFC does not address this issue. In developing my own API, I've written documentation telling API clients to drop the Authorization header on redirect to otherserver.com. However, based on mucking around with curl and Postman, it's not clear to me either (a) what the default behaviour is for a typical HTTP client library or (b) whether HTTP client libraries permit easy modification of the HTTP headers before following a 303 redirect. As a result, it's possible my suggestion isn't practical. I also know of no way for the server to instruct the client as to what it should do with headers on following the 303 redirect.
What should a HTTP client do with the headers when it follows a 303 redirect? Who is responsible for deciding whether to use the same headers on the redirect, the HTTP client or server?
You can argue that when sending the 303 with otherserver.com's Location, myserver.com trusted otherserver.com to handle your token. It could have sent the token in the background as well. From the client's perspective, the client trusts myserver.com to handle the token, store and verify it securely, etc. If myserver.com decides to send it on to otherserver.com, should the client override? In this case it can of course, but in general I don't think it should.
As an attacker does not control the response headers from myserver.com which is a legit resource, I think in general it is secure to send the token by default to the other server it specifies, maybe unless you have some good reason not to (say an explicit policy on the client).
I'm trying to retrieve 3 response headers (Rails Devise Auth Headers: uid, client, access-token) in every request to a Rails Server.
Using Postman (http client) it's working.
With OkHttp (java http client) the headers just don't show up in the client (i've checked using Wireshark).
When i'm in debug mode it just work...
The additional headers with postman are due to postman sending an Origin header and the server is replying with CORS headers, i.e. Access-Control-.... These headers are send within the normal HTTP header, i.e. not after the response.
But these access control headers are only relevant when the access is done from a browser because they control the cross origin behavior of XHR. Since you are not inside a browser they should be irrelevant for what you are doing. What is relevant are the body of the response and some of the other headers and here you'll find no differences. Also irrelevant should be if multiple requests are send within the same TCP connection (HTTP keep-alive done by postman) or with multiple connections (OkHttp) because each request is independent from the other and using the same TCP connection is only a performance optimization.
If you really want to get these special headers you should add an Origin header within you OkHttp request. See the OkHttp examples on how to add your own headers. But like I said: these access control headers should be irrelevant for the real task and there should be no need to get to these headers.
There is a property "config.batch_request_buffer_throttle" in the file "config/initializers/devise_token_auth.rb" of the Rails Project. We changed it from 5 seconds to 0 seconds.
It is a property to keep the current token available for that amount of time to the following requests.
As the original documentation: "Sometimes it's necessary to make several requests to the API at the same time. In this case, each request in the batch will need to share the same auth token. This setting determines how far apart the requests can be while still using the same auth token."
So when we did the request using Postman or in Java Debug the 5 seconds was running allowing Devise to generate new tokens then retrieve them to the client.
I want to redirect my users browser using HTTP code 303 to a GET URL that I secure using HMAC. Because the request will come from the users browser, I will not have fore-knowledge of the request headers. So I am generating the HMAC hash using the values of the HTTP method and URL only. For example, the URL I want the browser to do to might be:
GET /download
?name=report.pdf
&include=http://url1
&include=http://url2
This create report.pdf for me, containing the contents of all the urls specified using the include query param.
My HMAC code will change this URL to be
GET /download
?name=report.pdf
&include=http://url1
&include=http://url2
&hmac-algorithm=simple-hmac
&hmac-signature=idhihhoaiDOICNK
I can issue HTTP 303 to the user using this URL, and the user will get their report.pdf.
As I am not including the request headers in the signature, I am wondering two things:
1) Can a would-be attacker take advantage of the fact that I am not signing the request headers?
2) Is there a better way to achieve what I am trying to do?
When I realised that what I am talking about here is a signed URL, I checked the Amazon Docs and found "REST Authentication Example 3: Query String Authentication Example" in this document: http://s3.amazonaws.com/doc/s3-developer-guide/RESTAuthentication.html.
This example is about a signed URL for use through a browser. About signing the headers, the document says:
You know that when the browser makes the GET request, it won't provide a Content-Md5 or a Content-Type header, nor will it set any x-amz- headers, so those parts are all kept empty.
In other words, Amazon leave the headers out of the signature.
Amazon make no mention of potential security holes, so until I hear otherwise (or get hacked :) ), I will assume my approach above is fine.
It's my understanding that caching is one of the main utilities of a proxy server. I'm currently trying to develop a simple one and I would like to know exactly how caching works.
Intuitively I think that it's basically an association between a request and a response. For example: for the following request: "GET google.com" you have the following response: "HTTP/1.0 200 OK..."
That way, whenever the proxy server receives a request for that URL he can reply with the cached response (I'm not really worried right now about when to serve the cached response and when to actually send the request to the real destination).
What I don't understand is how to establish the association between a request and a response since the HTTP response doesn't have any field saying "hey this is the response you get when you request the X URL" (or does it?).
Should I get this information by analyzing the underlying protocols? If so, how?
Your cache proxy server is already putted into play when a request arrives. Therefore you have the requested resource URL. Then you look in your cache and try to find the cached resource for the requested resource URL, if you cannot find the resource in your cache (or the cache is outdated), you fetch the data from the source. Keep in Mind, that you have to invalidate the cached resource if you receive a PUT, POST or DELETE request.
In how many ways can an HTTP request be generated?
There are endless ways how you can create and from where you can send HTTP requests to a server. Actually your server has no idea, what the origin of such a request is (if it's AJAX or "regular" request, or sent from a console application or ...)
But there are HTTP methods (HTTP verbs) that (can) tell the server about the intent of the request: http://en.wikipedia.org/wiki/HTTP_Verbs#Request_methods
Also you can set headers in a request, for example the content-type or the accepted encoding: http://en.wikipedia.org/wiki/List_of_HTTP_header_fields
Most JavaScript libraries for example set the (non-standard) HTTP header X-Requested-With, so your application can differentiate between regular and ajax requests.
You see, it's even possible to set your own, non-standard headers. There are endless possible combinations...
HttpRequest is a C# class that wraps a petition sent by a client during a Web request.
There are many ways to generate it. The most usual one happens when your browser connects to an ASP.NET website.
You can, for example, create your own custom HttpRequest to petition a specific web page from a C# console application.
Are you trying to achieve something more specific?