I'm writing a NGINX module, and I would like to know if there is an option to send the client a response in parts? Meaning, in case of large files my server returns the response to the GET request in parts, I would like to send each part to the user without saving it to a temp file. I would like each part of the response to be sent and the client will get a few responses to the same request.
Related
I have read many discussions on this, such as the fact the PUT is idempotent and POST is not, etc. However, doesn't this ultimately depend on how the server is implemented? A developer can always build the backend server such that the PUT request is not idempotent and creates multiple records for multiple requests. A developer can also build an endpoint for a PUT request such that it acts like a DELETE request and deletes a record in the database.
So my question is, considering that we don't take into account any server side code, is there any real difference between the HTTP methods? For example, GET and POST have real differences in that you can't send a body using a GET request, but you can send a body using a POST request. Also, from my understanding, GET requests are usually cached by default in most browsers.
Are HTTP request methods anything more than just a logical structure (semantics) so that as developers we can "expect" a certain behavior based on the type of HTTP request we send?
You are right that most of the differences are on the semantic level, and if your components decide to assign other semantics, this will work as well. Unless there are components involved that you do not control (libraries, proxies, load balancers, etc).
For instance, some component might take advantage of the fact that PUT it idempotent and thus can re retried, while POST is not.
The Hypertext Transfer Protocol (HTTP) is designed to enable communications between clients and servers.
HTTP works as a request-response protocol between a client and server.
A web browser may be the client, and an application on a computer that hosts a web site may be the server.
Example: A client (browser) submits an HTTP request to the server; then the server returns a response to the client. The response contains status information about the request and may also contain the requested content.
HTTP Methods
GET
POST
PUT
HEAD
DELETE
PATCH
OPTIONS
The GET Method
GET is used to request data from a specified resource.
GET is one of the most common HTTP methods.
Note that the query string (name/value pairs) is sent in the URL of a GET request.
The POST Method
POST is used to send data to a server to create/update a resource.
The data sent to the server with POST is stored in the request body of the HTTP request.
POST is one of the most common HTTP methods.
The PUT Method
PUT is used to send data to a server to create/update a resource.
The difference between POST and PUT is that PUT requests are idempotent. That is, calling the same PUT request multiple times will always produce the same result. In contrast, calling a POST request repeatedly have side effects of creating the same resource multiple times.
The HEAD Method
HEAD is almost identical to GET, but without the response body.
In other words, if GET /users returns a list of users, then HEAD /users will make the same request but will not return the list of users.
HEAD requests are useful for checking what a GET request will return before actually making a GET request - like before downloading a large file or response body.
The DELETE Method
The DELETE method deletes the specified resource.
The OPTIONS Method
The OPTIONS method describes the communication options for the target resource.
src. w3schools
I'm trying to retrieve 3 response headers (Rails Devise Auth Headers: uid, client, access-token) in every request to a Rails Server.
Using Postman (http client) it's working.
With OkHttp (java http client) the headers just don't show up in the client (i've checked using Wireshark).
When i'm in debug mode it just work...
The additional headers with postman are due to postman sending an Origin header and the server is replying with CORS headers, i.e. Access-Control-.... These headers are send within the normal HTTP header, i.e. not after the response.
But these access control headers are only relevant when the access is done from a browser because they control the cross origin behavior of XHR. Since you are not inside a browser they should be irrelevant for what you are doing. What is relevant are the body of the response and some of the other headers and here you'll find no differences. Also irrelevant should be if multiple requests are send within the same TCP connection (HTTP keep-alive done by postman) or with multiple connections (OkHttp) because each request is independent from the other and using the same TCP connection is only a performance optimization.
If you really want to get these special headers you should add an Origin header within you OkHttp request. See the OkHttp examples on how to add your own headers. But like I said: these access control headers should be irrelevant for the real task and there should be no need to get to these headers.
There is a property "config.batch_request_buffer_throttle" in the file "config/initializers/devise_token_auth.rb" of the Rails Project. We changed it from 5 seconds to 0 seconds.
It is a property to keep the current token available for that amount of time to the following requests.
As the original documentation: "Sometimes it's necessary to make several requests to the API at the same time. In this case, each request in the batch will need to share the same auth token. This setting determines how far apart the requests can be while still using the same auth token."
So when we did the request using Postman or in Java Debug the 5 seconds was running allowing Devise to generate new tokens then retrieve them to the client.
I am having difficulties understanding the HTTP request and response flow. I am working with a system where I can "hijack" incoming HTTP request and give my own response. The problem I am having is that some type of GET request seem to assume that all data is sent back in first request.
For instance, JPEG image requests, no matter the size (my tests include 0-20 MB JPEG files) seems to assume that the entire data is sent in the first response. Even if I don't send any data and explicitly set range header to 0 I never get a response back from the client asking for the data.
Other data request types, such as mp4 video, the client seems perfectly fine with getting a response with only header information with no data and then sends a new request explicitly asking for byte range 0-.
Is there some kind of agreement between the the client and server that some types should be sent back in one request while others can be split up in a number of requests?
I am trying to create an HTTP parser, and I am interested to know how can I retrieve the URL from HTTP response message text. Does every response contain the URL of the page? Do I need to look at some fields in the header for that or just look at the packet's body? Do I need to save information from previous packets (such as location message)?
Requests have URLs, responses are data packets sent back to the client. But if you make an HTTP request, the immediate response will come from the same URL.
I am setting up a back end API in a script of mine that contacts one of my sites by sending XML to my web server in the form of POST data. This script will be used by many and I want to limit the bandwidth waste for people that accidentally turn the feature on without a proper access key.
I will be denying requests that do not have the correct access key by maybe generating a 403 access code.
Lets say the POST data is ~500kb of data. Does the server receive all 500kb of data when this attempt is made regardless of the status code?
How about if I made the url contain the key mydomain/api/123456789 and generate 403 status on all bad access keys.
Does the POST data still get sent/received regardless or is it negotiated before the data is finally sent.
Thanks in advance!
Generally speaking, the entire request will be sent, including post data. There is often no way for the application layer to return a response like a 403 until it has received the entire request.
In reality, it will depend on the language/framework used and how closely it is linked to the HTTP server. Section 8.2.2 of RFC2616 HTTP/1.1 specification has this to say
An HTTP/1.1 (or later) client sending
a message-body SHOULD monitor the
network connection for an error status
while it is transmitting the request.
If the client sees an error status, it
SHOULD immediately cease transmitting
the body. If the body is being sent
using a "chunked" encoding (section
3.6), a zero length chunk and empty trailer MAY be used to prematurely
mark the end of the message. If the
body was preceded by a Content-Length
header, the client MUST close the
connection.
So, if you can find a language environemnt closely linked with the HTTP server (for example, mod_perl), you could do this in a way which does comply with standards.
An alternative approach you could take is to make an initial, smaller request to obtain a URL to use for the larger POST. The application can then deny providing the URL to clients without an appropriate key.
Here is great book about RESTful Web Services, where it's explained how HTTP works: http://oreilly.com/catalog/9780596529260
You can consider any request as envelope, where on top of it it's written address (URL), some properties (HTTP Headers) and inside it there's some data (if request is initiated by post method). So as you might guess you can't receive envelope partially.
Oh I forgot, it's when you are using HTTP Post with standard HTTP header "application/x-www-form-urlencoded" but if you are uploading files (correspondingly using ""multipart/form-data") Django gives you control over streamed chunks of files using Middleware classes: http://docs.djangoproject.com/en/dev/topics/http/middleware/