I am trying to create an http server which will start sending responses as early as I receive the first chunk.
I get a voice sample and then I return something back for each chunk of the voice sample. I was able to do this and test it out with apache http client and it worked. But I can't seems to get curl working with this. I start sending 200 O.K as as soon as I recieve the first chunk and the response starting from then on. And at the end of the request chunk I send an end of response chunk .Why does this not work with curl but with java http client. I saw that curl is sending variable size chunks but this may not be the actual reason.
Please help.
Related
I came across the term HTTP. I have done some research and wanted to ensure that I correctly understood the term.
So, is it true that HTTP, in simple words, a letter containing information in the language that both client and server can understand.
Then, that letter is sent to the server thanks to TCP/IP which serves as a car that takes that letter to the server.
Then, after the letter is delivered to the server, the server reads the content of the letter and if it is GET request, the server takes the necessary data and ATTACHES that data to the letter and sends back to the client via again TCP/IP. But if it was POST request then the client ATTACHES the DATA to the letter and sends it to the server so that it saves that data in the database.
Is that true?
Basically, it is true.
However, the server can decide what to do if it is a GET or POST or any other request(it doesn't need to e.g. append it to a file).
I will show you some additional information/try to explain it in my words:
TCP is another communication protocol protocol. It allows a client to open a connection to a server and they can communicate afterwards.
HTTP(hyper text transfer protocol) builds up on TCP.
At first, the client opens a connection to the server.
After that, the client sends the HTTP Request. The first line contains the type of the request, the path and the version. For example, it could be GET / HTTP/1.1.
The next part of the request contains the Request parameters. Every parameter is a line. The parameters are sent like the following: paramName: paramValue
This part of the request ends with an empty line.
If it is a POST Request, query parameters are added next. If it is a GET Request, these query parameters are added with the path(e.g. /index.html?paramName=paramValue)
After rescieving the Request, the server sends a HTTP Response back to the client.
The first line of the response contains the HTTP version, the status code and the status message. For example, it could be HTTP/1.1 200 OK.
Then, just like in the request, the response parameters are following. For example Content-Length: 1024.
The response parameters also end with an empty line.
The last part of the response is the body/content. For example, this could be the HTML code of the website you are visiting.
Obviously, the length of the content/body of the response has to match the Content-Length parameter(in bytes).
After that, the connection will be closed(normally). If the client to e.g. request resources, it will send another request. The server has NO POSSIBILITY to send data to the client after that unless the client sends another request(websockets can bypass this issue).
GET is meant to get the content of a site A web browser will send a GET request if you type in a URL. POST can be used to update a site but in fact, the server can decide that. POST can be also used if the server doesn't want query parameters to be shown in the address bar.
There are other methods like PATCH or DELETE that are used by some APIs.
Some important status codes (and status messages) are:
200 OK (everything went well)
204 No content (like ok but there is no body in the response)
400 Bad Request (something is wrong with the Request)
404 Not found (the requested file(the path) was not found on the server)
500 Internal server error (An error occured while processing the request)
Every status code beginning with 1 is related to inform the client of something.
If it is starting with 2, everything went right.
Status code beginning with 3 forward the client to another site.
If it starts with 4, there is a error on the client side.
Codes starting with 5 represent an error that occured on the server side.
TCP is a network protocol that establishes a connection with the server over a network (or the Internet) and allows two-way communication. The HTTP will traffic inside this TCP tunnel. TCP is a very useful protocol that helps keep things sane, it ensures data packets are read in the correct order and that packets that went missing during transmission are sent again.
Sometimes there will be another protocol layer between HTTP and TCP, called SSL. It is responsible for encrypting the data that traffics over TCP, so that it is transmitted safely over unsafe networks. This is know as HTTPS, and is just HTTP but using this additional layer.
Although almost always true, HTTP doesn't necessarily uses TCP. UPnP requests use HTTP over UDP, a network protocol that uses standalone packets instead of a connection.
HTTP is a plain text protocol, meaning it's designed in such a way that a human can understand it without using any tools. This is very convenient for learning.
If you're using Firefox or Chrome, you can press Ctrl-Shift-C to open the Developer Tools, and under the Network tab you will see every HTTP request your browser is making, see exactly what's the request, what the server answered etc, and get a better view of how this protocol works.
Explaining it in details is... too extensive for this answer. But as you will see it's not that complicated.
I am trying to upload data from an Arduino to data.sparkfun.com, but somehow it always fails. To make sure that the HTTP request I am sending is correct, I would like to send it from a computer to the server and see if it uploads the correct values.
According to some examples, the request should be formulated like this:
GET /input/publicKey?private_key=privateKey&dht1_t=24.23&dht1_h=42.4&dht2_t=24.48&dht2_h=41.5&bmp_t=23.3&bmp_p=984021 HTTP/1.1\n
Host: 54.86.132.254\n
Connection: close\n
\n
How do I send this request to the server from my computer? Do I just type in the terminal? Im not sure where to start.
Have a look at curl which should be able to handle your needs.
Even easier and more low level is netcat (here is an example on SO)
If browser sends big http request (file upload),
and server notices that the file is bigger than the server could handle.
The server sends some error message back without recieving the whole request.
Are all (or any) browsers able to read this server response, or they will wait with reading the response if they are not ready with sending the request?
http protocol is "request - response".
Does it mean the request must be completed before browser starts to wait for response?
Thanks
The first line in an HTTP server response indicates whether the client request was successful or not, and why. The status is given with a three-digit server response code (also known as a status code) and a descriptive message.
Status codes are usually generated by Web servers, but they might also be generated by CGI scripts that bypass the server's precooked headers and supply their own. Status codes are grouped as follows:
Code Range Response Meaning
100-199 Informational
200-299 Client request successful
300-399 Client request redirected, further action necessary
400-499 Client request incomplete
500-599 Server errors
HTTP defines only a few specific codes in each range, although servers may define their own as needed. If a client receives a code that it does not recognize, it should understand its basic meaning from its numerical range. While most Web browsers handle codes in the 100-, 200-, and 300- range silently, some error codes in the 400- and 500- range are commonly reported back to the user (e.g., "404 Not Found").
I'm pulling data from a server but need to know the type of data before I pull it. I know I can look at content-type in the response header, and I've looked into using
curl --head http://x.com/y/z
however some servers do not support the "HEAD" command (I get a 501 not implemented response).
Is it possible to somehow do a GET with curl, and immediately disconnect after all headers have been received?
Check out the following answer:
https://stackoverflow.com/a/5787827
Streaming. UNIX philosphy and pipes: they are data streams. Since curl and GET are unix filters, ending the receiving pipe (dd) will terminate curl or GET early (SIGPIPE). There is no telling whether the server will be smart enough to stop transmission. However on a TCP level I suppose it would stop retrying packets once there is no more response. #sehe
Using this method you should be able to download as many bytes as you want, and then cancel the request. You could also work some magic to terminate after receiving a blank line, which would mean the end of the header.
I have an HTTP server that returns large bodies in response to POST requests (it is a SOAP server). These bodies are "streamed" via chunking. If I encounter an error midway through streaming the response how can I report that error to the client and still keep the connection open? The implementation uses a proprietary HTTP/SOAP stack so I am interested in answers at the HTTP protocol level.
Once the server has sent the status line (the very first line of the response) to the client, you can't change the status code of the response anymore. Many servers delay sending the response by buffering it internally until the buffer is full. While the buffer is filling up, you can still change your mind about the response.
If your client has access to the response headers, you could use the fact that chunked encoding allows the server to add a trailer with headers after the chunked-encoded body. So, your server, having encountered the error, could gracefully stop sending the body, and then send a trailer that sets some header to some value. Your client would then interpret the presence of this header as a sign that an error happened.
Also keep in mind that chunked responses can contain "footers" which are just like HTTP headers. After failing, you can send a footer such as:
X-RealStatus: 500 Some bad stuff happened
Or if you succeed:
X-RealStatus: 200 OK
you can change the status code as long as response.iscommitted() returns false.
(fot HttpServletResponse in java, im sure there exists an equivalent in other languages)