What "Range: bytes=0-" header means ? Is the entire file ? I tried sending back 0 bytes and is not working, when I send the entire file it works, but I receive this request more than once in a streaming context, it doesn't look right.
Is the entire file ?
Yes, exactly that.
The spec has the grammar:
byte-range-set = 1#( byte-range-spec / suffix-byte-range-spec )
byte-range-spec = first-byte-pos "-" [ last-byte-pos ]
and also notes:
If the last-byte-pos value is
absent, or if the value is greater than or equal to the current
length of the representation data, the byte range is interpreted as
the remainder of the representation
Additionally:
A client can request the last N bytes of the selected representation
using a suffix-byte-range-spec.
suffix-byte-range-spec = "-" suffix-length
So, valid examples from the spec include:
bytes=-500
bytes=9500-
bytes=0-0,-1
I receive this request more than once in a streaming context
The header indicates that this client understands range requests, and would accept a 206 Partial Content response, rather than the entire file, for efficient streaming (What does the HTTP 206 Partial Content status message mean and how do I fully load resources?).
Related
I want to understand how the POST data is treated when in transit.
If I send a JSON object as POST data, the content-length of the request body is equal to the number of characters in the stringyfied version of the JSON object.
Question :
Does this imply that the POST data is always treated as a string when being sent ?
The content length of the post data is actually the count of the characters as each character is treated as to hold 1 byte and accordingly should respond to the POST data limit of the server i.e, if I have the limit odf 1MB of post data on server then the max content-length can be only 1024*1024?
Yes, an HTTP request takes the form of a sequence of bytes, i.e. a string. What else would it be? HTTP doesn't know about JSON; it doesn't know your bytes are to be later interpreted as JSON; even if it did, it would not care! It is a transport protocol.
And yes, if there is a limit of 1MB POST data, then you can only send 1MB worth of POST data.
The type of a value passed through a query string is always a string. When a HTTP client need to send a number, let's say 42, in is actually "42". Everything inside the query string is actually a string.
Is it the same for the type of values passed through a header ?
In other words, if we send an HTTP request with a "FooBar" header with the value of 42, and another request where the value of the header set to "42", will the server perceive the two received values as of the same type and value (i.e. "42")?
It's always a string, even in HTTP/2
tldr; Headers are text, sometimes ISO 8859, but usually just US-ASCII.
According to the 2014 RFC7230 (last paragraph), HTTP fields have used to be text and new headers should continue to do so, restricting the values to consist of US-ASCII octets.
The 1982 RFC822 specifies ASCII as the format of the header body.
References (found through List of HTTP Headers):
Section 3.2 of RFC822
RFC7230
I am learning HTTP. I enclose a request payload in XML or JSON format in my POST requests. What I wanted to know is whether a request payload and request body mean the same thing?
Definition of: payload : The "actual data" in a packet or file minus all headers attached for transport and minus all descriptive meta-data. In a network packet, headers are appended to the payload for transport and then discarded at their destination.
Edit: In Http protocol, an http packet has http headers and http payload.So payload section of http packet may or may not have a body depending upon the type of request (e.g. POST vs GET). So payload and body are not the same thing.
Payload is the "wrapper" to the body
Payload is something one carries. A paperboy's payload is a pile of newspapers and a HTTP POST request's payload is whatever comes in the "body".
What I wanted to know is whether a request payload and request body mean the same thing?
No, they have different meanings. A payload (a.k.a. content) is a part of representation data while a body is a part of a message, which are two different HTTP concepts. A representation (data and metadata) is transferred as a single or multiple messages, so a message encloses a complete or partial representation. The representation metadata are enclosed in the header fields of a message and the representation data, the payload, are enclosed in the body of a message, as is or transfer-encoded.
References
RFC 9110: HTTP Semantics defines the term representation:
3.2. Representations
A "representation" is information that is intended to reflect a past, current, or desired state of a given resource, in a format that can be readily communicated via the protocol. A representation consists of a set of representation metadata and a potentially unbounded stream of representation data (Section 8).
Notice that the definition is independent of the version of HTTP because it is about semantics.
RFC 9112: HTTP/1.1 defines the term message:
2.1. Message Format
An HTTP/1.1 message consists of a start-line followed by a CRLF and a sequence of octets in a format similar to the Internet Message Format [RFC5322]: zero or more header field lines (collectively referred to as the "headers" or the "header section"), an empty line indicating the end of the header section, and an optional message body.
HTTP-message = start-line CRLF
*( field-line CRLF )
CRLF
[ message-body ]
Notice that the definition depends on the version of HTTP because it is about syntax.
RFC 9110: HTTP Semantics defines the term content:
6.4. Content
HTTP messages often transfer a complete or partial representation as the message "content": a stream of octets sent after the header section, as delineated by the message framing.
This abstract definition of content reflects the data after it has been extracted from the message framing. For example, an HTTP/1.1 message body (Section 6 of [HTTP/1.1]) might consist of a stream of data encoded with the chunked transfer coding -- a sequence of data chunks, one zero-length chunk, and a trailer section -- whereas the content of that same message includes only the data stream after the transfer coding has been decoded; it does not include the chunk lengths, chunked framing syntax, nor the trailer fields (Section 6.5).
Note: Some field names have a "Content-" prefix. This is an informal convention; while some of these fields refer to the content of the message, as defined above, others are scoped to the selected representation (Section 3.2). See the individual field's definition to disambiguate.
RFC 9110: HTTP Semantics substitutes the term content for payload used in previous RFCs:
B.3. Changes from RFC 7231
[…]
The terms "payload" and "payload body" have been replaced with "content", to better align with its usage elsewhere (e.g., in field names) and to avoid confusion with frame payloads in HTTP/2 and HTTP/3. (Section 6.4)
Header identifies source & destination of the sent packet, whereas the actual data i.e Body is referred to as Payload
The start-line and HTTP headers of the HTTP message are collectively known as the head of the requests, whereas its payload is known as the body
So Yes, they are the same thing.
Got this from https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages
Payload of HTTP message is known as the body. link1
The HTTP message payload body is the information ("payload") part of the data that is sent in the HTTP Message Body (if any), prior to transfer encoding being applied. If transfer encoding is not used, the payload body and message body are the same thing! link2
So basically the only difference between HTTP message body and HTTP message payload body is encoding (but only if present). So generalizing the term request payload = request body.
I need to handle the Range header programatically in Java for supporting media files on iOS.
If my file is 23843 bytes, for example, I'm getting a request with a range header:
Range: bytes 23842-23842
What am I meant to return in this case? Is it just the last byte of the file?
You should send the file from offset 23842 to offset 23842, so yes, that comes out as one byte.
The spec actually gives a similar example:
The first and last bytes only (bytes 0 and 9999): bytes=0-0,-1
(The important bit here being that 0-0 = first byte)
I have been trying to stream audio from a particular point by using the Range header values but I always get the song right from the beginning. I am doing this through a program so am not sure whether the problem lies in my code or on the server.
How can I find out whether the server supports the Range header param?
Thanks.
The way the HTTP spec defines it, if the server knows how to support the Range header, it will. That in turn, requires it to return a 206 Partial Content response code with a Content-Range header, when it returns content to you. Otherwise, it will simply ignore the Range header in your request, and return a 200 response code.
This might seem silly, but are you sure you're crafting a valid HTTP request header? All too commonly, I forget to specify HTTP/1.1 in the request, or forget to specify the Range specifier, such as "bytes".
Oh, and if all you want to do is check, then just send a HEAD request instead of a GET request. Same headers, same everything, just "HEAD" instead of "GET". If you receive a 206 response, you'll know Range is supported, and otherwise you'll get a 200 response.
This is for others searching how to do this. You can use curl:
curl -I http://exampleserver.com/example_video.mp4
In the header you should see
Accept-Ranges: bytes
You can go further and test retrieving a range
curl --header "Range: bytes=100-107" -I http://exampleserver.com/example_vide0.mp4
and in the headers you should see
HTTP/1.1 206 Partial Content
and
Content-Range: bytes 100-107/10000000
Content-Length: 8
[instead of 10000000 you'll see the length of the file]
Although I am a bit late in answering this question, I think my answer will help future visitors. Here is a python method that detects whether a server supports range queries or not.
def accepts_byte_ranges(self, effective_url):
"""Test if the server supports multi-part file download. Method expects effective (absolute) url."""
import pycurl
import cStringIO
import re
c = pycurl.Curl()
header = cStringIO.StringIO()
# Get http header
c.setopt(c.URL, effective_url)
c.setopt(c.NOBODY, 1)
c.setopt(c.HEADERFUNCTION, header.write)
c.perform()
c.close()
header_text = header.getvalue()
header.close()
verbose_print(header_text)
# Check if server accepts byte-ranges
match = re.search('Accept-Ranges:\s+bytes', header_text)
if match:
return True
else:
# If server explicitly specifies "Accept-Ranges: none" in the header, we do not attempt partial download.
match = re.search('Accept-Ranges:\s+none', header_text)
if match:
return False
else:
c = pycurl.Curl()
# There is still hope, try a simple byte range query
c.setopt(c.RANGE, '0-0') # First byte
c.setopt(c.URL, effective_url)
c.setopt(c.NOBODY, 1)
c.perform()
http_code = c.getinfo(c.HTTP_CODE)
c.close()
if http_code == 206: # Http status code 206 means byte-ranges are accepted
return True
else:
return False
One way is just to try, and check the response. In your case, it appears the server doesn't support ranges.
Alternatively, do a GET or HEAD on the URI, and check for the Accept-Ranges response header.
You can use GET method with 0-0 Range request header, and check whether the response code is 206 or not, which will respond with
the first and last bytes of the response body
You also can use HEAD method do the same thing as the first session which will get the same response header and code without response body
Furthermore, you can check Accept-Ranges on the response header to judge whether it can support range, but please notice if the value is none on Accept-Ranges field, it means it can't support range, and if the response header doesn't have Accept-Ranges field you also can't finger out it can't support range from it.
There is another thing you have to know if you are using 0- Range on the request header with GET method to check the response code, the response body message will be cached automatically on the TCP receive window until the cache is full.