What does ajax GET data look like at the lowest level? - http

GET requests can be used to retrieve images and text and other things I'm guessing. There is no need to specify Content-type.
What does this data look like at the bit level? If you are looking at the 1s and 0s in the HTTP packet, what specification governs what can be put here.
Using the client, when I send / receive data via ajax GET, is the data directly transferred into 1s and 0s in the packet or is there some sort of transformation?
For example:
xhr = new win.XMLHttpRequest();
xhr.open('GET', config_ajax.url, true);
xhr.onload = function () {
if (this.status === 200) {
config_ajax.callback(xhr.responseText);
}
};
xhr.send(send);
// example data
send = "0xFF";
xhr.responseText = "0x0A"
Would one see 11111111 being sent and 00001010 being received if they were analyzing the bit stream?

I guess there is no need to specify Content-type. What does this data look like at the bit level? If you are looking at the 1s and 0s in the HTTP packet, what specification governs what can be put here.
You're looking for the HTTP specification itself, section 7. Indeed it works quite like you assumed:
7 Entity
Request and Response messages MAY transfer an entity if not otherwise
restricted by the request method or response status code. An
entity consists of entity-header fields and an entity-body, although
some responses will only include the entity-headers.
In this section, both sender and recipient refer to either the client
or the server, depending on who sends and who receives the entity.
In this section, both sender and recipient refer to either the
client or the server, depending on who sends and who receives the
entity.
7.1 Entity Header Fields
Entity-header fields define metainformation about the entity-body
or, if no body is present, about the resource identified by the
request. Some of this metainformation is OPTIONAL; some might be
REQUIRED by portions of this specification.
entity-header = Allow ; Section 14.7
| Content-Encoding ; Section 14.11
| Content-Language ; Section 14.12
| Content-Length ; Section 14.13
| Content-Location ; Section 14.14
| Content-MD5 ; Section 14.15
| Content-Range ; Section 14.16
| Content-Type ; Section 14.17
| Expires ; Section 14.21
| Last-Modified ; Section 14.29
| extension-header
extension-header = message-header
The extension-header mechanism allows additional entity-header
fields to be defined without changing the protocol, but these
fields cannot be assumed to be recognizable by the recipient.
Unrecognized header fields SHOULD be ignored by the recipient and
MUST be forwarded by transparent proxies.
7.2 Entity Body
The entity-body (if any) sent with an HTTP request or response is
in a format and encoding defined by the entity-header fields.
entity-body = *OCTET
An entity-body is only present in a message when a message-body is
present, as described in section 4.3. The entity-body is obtained
from the message-body by decoding any Transfer-Encoding that might
have been applied to ensure safe and proper transfer of the message.
7.2.1 Type
When an entity-body is included with a message, the data type of
that body is determined via the header fields Content-Type and
Content- Encoding. These define a two-layer, ordered encoding
model:
entity-body := Content-Encoding( Content-Type( data ) )
Content-Type specifies the media type of the underlying data.
Content-Encoding may be used to indicate any additional content
codings applied to the data, usually for the purpose of data
compression, that are a property of the requested resource. There is
no default encoding.
Any HTTP/1.1 message containing an entity-body SHOULD include a
Content-Type header field defining the media type of that body. If
and only if the media type is not given by a Content-Type field, the
recipient MAY attempt to guess the media type via inspection of its
content and/or the name extension(s) of the URI used to identify the
resource. If the media type remains unknown, the recipient SHOULD
treat it as type "application/octet-stream".
7.2.2 Entity Length
The entity-length of a message is the length of the message-body
before any transfer-codings have been applied. Section 4.4 defines
how the transfer-length of a message-body is determined.

Related

HTTP request header field "optdata"

So, I googled a lot but couldn't find a HTTP request header field called "optdata". I am working on allowing the user to watch DRM protected videos on chromecast device. I am following a document from drmtoday.com which says:
For Widevine, the metadata must be carried inside the “optdata” field,
inside the HTTP request header with name “dt-custom-data” or
“x-dt-custom-data”.
I understand that the header key name is "dt-custom-data" but couldn't find any reference explaining what exactly is optdata.
If I just encode the following data to base64 and pass it as 'dt-custom-data' header, the request fails saying that 'HTTP Status 412 - Precondition failed'.
{
"userId":"12345",
"sessionId":"RWFzdGVyZWdn",
"merchant":"a-merchant"
}
It just means that dt-custom-data and x-dt-custom-data headers support key/value pairs. optdata is a recognised key.
E.g.
dt-custom-data: optdata={ "userId":"12345", "sessionId":"RWFzdGVyZWdn", "merchant":"a-merchant" }

Possible types of a HTTP header value

The type of a value passed through a query string is always a string. When a HTTP client need to send a number, let's say 42, in is actually "42". Everything inside the query string is actually a string.
Is it the same for the type of values passed through a header ?
In other words, if we send an HTTP request with a "FooBar" header with the value of 42, and another request where the value of the header set to "42", will the server perceive the two received values as of the same type and value (i.e. "42")?
It's always a string, even in HTTP/2
tldr; Headers are text, sometimes ISO 8859, but usually just US-ASCII.
According to the 2014 RFC7230 (last paragraph), HTTP fields have used to be text and new headers should continue to do so, restricting the values to consist of US-ASCII octets.
The 1982 RFC822 specifies ASCII as the format of the header body.
References (found through List of HTTP Headers):
Section 3.2 of RFC822
RFC7230

HTTP multirange requests - headers in response

I'm using multirange http requests like this
"curl --range 1-2,2-3 http://some.url"
The response is like
--00000000000000030705 Content-Type: text/html; charset=utf-8 Content-Range: bytes 1-2/13882393
il
--00000000000000030705 Content-Type: text/html; charset=utf-8 Content-Range: bytes 2-3/13882393
le
--00000000000000030705--
How can I remove fields Content-Type and Content-Range from response to get a raw data from server (without parsing on client side)?
I want to get response like:
"ille"
Thanks a lot!
You probably can't. The server is conforming to the spec, as described by the RFC.
If multiple parts are being transferred, the server generating the 206 response must generate a "multipart/byteranges" payload, as defined in Appendix A, and a Content-Type header field containing the multipart/byteranges media type and its required boundary parameter. To avoid confusion with single-part responses, a server must not generate a Content-Range header field in the HTTP header section of a multiple part response (this field will be sent in each part instead).
In the case of contiguous multi ranges the server may send the response without the multipart boundaries but this is optional.
When multiple ranges are requested, a server may coalesce any of the ranges that overlap, or that are separated by a gap that is smaller than the overhead of sending multiple parts, regardless of the order in which the corresponding byte-range-spec appeared in the received Range header field. Since the typical overhead between parts of a multipart/byteranges payload is around 80 bytes, depending on the selected representation's media type and the chosen boundary parameter length, it can be less efficient to transfer many small disjoint parts than it is to transfer the entire selected representation.
Within the header area of each body part in the multipart payload, the server must generate a Content-Range header field corresponding to the range being enclosed in that body part. If the selected representation would have had a Content-Type header field in a 200 (OK) response, the server should generate that same Content-Type field in the header area of each body part. For example:
Assuming your server conforms to the spec, sending a single range 1-3 you will get a single body.

What is the difference between a request payload and request body?

I am learning HTTP. I enclose a request payload in XML or JSON format in my POST requests. What I wanted to know is whether a request payload and request body mean the same thing?
Definition of: payload : The "actual data" in a packet or file minus all headers attached for transport and minus all descriptive meta-data. In a network packet, headers are appended to the payload for transport and then discarded at their destination.
Edit: In Http protocol, an http packet has http headers and http payload.So payload section of http packet may or may not have a body depending upon the type of request (e.g. POST vs GET). So payload and body are not the same thing.
Payload is the "wrapper" to the body
Payload is something one carries. A paperboy's payload is a pile of newspapers and a HTTP POST request's payload is whatever comes in the "body".
What I wanted to know is whether a request payload and request body mean the same thing?
No, they have different meanings. A payload (a.k.a. content) is a part of representation data while a body is a part of a message, which are two different HTTP concepts. A representation (data and metadata) is transferred as a single or multiple messages, so a message encloses a complete or partial representation. The representation metadata are enclosed in the header fields of a message and the representation data, the payload, are enclosed in the body of a message, as is or transfer-encoded.
References
RFC 9110: HTTP Semantics defines the term representation:
3.2. Representations
A "representation" is information that is intended to reflect a past, current, or desired state of a given resource, in a format that can be readily communicated via the protocol. A representation consists of a set of representation metadata and a potentially unbounded stream of representation data (Section 8).
Notice that the definition is independent of the version of HTTP because it is about semantics.
RFC 9112: HTTP/1.1 defines the term message:
2.1. Message Format
An HTTP/1.1 message consists of a start-line followed by a CRLF and a sequence of octets in a format similar to the Internet Message Format [RFC5322]: zero or more header field lines (collectively referred to as the "headers" or the "header section"), an empty line indicating the end of the header section, and an optional message body.
HTTP-message = start-line CRLF
*( field-line CRLF )
CRLF
[ message-body ]
Notice that the definition depends on the version of HTTP because it is about syntax.
RFC 9110: HTTP Semantics defines the term content:
6.4. Content
HTTP messages often transfer a complete or partial representation as the message "content": a stream of octets sent after the header section, as delineated by the message framing.
This abstract definition of content reflects the data after it has been extracted from the message framing. For example, an HTTP/1.1 message body (Section 6 of [HTTP/1.1]) might consist of a stream of data encoded with the chunked transfer coding -- a sequence of data chunks, one zero-length chunk, and a trailer section -- whereas the content of that same message includes only the data stream after the transfer coding has been decoded; it does not include the chunk lengths, chunked framing syntax, nor the trailer fields (Section 6.5).
Note: Some field names have a "Content-" prefix. This is an informal convention; while some of these fields refer to the content of the message, as defined above, others are scoped to the selected representation (Section 3.2). See the individual field's definition to disambiguate.
RFC 9110: HTTP Semantics substitutes the term content for payload used in previous RFCs:
B.3. Changes from RFC 7231
[…]
The terms "payload" and "payload body" have been replaced with "content", to better align with its usage elsewhere (e.g., in field names) and to avoid confusion with frame payloads in HTTP/2 and HTTP/3. (Section 6.4)
Header identifies source & destination of the sent packet, whereas the actual data i.e Body is referred to as Payload
The start-line and HTTP headers of the HTTP message are collectively known as the head of the requests, whereas its payload is known as the body
So Yes, they are the same thing.
Got this from https://developer.mozilla.org/en-US/docs/Web/HTTP/Messages
Payload of HTTP message is known as the body. link1
The HTTP message payload body is the information ("payload") part of the data that is sent in the HTTP Message Body (if any), prior to transfer encoding being applied. If transfer encoding is not used, the payload body and message body are the same thing! link2
So basically the only difference between HTTP message body and HTTP message payload body is encoding (but only if present). So generalizing the term request payload = request body.

Set more than one HTTP header with the same name?

As far as I know it is allowed by the HTTP spec to set more than one HTTP header with the same name. Is there any use case to do so (from client to server and vice versa)?
HTTP 1.1 Section 4.2:
Multiple message-header fields with
the same field-name MAY be present in
a message if and only if the entire
field-value for that header field is
defined as a comma-separated list
[i.e., #(values)]. It MUST be possible
to combine the multiple header fields
into one "field-name: field-value"
pair, without changing the semantics
of the message, by appending each
subsequent field-value to the first,
each separated by a comma. The order
in which header fields with the same
field-name are received is therefore
significant to the interpretation of
the combined field value, and thus a
proxy MUST NOT change the order of
these field values when a message is
forwarded.
If I'm not wrong there is no case where multiple headers with the same name are needed.
It's commonly used for Set-Cookie:. Many servers set more than one cookie.
Of course, you can always set them all in a single header.
Actually, I think you cannot set multiple cookies in one header. So that's a necessary use-case.
The Cookie spec (RFC 2109) does claim that you can combine multiple cookies in one header the same way other headers can be combined (comma-separated), but it also points out that non-conforming syntaxes (like the Expires parameter, which has ,s in its value) are still common and must be dealt with by implementations.
So, if you use Expires params in your Set-Cookie headers and you don't want all your cookies to expire at the same time, you probably need to use multiple headers.
Update: Evolution of the Cookie spec
RFC 2109 has been obsoleted by RFC 2965 that in turn got obsoleted by RFC 6265, which is stricter on the issue:
Origin servers SHOULD NOT fold multiple Set-Cookie header fields into a single header field. The usual mechanism for folding HTTP headers fields (i.e., as defined in [RFC2616]) might change the semantics of the Set-Cookie header field because the %x2C (",") character is used by Set-Cookie in a way that conflicts with such folding.
Side note
RFC 6265 uses the verb "folding" when it refers to combining multiple header fields into one, which is ambiguous in the context of the HTTP/1 specs (both by RFC2616, and its successor, RFC 7230) where:
"folding" consistently refers to line folding, and
the verb "combine" is used to describe merging same headers.
Combining header fields:
See RFC 2616, Section 4.2, Message Headers (quoted in the question), but searching for the for the word "combine" will bring up special cases.
The above item obsoleted by RFC 7230, Section 3.2.2, Field Order:
A recipient MAY combine multiple header fields with the same field name into one field-name: field-value pair, without changing the semantics of the message, by appending each subsequent field value to the combined field value in order, separated by a comma. The order in which header fields with the same field name are received is therefore significant to the interpretation of the combined field value; a proxy MUST NOT change the order of these field values when forwarding a message.
Note: In practice, the "Set-Cookie" header field (RFC6265) often appears multiple times in a response message and does not use the list syntax, violating the above requirements on multiple header fields with the same name. Since it cannot be combined into a single field-value, recipients ought to handle Set-Cookie as a special case while processing header fields. (See Appendix A.2.3 of [Kri2001] for details.)
Line folding:
From RFC 2616, Section 2.2, Basic Rules:
HTTP/1.1 header field values can be folded onto multiple lines if the continuation line begins with a space or horizontal tab. All linear white space, including folding, has the same semantics as SP. A recipient MAY replace any linear white space with a single SP before interpreting the field value or forwarding the message downstream.
The above section obsoleted by RFC 7230, Section 3.2.4, Field Parsing:
Historically, HTTP header field values could be extended over multiple lines by preceding each extra line with at least one space or horizontal tab (obs-fold). This specification deprecates such line folding except within the message/http media type (Section 8.3.1). A sender MUST NOT generate a message that includes line folding (i.e., that has any field-value that contains a match to the obs-fold rule) unless the message is intended for packaging within the message/http media type.
A server that receives an obs-fold in a request message that is not within a message/http container MUST either reject the message by sending a 400 (Bad Request), preferably with a representation explaining that obsolete line folding is unacceptable, or replace each received obs-fold with one or more SP octets prior to interpreting the field value or forwarding the message downstream.
A proxy or gateway that receives an obs-fold in a response message that is not within a message/http container MUST either discard the message and replace it with a 502 (Bad Gateway) response, preferably with a representation explaining that unacceptable line folding was received, or replace each received obs-fold with one or more SP octets prior to interpreting the field value or forwarding the message downstream.
A user agent that receives an obs-fold in a response message that is not within a message/http container MUST replace each received obs-fold with one or more SP octets prior to interpreting the field value.
Since duplicate headers can cause issues with various web-servers and APIs (regardless of what the spec says), I doubt there is any general purpose use case where this is best practice. That's not to say someone somewhere isn't doing it, of course.
As you're looking for use-cases, maybe Accept would be a valid one.
Accept: application/json
Accept: application/xml
It's only allowed for headers using a very specific format, see RFC 2616, Section 4.2.
Old thread, but I was looking into this same issue. Anyway, the Accept and Accept-Encoding headers are typical examples that uses multiple values, comma separated. Even if these are request specific header, the specs do not differentiate between request and response at this level. Check the one from this page.
What the spec says is that if you have commas as character in the value of the header, you cannot use multiple headers of the same name, unless you disambiguate the use of the comma.

Resources