HTTP Header Key can be repeated? - http

In JAVA HttpUrlConnection , the main logic code of request Header settings as following:
public synchronized void set(String k, String v) {
for (int i = nkeys; --i >= 0;)
if (k.equalsIgnoreCase(keys[i])) {
values[i] = v;
return;
}
add(k, v);
}
It is verified that the key should be unique, the key has to keep one to one mapping relationship with the value.
On the contrary, in HeaderFields of Response module, structure is defined as Entry >. That is, the key does not keep one to one mapping relationship with the value.
Why is this? Does the HTTP protocol has relevant agreement?
Add:
In HttpClient4 ,the main logic code of request Header settings as following:
/**
* Replaces the first occurence of the header with the same name. If no header with
* the same name is found the given header is added to the end of the list.
*
* #param header the new header that should replace the first header with the same
* name if present in the list.
*/
public void updateHeader(final Header header) {
if (header == null) {
return;
}
// HTTPCORE-361 : we don't use the for-each syntax, i.e.
// for (Header header : headers)
// as that creates an Iterator that needs to be garbage-collected
for (int i = 0; i < this.headers.size(); i++) {
final Header current = this.headers.get(i);
if (current.getName().equalsIgnoreCase(header.getName())) {
this.headers.set(i, header);
return;
}
}
this.headers.add(header);
}
Header of response
/**
* Gets all of the headers with the given name. The returned array
* maintains the relative order in which the headers were added.
*
* <p>Header name comparison is case insensitive.
*
* #param name the name of the header(s) to get
*
* #return an array of length >= 0
*/
public Header[] getHeaders(final String name) {
final List<Header> headersFound = new ArrayList<Header>();
// HTTPCORE-361 : we don't use the for-each syntax, i.e.
// for (Header header : headers)
// as that creates an Iterator that needs to be garbage-collected
for (int i = 0; i < this.headers.size(); i++) {
final Header header = this.headers.get(i);
if (header.getName().equalsIgnoreCase(name)) {
headersFound.add(header);
}
}
return headersFound.toArray(new Header[headersFound.size()]);
}
They are the same of HttpUrlConnection

Does the HTTP protocol has relevant agreement?
Yes. RFC 2616 Section 4.2 "Message Headers" says:
Multiple message-header fields with the same field-name MAY be
present in a message if and only if the entire field-value for that
header field is defined as a comma-separated list [i.e., #(values)].
It MUST be possible to combine the multiple header fields into one
"field-name: field-value" pair, without changing the semantics of the
message, by appending each subsequent field-value to the first, each
separated by a comma. The order in which header fields with the same
field-name are received is therefore significant to the
interpretation of the combined field value, and thus a proxy MUST NOT
change the order of these field values when a message is forwarded.
This is expanded further by RFC 7230 Section 3.2.2 "Field Order":
A sender MUST NOT generate multiple header fields with the same field
name in a message unless either the entire field value for that
header field is defined as a comma-separated list [i.e., #(values)]
or the header field is a well-known exception (as noted below).
A recipient MAY combine multiple header fields with the same field
name into one "field-name: field-value" pair, without changing the
semantics of the message, by appending each subsequent field value to
the combined field value in order, separated by a comma. The order
in which header fields with the same field name are received is
therefore significant to the interpretation of the combined field
value; a proxy MUST NOT change the order of these field values when
forwarding a message.

Related

How to detect if HttpServletRequest has body?

I am trying to detect if an http request has body without reading it or doing anything that makes it impossible for the existing code to do with it whatever it does later in the flow.
boolean hasBody(HttpServletRequest http) {
boolean hasBody = false;
try {
hasBody = http.getInputStream() != null; // always gives true
hasBody = http.getReader().ready(); // always gives IllegalStateException
} catch (IOException e) {
}
return hasBody;
}
Unfortunately both checks I could come up with don't work when I test them both as GET and as POST with body.
Note, I don't want to do: "POST".equals(http.getMethod()) or !"GET".equals(http.getMethod()) if possible, 'cause I'm not sure what methods there could be with or without body.
You can use getContentLength() or getContentLengthLong() and check if it is positive.
A request that includes a body should have a certain headers set if it follows https://www.rfc-editor.org/rfc/rfc7230#section-3.3
The presence of a message body in a request is signaled by a
Content-Length or Transfer-Encoding header field.
You can check the presence of these headers with http.getHeader("transfer-encoding") != null || http.getHeader("content-length") != null.
The request body may be present but empty. If you want to catch that you could add a check for content length > 0, but that will only work if the request includes a content length header. It looks like you have to try to read the request body to see if it is empty when there is no content length header.

nginx module: capture whole response body

Although Nginx is a really interesting piece of software, the lack of documentation is making me crazy.
Goal: capture the whole response body, which would be logged on the server.
Problem: I have always a single buffer which size is ZERO.
Approach
I would expect to be able to accomplish this requirement with a body filter, which would "wait" for last_buf before iterate the full buffers chain.
/**
* #param ngx_http_request_t *r HTTP request
* #param ngx_chain_t *in Buffer chain
*/
static ngx_int_t
create_response_snapshot(ngx_http_request_t *r, ngx_chain_t *in)
{
ngx_chain_t *chain = NULL;
int chain_contains_last_buffer = 0;
size_t buffer_size = 0;
// check if body is complete
chain = in;
for ( ; ; )
{
if (chain->buf->last_buf)
{
chain_contains_last_buffer = 1;
}
if (NULL == chain->next)
break;
chain = chain->next;
}
if (0 == chain_contains_last_buffer)
{
// response is not complete
return ngx_http_next_body_filter(r, in);
}
// Response Content-Length
ngx_log_error(NGX_LOG_ALERT,r->connection->log,0,"Content-Length: %d",
r->headers_out.content_length_n);
// lets iterate buffers chain
for (chain = in; NULL != chain; chain = chain->next)
{
buffer_size = ngx_buf_size(chain->buf);
ngx_log_error(NGX_LOG_ALERT,r->connection->log,0,"buffer_size#%d",buffer_size);
}
return ngx_http_next_body_filter(r, in);
}
My comment got too big to be a comment, but I don't feel like it's a proper answer - oh well.
To re-iterate, the problem with the code you've posted is that your module's body filter function won't be called on the whole chain at once. It gets called on the first piece, then the second piece, until the nth piece. Finally it gets called on a completely empty chain, for whatever reason the buf with last_buf = 1 is always by itself and empty.
So I think what you want to do is "dam" the flow of buffers by accumulating them in your module without releasing any to the next filter until you have all of them at once.
Check out the substitution filter module: http://lxr.nginx.org/source//src/http/modules/ngx_http_sub_filter_module.c
It uses a "busy" chain which is what I was referring to. From what I've been able to tell it uses it to keep track of which buffers have actually been sent (when this happens the size gets set to zero) and adds those to the module context's free list for re-use. See ngx_http_sub_output on line 438 for this behavior.
My suggestion was to do something like what that module does, except without calling the next filter until you have the entire page. You can't call next_filter if you want to process the entire page as a whole, since doing that will result in data getting sent to the client. Again this runs counter to Nginx's design, so I think you should find an alternative that doesn't require the whole response body at once if you can.

Setting the Content-Type of an empty response in ASP.NET MVC

In order to support a legacy application that's in the field, I need my ASP.NET MVC app to return an empty response that also has a Content-Type. One of IIS, ASP.NET, or ASP.NET MVC is removing my Content-Type when I send back a null response. Is there any way around this?
(While not requiring an empty response with a set Content-Type would obviously be the ideal solution, the clients are already out there, and many of them cannot be upgraded.)
EDIT: Since there was a request for code: I'm proxying the request from the new web application to the one that older clients rely on. To do this, I have a subclass of ActionResult, called LegacyResult, that you can simply return for those methods that need to be handled by the old software. This is the relevant part of its code:
public override void ExecuteResult(ControllerContext context)
{
using (var legacyResponse = GetLegacyResponse(context))
{
var clientResponse = context.HttpContext.Response;
clientResponse.Buffer = false;
clientResponse.ContentType = legacyResponse.ContentType; /* Yes, I checked that legacyResponse.ContentType is never string.IsNullOrEmpty */
if (legacyResponse.ContentLength >= 0) clientResponse.AddHeader("Content-Length", legacyResponse.ContentLength.ToString());
var legacyInput = legacyResponse.GetResponseStream();
using (var clientOutput = clientResponse.OutputStream)
{
var rgb = new byte[32768];
int cb;
while ((cb = legacyInput.Read(rgb, 0, rgb.Length)) > 0)
{
clientOutput.Write(rgb, 0, cb);
}
clientOutput.Flush();
}
}
}
If legacyInput has data, then Content-Type is set appropriately. Otherwise, it's not. I can actually kluge the old backend to send an empty v. non-empty response for exactly the same request, and observe the difference in Fiddler.
EDIT 2: Poking around with Reflector reveals that, if headers have not been written at the time that HttpResponse.Flush is called, then Flush writes out the headers itself. The problem is that it only writes out a tiny subset of the headers. One of the missing ones is Content-Type. So it seems that, if I can force headers out to the stream, I can avoid this problem.
You have to trick the response into writing the headers, by falsely telling it there's content, then suppressing it:
/// [inside the writing block]
var didWrite = false;
while ((cb = legacyInput.Read(rgb, 0, rgb.Length)) > 0)
{
didWrite = true;
clientOutput.Write(rgb, 0, cb);
}
if (!didWrite)
{
// The stream needs a non-zero content length to write the correct headers, but...
clientResponse.AddHeader("Content-Length", "1");
// ...this actually writes a "Content-Length: 0" header with the other headers.
clientResponse.SuppressContent = true;
}

Change Single URL query string value

I have an ASP.NET page which takes a number of parameters in the query string:
search.aspx?q=123&source=WebSearch
This would display the first page of search results. Now within the rendering of that page, I want to display a set of links that allow the user to jump to different pages within the search results. I can do this simply by append &page=1 or &page=2 etc.
Where it gets complicated is that I want to preserve the input query string from the original page for every parameter except the one that I'm trying to change. There may be other parameters in the url used by other components and the value I'm trying to replace may or may not already be defined:
search.aspx?q=123&source=WebSearch&page=1&Theme=Blue
In this case to generate a link to the next page of results, I want to change page=1 to page=2 while leaving the rest of the query string unchanged.
Is there a builtin way to do this, or do I need to do all of the string parsing/recombining manually?
You can't modify the QueryString directly as it is readonly. You will need to get the values, modify them, then put them back together. Try this:
var nameValues = HttpUtility.ParseQueryString(Request.QueryString.ToString());
nameValues.Set("page", "2");
string url = Request.Url.AbsolutePath;
string updatedQueryString = "?" + nameValues.ToString();
Response.Redirect(url + updatedQueryString);
The ParseQueryString method returns a NameValueCollection (actually it really returns a HttpValueCollection which encodes the results, as I mention in an answer to another question). You can then use the Set method to update a value. You can also use the Add method to add a new one, or Remove to remove a value. Finally, calling ToString() on the name NameValueCollection returns the name value pairs in a name1=value1&name2=value2 querystring ready format. Once you have that append it to the URL and redirect.
Alternately, you can add a new key, or modify an existing one, using the indexer:
nameValues["temp"] = "hello!"; // add "temp" if it didn't exist
nameValues["temp"] = "hello, world!"; // overwrite "temp"
nameValues.Remove("temp"); // can't remove via indexer
You may need to add a using System.Collections.Specialized; to make use of the NameValueCollection class.
You can do this without all the overhead of redirection (which is not inconsiderable). My personal preference is to work with a NameValueCollection which a querystring really is, but using reflection:
// reflect to readonly property
PropertyInfo isReadOnly = typeof(System.Collections.Specialized.NameValueCollection).GetProperty("IsReadOnly", BindingFlags.Instance | BindingFlags.NonPublic);
// make collection editable
isReadOnly.SetValue(this.Request.QueryString, false, null);
// remove
this.Request.QueryString.Remove("foo");
// modify
this.Request.QueryString.Set("bar", "123");
// make collection readonly again
isReadOnly.SetValue(this.Request.QueryString, true, null);
Using this QueryStringBuilder helper class, you can grab the current QueryString and call the Add method to change an existing key/value pair...
//before: "?id=123&page=1&sessionId=ABC"
string newQueryString = QueryString.Current.Add("page", "2");
//after: "?id=123&page=2&sessionId=ABC"
Use the URIBuilder Specifically the link textQuery property
I believe that does what you need.
This is pretty arbitrary, in .NET Core at least. And it all boils down to asp-all-route-data
Consider the following trivial example (taken from the "paginator" view model I use in virtually every project):
public class SomeViewModel
{
public Dictionary<string, string> NextPageLink(IQueryCollection query)
{
/*
* NOTE: how you derive the "2" is fully up to you
*/
return ParseQueryCollection(query, "page", "2");
}
Dictionary<string, string> ParseQueryCollection(IQueryCollection query, string replacementKey, string replacementValue)
{
var dict = new Dictionary<string, string>()
{
{ replacementKey, replacementValue }
};
foreach (var q in query)
{
if (!string.Equals(q.Key, replacementKey, StringComparison.OrdinalIgnoreCase))
{
dict.Add(q.Key, q.Value);
}
}
return dict;
}
}
Then to use in your view, simply pass the method the current request query collection from Context.Request:
<a asp-all-route-data="#Model.NextPageLink(Context.Request.Query)">Next</a>

Asp.NET 1.1 HttpResponse headers

i have part of Asp.NET 1.1 project.
I work with remote site, which works incorrect in some cases - sometimes it write incorrect Content-Encoding header.
In my code i get HttpResponse from this remote site. And if Content-Encoding header is equals, for example, "gzip", i need to set Content-Encoding header to "deflate".
But there is no properties or methods in HttpResponse class to get Content-Encoding header.
Content-Encoding property returns, in my case, "UTF-8". In Watch window i see _customProperties field, which contain wrong string value. How can i change header value with Asp.NET 1.1?
There is no way to change custom headers in Asp.NET 1.1.
I solve problem only using reflection.
// first of all we need get type ArrayList with custom headers:
Type responseType = Response.GetType();
ArrayList fieldCustomHeaders = ArrayList)responseType.InvokeMember("_customHeaders",BindingFlags.GetField|BindingFlags.Instance|BindingFlags.NonPublic, null, Response,null);
// next we go thru all elements of list and search our header
for(int i=0; i < fieldCustomHeaders.Count; i++)
{
// see all headers
PropertyInfo propHeaderName = fieldCustomHeaders[i].GetType().GetProperty("Name", BindingFlags.Instance|BindingFlags.NonPublic);
String headerName = (String)propHeaderName.GetValue(fieldCustomHeaders[i], null);
// if we find needed header
if(headerName == "Content-Encoding")
{
// get value of header from its field
FieldInfo fieldHeaderValue = _fieldCustomHeaders[i].GetType().GetField("_value", BindingFlags.Instance|BindingFlags.NonPublic);
String headerValue = (String)fieldHeaderValue.GetValue(fieldCustomHeaders[i]);
// if we find needed value
if (headerValue == "gzip")
{
// just set new value to it
fieldHeaderValue.SetValue(_fieldCustomHeaders[i], "deflate");
break;
}
}
}

Resources