ASP.NET MVC ignoring Content-Length? - asp.net

I've been having some problems with missing post data in ASP.NET MVC which has lead me to investigate how ASP.NET MVC deals with invalid content lengths. I had presumed that a post with a invalid content length should be ignored by MVC.NET but this doesn't seem to be the case.
As an example, try creating a new ASP.NET MVC 2 web application and add this action to the HomeController:
public ActionResult Test(int userID, string text)
{
return Content("UserID = " + userID + " Text = " + text);
}
Try creating a simple form that posts to the above action, run fiddler and (using "Request Builder") modify the raw data so that some of the form data is missing (e.g. remove the text parameter). Before executing the request, remember to un-tick the "Fix Content-Length header" checkbox under the Request Builder options then set a break point on the code above and execute the custom http request.
I find that the request takes a lot longer than normal (30 seconds or so) but to my amazement is still processed by the controllers action. Does anyone know if this is expected behavior and, if so, what would you recommend to safeguard against invalid content-lengths?

ASP.NET does not ignore the Content-Length request header. Consider the following controller action as an example which simply echoes back the foo parameter:
[HttpPost]
public ActionResult Index(string foo)
{
return Content(foo, "text/plain");
}
Now let's make a valid POST request to it:
using (var client = new TcpClient("127.0.0.1", 2555))
using (var stream = client.GetStream())
using (var writer = new StreamWriter(stream))
using (var reader = new StreamReader(stream))
{
writer.Write(
#"POST /home/index HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: localhost:2555
Content-Length: 10
Connection: close
foo=foobar");
writer.Flush();
Console.WriteLine(reader.ReadToEnd());
}
As expected this prints the response HTTP headers (which are not important) and in the body we have foobar. Now try reducing the Content-Length header of the request:
POST /home/index HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: localhost:2555
Content-Length: 5
Connection: close
foo=foobar
Which returns a single f in the response body. So as you can see an invalid HTTP request could lead to incorrect parsing of the parameters.

Related

ArangoDB can't send request with curl

I can't unserstand what I am doing wrong, but when I am sending next request with curl, I am getting error:
echo {"id":1,"question":"aaa"},{"id":2,"question":"bbb?"} | curl -X POST --data-binary #- --dump - http://localhost:8529/_db/otest/_api/document/?collection=sitetestanswers
HTTP/1.1 100 (Continue)
HTTP/1.1 400 Bad Request
Server: ArangoDB
Connection: Keep-Alive
Content-Type: application/json; charset=utf-8
Content-Length: 100
{"error":true,"errorMessage":"failed to parse json object: expecting EOF","code":400,"errorNum":600}
Any ideas? I tied wrap it's to [...]. Nothing do not help.
With [...] validator mark this as valid
Same with D. Here is my code:
void sendQuestionsToArangoDB(Json questions)
{
string collectionUrl = "http://localhost:8529/_db/otest/_api/document/?collection=sitetestanswers";
auto rq = Request();
rq.verbosity = 2;
string s = `{"id":"1","question":"foo?"},{"id":2}`;
auto rs = rq.post(collectionUrl, s, "application/json");
writeln("SENDED");
}
--
POST /_db/otest/_api/document/?collection=sitetestanswers HTTP/1.1
Content-Length: 37
Connection: Close
Host: localhost:8529
Content-Type: application/json
HTTP/1.1 400 Bad Request
Server: ArangoDB
Connection: Close
Content-Type: application/json; charset=utf-8
Content-Length: 100
100 bytes of body received
For D I use this lib: https://github.com/ikod/dlang-requests
Same issue with vibed.
ArangoDB do not understand JSON if it's come ass array like [...]. It should be passed as key-value. So if you need pass array it should have key mykey : [].
Here is working code:
import std.stdio;
import requests.http;
void main(string[] args)
{
string collectionUrl = "http://localhost:8529/_db/otest/_api/document?collection=sitetestanswers";
auto rq = Request();
rq.verbosity = 2;
string s = `{"some_data":[{"id":1, "question":"aaa"},{"id":2, "question":"bbb"}]}`;
auto rs = rq.post(collectionUrl, s, "application/json");
writeln("SENDED");
}
otest - DB name
sitetestanswers - collection name (should be created in DB)
echo '[{"id":1,"question":"aaa"},{"id":2,"question":"bbb?"}]'
should do the trick. You need to put ticks around the JSON. The array brackets are necessary otherwise this is not valid JSON.
You are trying to send multiple documents. The data in the original question separates the documents by comma ({"id":1,"question":"aaa"},{"id":2,"question":"bbb?"}) which is invalid JSON. Thus the failed to parse json object answer from ArangoDB.
Putting the documents into angular brackets ([ ... ]) as some of the commentors suggested will make the request payload valid JSON again.
However, you're sending the data to a server endpoint that handles a single document. The API for POST /_api/document/?collection=... currently accepts a single document at a time. It does not work with multiple documents in a single request. It expects a JSON object, and whenever it is sent something different it will respond with an error code.
If you're looking for batch inserts, please try the API POST /_api/import, described in the manual here: https://docs.arangodb.com/HttpBulkImports/ImportingSelfContained.html
This will work with multiple documents in a single request. ArangoDB 3.0 will also allow sending multiple documents to the POST /_api/document?collection=... API, but this version is not yet released. A technical preview will be available soon however.

How do I inject new request header with json data in proxy request flow?

I am trying to inject a new request header in the proxy request flow using JS policy to be sent to the backend server. When I look at the debug trace, I see that the json data in the request header is distorted.
I am trying to inject some string like
{"scope":"","time_till":2264,"id_1":"hUXLXVqpA1J4vA9sayk2UttWNdM","custom_data":{"c_id":"test_data"}}
But when I look at the trace window I see this
{"scope":"","time_till":2264,id_1":"hUXLXVqpA1J4vA9sayk2UttWNdM,"custom_data":{"c_id":"test_data"}}
what am I doing wrong?
var obj = {"scope":"","time_till":2264,"id_1":"hUXLXVqpA1J4vA9sayk2UttWNdM","custom_data":{"c_id":"test_data"}};
var header_str = JSON.stringify(obj);
context.setVariable('json-header',header_str);
request.headers['x-json-hedar']= header_str;
I tested your code and it seems to work. Here's an example response where I set the header string as a response:
HTTP/1.1 200 OK
User-Agent: curl/7.30.0
Accept: */*
x-json-header: {"scope":"","time_till":2264,"id_1":"hUXLXVqpA1J4vA9sayk2UttWNdM","custom_data":{"c_id":"test_data"}}
Content-Length: 0
It appears this is only an issue with the Apigee debug session / trace tool as the header value was set correctly. Here was the JSON download of the debug session showing this header value:
{
"name": "x-json-header",
"value": "{\"scope\":\"\",\"time_till\":2264,id_1\":\"hUXLXVqpA1J4vA9sayk2UttWNdM,\"custom_data\":{\"c_id\":\"test_data\"}}"
}
You can see that the value passed to the UI for displaying the debug info has the malformed json:
id_1\":\"hUXLXVqpA1J4vA9sayk2UttWNdM,
This does not appear to be a problem with the Apigee debug/trace UI. I see the malformed JSON trickle down to my backend service.
Here is the header I'm trying to send -
{"timeStamp":"2349218349381274","latitude":"34.589","longitude":"-37.343","clientIp":"127.0.0.0","deviceId":"MOBILE_TEST_DEVICE_AGAIN","macAddress":"23:45:345:345","deviceType":"phone","deviceOS":"iOS","deviceModel":"iPhone 5S","connection":"5G","carrier":"Vodafone","refererURL":"http://www.google.com","xforwardedFor":"129.0.0.0","sessionId":"kfkls498327ksdjf","application":"mobile-app","appVersion":"7.6.5","serviceVersion":"1.0","userAgent":"Gecko"}
But Apigee reads the header as below. Note the missing start quotes from some fields.
{"timeStamp":"2349218349381274",latitude":"34.589,longitude":"-37.343,clientIp":"127.0.0.0,deviceId":"MOBILE_TEST_DEVICE_AGAIN,macAddress":"23:45:345:345,deviceType":"phone,deviceOS":"iOS,deviceModel":"iPhone 5S,connection":"5G,carrier":"Vodafone,refererURL":"http://www.google.com,xforwardedFor":"129.0.0.0,sessionId":"kfkls498327ksdjf,application":"mobile-app,appVersion":"7.6.5,serviceVersion":"1.0,"userAgent":"Gecko"}
The header is used in a service callout to a backend service which parses it. And rightly so, I get the below error -
com.fasterxml.jackson.core.JsonParseException: Unexpected character ('l' (code 108)): was expecting double-quote to start field name
at [Source: java.io.StringReader#22549cdc; line: 1, column: 35]
at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1378)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:599)
at com.fasterxml.jackson.core.base.ParserMinimalBase._reportUnexpectedChar(ParserMinimalBase.java:520)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._handleUnusualFieldName(ReaderBasedJsonParser.java:1275)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser._parseFieldName(ReaderBasedJsonParser.java:1170)
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.nextToken(ReaderBasedJsonParser.java:611)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:301)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2796)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:1942)
I encounter strange behaviour when adding JSON to a context variable for example like the following:
var header_str = JSON.stringify(obj);
context.setVariable('json-header',header_str);
I appreciate this is an example so you may not have included the full extent of the problem but this normally works (now it is not added to a variable first):
request.headers['x-json-header'] = JSON.stringify(obj);
Code like this also works if you can send the request from JavaScript
var headers = {"Accept": "application/json", "Accept-Language": "en"};
var sessionRequest = new Request(url, 'POST', headers, body);
var exchange = httpClient.send(sessionRequest);
exchange.waitForComplete()
if (exchange.isSuccess()){
var responseObj = exchange.getResponse().content.asJSON;
if (responseObj.error){
request.content += JSON.stringify(responseObj);
}
}
Also, I have had success with using an AssignMessage policy to build a request, followed by a Callout policy to read the stored request and then make that request and store the result in a response object which can then be read by an Extract Variables policy.

spring-mvc The request sent by the client was syntactically incorrect. for json post

I am trying to build a springmvc app. On one of the action I want it to take a POST of JSON content. I am sending the following values:
POST /TestResults/create.json HTTP/1.1
Host: localhost:8080
Content-Type: application/json
Cache-Control: no-cache
[ { "test_name": "value1", "test_time": "111"} ]
However, when I send it I am getting the following error:
The request sent by the client was syntactically incorrect.
My action looks like this:
#RequestMapping(value="/create", method=RequestMethod.POST, consumes = "application/json")
#ResponseBody
public ModelAndView createTest(#RequestBody TestResult test_result) {
When I change the action to bepublic ModelAndView createTest(#RequestBody String test_result) { the action does succeed but I am using "String" at that time.
Is it possible to know what I might be doing incorrectly?
The JSON you are receiving is a JSON array. A JSON array unless you have a custom deserializer cannot be mapped to a type like TestResult, unless it's some subtype of Collection. You'll want to use something like
public ModelAndView createTest(#RequestBody List<TestResult> testResults) {
For #RequestBody String it works because all Spring has to do is read from the request body and convert the bytes into a String.

Web API as a Proxy and Chunked Transfer Encoding

I have been playing around with using Web API (Web Host) as a proxy server and have run into an issue with how my Web API proxy handles responses with the "Transfer-Encoding: chunked" header.
When bypassing the proxy, the remote resource sends the following response headers:
Cache-Control:no-cache
Content-Encoding:gzip
Content-Type:text/html
Date:Fri, 24 May 2013 12:42:27 GMT
Expires:-1
Pragma:no-cache
Server:Microsoft-IIS/8.0
Transfer-Encoding:chunked
Vary:Accept-Encoding
X-AspNet-Version:4.0.30319
X-Powered-By:ASP.NET
When going through my Web API based proxy, my request hangs unless I explicitly reset the TransferEncodingChunked property on the response header to false:
response.Headers.TransferEncodingChunked = false;
I admit, I don't fully understand what impact setting the TransferEncodingChunked property has, but it seems strange to me that in order to make the proxy work as expected, I need to set this property to false when clearly the incoming response has a "Transfer-Encoding: chunked" header. I am also concerned about side effects to explicitly setting this property. Can anyone help me understand what is going on and why setting this property is required?
UPDATE: So I did a little more digging into the difference in the response when going through the proxy vs. not. Whether I explicitly set the TransferEncodingChunked property to false, the response headers when coming through the proxy are exactly the same as when not going through the proxy. However, the response content is different. Here are a few samples (I turned off gzip encoding):
// With TransferEncodingChunked = false
2d\r\n
This was sent with transfer-encoding: chunked\r\n
0\r\n
// Without explicitly setting TransferEncodingChunked
This was sent with transfer-encoding: chunked
Clearly, the content sent with TransferEncodingChunked set to false is in fact transfer encoded. This is actually the correct response as it is what was received from the requested resource behind the proxy. What continues to be strange is the second scenario in which I don't explicitly set TransferEncodingChunked on the response (but it is in the response header received from the proxied service). Clearly, in this case, the response is NOT in fact transfer encoded by IIS, in spite of the fact that the actual response is. Strange...this is starting to feel like designed behavior (in which case, I'd love to know how / why) or a bug in IIS, ASP.Net, or Web API.
Here is a simplified version of the code I am running:
Proxy Web API application:
// WebApiConfig.cs
config.Routes.MapHttpRoute(
name: "Proxy",
routeTemplate: "{*path}",
handler: HttpClientFactory.CreatePipeline(
innerHandler: new HttpClientHandler(), // Routes the request to an external resource
handlers: new DelegatingHandler[] { new ProxyHandler() }
),
defaults: new { path = RouteParameter.Optional },
constraints: null
);
// ProxyHandler.cs
public class ProxyHandler : DelegatingHandler
{
protected override async System.Threading.Tasks.Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, System.Threading.CancellationToken cancellationToken)
{
// Route the request to my web application
var uri = new Uri("http://localhost:49591" + request.RequestUri.PathAndQuery);
request.RequestUri = uri;
// For GET requests, somewhere upstream, Web API creates an empty stream for the request.Content property
// HttpClientHandler doesn't like this for GET requests, so set it back to null before sending along the request
if (request.Method == HttpMethod.Get)
{
request.Content = null;
}
var response = await base.SendAsync(request, cancellationToken);
// If I comment this out, any response that already has the Transfer-Encoding: chunked header will hang in the browser
response.Headers.TransferEncodingChunked = false;
return response;
}
}
And my web application controller which creates a "chunked" response (also Web API):
public class ChunkedController : ApiController
{
public HttpResponseMessage Get()
{
var response = Request.CreateResponse(HttpStatusCode.OK);
var content = "This was sent with transfer-encoding: chunked";
var bytes = System.Text.Encoding.ASCII.GetBytes(content);
var stream = new MemoryStream(bytes);
response.Content = new ChunkedStreamContent(stream);
return response;
}
}
public class ChunkedStreamContent : StreamContent
{
public ChunkedStreamContent(Stream stream)
: base(stream) { }
protected override bool TryComputeLength(out long length)
{
length = 0L;
return false;
}
}
From an HttpClient standpoint, content chunking is essentially a detail of the transport. The content provided by response.Content is always de-chunked for you by HttpClient.
It looks like there's a bug in Web API that it doesn't correctly (re-)chunk content when requested by the response.Headers.TransferEncodingChunked property when running on IIS. So the problem is that the proxy is telling the client, via the headers, that the content is chunked when in fact it is not. I've filed the bug here:
https://aspnetwebstack.codeplex.com/workitem/1124
I think your workaround is the best option at the moment.
Also notice that you have multiple layers here that likely weren't designed/tested for proxying scenarios (and may not support it). On the HttpClient side, note that it will automatically decompress and follow redirects unless you turn that behavior off. At a minimum, you'll want to set these two properties:
http://msdn.microsoft.com/en-us/library/system.net.http.httpclienthandler.allowautoredirect.aspx
http://msdn.microsoft.com/en-us/library/system.net.http.httpclienthandler.automaticdecompression.aspx
On the WebApi/IIS side, you've found at least one bug, and it wouldn't be suprising to find others as well. Just be forewarned there may be bugs like this currently writing a proxy using these technologies outside their main design use cases.

Caching of Web Service not working when request has query string

I'm trying to implement the client-side caching of web service calls, and based on information from the web, I was able to do it according to the SetCachingPolicy() function as shown in code 1 below.
I was able to successfully get it working with a web method, RetrieveX, but not with method RetrieveY. I noticed that RetrieveX has no parameters and RetrieveY has one string parameter, and on inspection under Fiddler, the difference seems to be that the HTTP GET request of the web service call from RetrieveY has a query string for the parameter.
All HTTP GET web service calls so far without a query string is doing the caching properly, but not this call that has a query string in it.
Examination under Fiddler indicates that RetrieveX has the following caching information in output 1, and RetrieveY has the information in output 2.
Is this a limitation of this caching method or can I do something to get the client side caching of RetrieveY working?
Code 1: SetCachingPolicy
private void SetCachingPolicy()
{
HttpCachePolicy cache = HttpContext.Current.Response.Cache;
cache.SetCacheability(HttpCacheability.Private);
cache.SetExpires(DateTime.Now.AddSeconds((double)30));
FieldInfo maxAgeField = cache.GetType().GetField(
"_maxAge", BindingFlags.Instance | BindingFlags.NonPublic);
maxAgeField.SetValue(cache, new TimeSpan(0, 0, 30));
}
Code 2: RetrieveX
[System.Web.Services.WebMethod]
[System.Web.Script.Services.ScriptMethod(UseHttpGet = true)]
public string[] RetrieveX()
{
SetCachingPolicy();
// Implementation details here.
return array;
}
Code 3: RetrieveY
[System.Web.Services.WebMethod]
[System.Web.Script.Services.ScriptMethod(UseHttpGet = true)]
public string[] RetrieveY(string arg1)
{
SetCachingPolicy();
// Implementation details here.
return array;
}
Output 1: RetrieveX caching info
HTTP/200 responses are cacheable by default, unless Expires, Pragma, or Cache-Control headers are present and forbid caching.
HTTP/1.0 Expires Header is present: Wed, 12 Sep 2012 03:16:50 GMT
HTTP/1.1 Cache-Control Header is present: private, max-age=30
private: This response MUST NOT be cached by a shared cache.
max-age: This resource will expire in .5 minutes. [30 sec]
Output 2: RetrieveY caching info
HTTP/200 responses are cacheable by default, unless Expires, Pragma, or Cache-Control headers are present and forbid caching.
HTTP/1.0 Expires Header is present: -1
Legacy Pragma Header is present: no-cache
HTTP/1.1 Cache-Control Header is present: no-cache
no-cache: This response MUST NOT be reused without successful revalidation with the origin server.
I ran into this issue as well, I thought I'd share what worked for me. The underlying issue is that VaryByParams is not being set on the response. If you add this to your SetCachingPolicy() method RetrieveY should begin working as expected:
cache.VaryByParams["*"] = true

Resources