Strange characters preceding and following HTML in HTTP request - http

Background
I am building a custom HTTP parser in C++/CX using sockets. As such, I have full control over the entire HTTP request and response.
Request
GET /posts/html-android-app?referrer=rss HTTP/1.1
Host: mixturatech.com
Connection: close
Response
HTTP/1.1 200 OK
Date: Thu, 30 Apr 2015 04:44:59 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Access-Control-Allow-Origin: *
Cache-Control: public
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
6a2f
<!DOCTYPE html>
[trimmed document content]
</html>
0
Additional Data
If I navigate to the webpage with Chrome, WireShark captures the same data that I am seeing (with extraneous characters), yet Chrome manages to trim that content out. (I am looking at Chrome's data in the Network tab in Developer Tools.)
I do not see this problem on every site I retrieve, but the problem, if it exists, seems to be sitewide.
Questions
What is up with the 6a2f and 0 preceding and following the document?
Is this an encoding issue?
Is there some way that I can positively identify, without hardcoding boundaries for the document, such as it must start with < and end with >, where the actual content lies?
Will those characters, if they exist in a page, always be limited to length 4 and 1 respectively?


This is "chunked transfer encoding". Read http://greenbytes.de/tech/webdav/rfc7230.html#chunked.encoding.

Related

How to use TCP send out a HTTP response?

I try to use c++ develop a HTTP server on Windows,and when i reponse a HTTP by use WSASend to send out
char response[] =
"HTTP/1.1 200 OK\r\n\
Date: Mon, 27 Jul 2009 12:28:53 GMT\n\r\
Server: Apache/2.2.14 (Win32)\n\r\
Last-Modified: Wed, 22 Jul 2009 19:15:56 GMT\n\r\
Content-Length: 88\n\r\
Content-Type: text/html\n\r\
Connection: Closed\n\r\n\r\
<html><body><h1>Hello, World!</h1></body></html>"
Althrougn the browser did show Hello, World! when i type in 127.0.0.1,but the browser just keep show loading sigh as if the pages not yet load complete.And the browser's console never show the response message.Why?
Is there some format issue with my response message?

Content-Length: 88\n\r\
....
Connection: Closed\n\r\n\r\
There are several problems with your code. All over your code you use \n\r instead of \r\n. Therefore the response is invalid HTTP. And the Content-length header must reflect the actual length of the body: <html><body><h1>Hello, World!</h1></body></html> has 48 bytes and not 88 bytes as your code claims. Apart from that it must be Connection: Close instead of Connection: Closed.
Note that HTTP is way more complex than you think. If you really need to implement it yourself instead of using established libraries please study the actual standard (that's what standards are for!) instead of fiddling around until it seems to work. Otherwise it might work only within your specific environment and with a specific browser and you'll get strange problems later.

What is a chunked request in OpenTSDB?

I'm using Qt4 to post some data points to a OpenTSDB server, which doesn't supports chunked HTTP requests.
The code is basically this:
QNetworkRequest request(m_url);
request.setHeader(QNetworkRequest::ContentTypeHeader, QString("application/json"));
request.setHeader(QNetworkRequest::ContentLengthHeader, jsonRequest.toAscii().size());
m_networkAccessManager.post(request, jsonRequest.toAscii());
jsonRequest is a QString containing the data points. This code is called from time to time to upload data to the server, and it usually works fine. However, sometimes I receive an error from openTSDB stating that "Chunked request not supported.".
This seems to happen when the request gets a little bigger (and by bigger, I mean some KB of data).
edit:
I've done a tcpdump of the request when the problem arises, and in fact it doesn't seen to be chunked:
POST /api/put HTTP/1.1
Content-Type: application/json
Content-Length: 14073
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en,*
User-Agent: Mozilla/5.0
Host: 192.168.xx.xxx:xxxx
[{"metric":"slt.reader.temperature","timestamp":1420736269427,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736280628,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736291637,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736302748,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736313840,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736325011,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736336039,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736347182,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736358210,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736369372,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736380401,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736385286,"value":0,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736385286,"value":10,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736385286,"value":7,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736385287,"value":6,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736385287,"value":13,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736385287,"value":99,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736385287,"value":102,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736385287,"value":93,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.transactionsNotDeciphered","timestamp":1420736385287,"value":0,"tags":{"sltId":"5036","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736391436,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736402608,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736413642,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736424676,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736435823,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736446850,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736458007,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736469060,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736480207,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736491418,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736502620,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736513638,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736524682,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736535712,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736546742,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736557834,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736568858,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736579932,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736590966,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736601993,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736613183,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736624357,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736635387,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736646414,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736657493,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736668624,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736679743,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736685286,"value":0,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736685286,"value":8,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736685286,"value":9,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736685295,"value":5,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736685295,"value":4,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736685295,"value":88,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736685295,"value":130,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736685296,"value":123,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.transactionsNotDeciphered","timestamp":1420736685296,"value":0,"tags":{"sltId":"5036","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736690786,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736701910,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736712968,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736723999,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736735075,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736746106,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736757266,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736768455,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736779473,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736790606,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736801633,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736812713,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736823740,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736834856,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736845958,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736857103,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736868216,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736879292,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736890320,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736901503,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736912608,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736923761,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736934850,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736946033,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736957061,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736968223,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736979256,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736985284,"value":0,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736985285,"value":16,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736985285,"value":9,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736985285,"value":11,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736985285,"value":9,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736985285,"value":162,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736985285,"value":166,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736985285,"value":157,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.transactionsNotDeciphered","timestamp":1420736985286,"value":0,"tags":{"sltId":"5036","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736990353,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737001532,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737012658,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737023691,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737034823,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737045906,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737056942,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737068032,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}}]HTTP/1.1 400 Bad Request
Content-Length: 1080
Content-Type: text/html; charset=UTF-8
Date: Thu, 08 Jan 2015 17:18:43 GMT
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head><meta http-equiv=content-type content="text/html;charset=utf-8"><title>Bad Request</title>
<style><!--
body{font-family:arial,sans-serif;margin-left:2em}A.l:link{color:#6f6f6f}A.u:link{color:green}.subg{background-color:#e2f4f7}.fwf{font-family:monospace;white-space:pre-wrap}//--></style></head>
<body text=#000000 bgcolor=#ffffff><table border=0 cellpadding=2 cellspacing=0 width=100%><tr><td rowspan=3 width=1% nowrap><b><font color=#c71a32 size=10>T</font><font color=#00a189 size=10>S</font><font color=#1a65b7 size=10>D</font> </b><td> </td></tr><tr><td class=subg><font color=#507e9b><b>Looks like it's your fault this time</b></td></tr><tr><td> </td></tr></table><blockquote><h1>Bad Request</h1>Sorry but your request was rejected as being invalid.<br/><br/>The reason provided was:<blockquote>Chunked request not supported.</blockquote></blockquote><table width=100% cellpadding=0 cellspacing=0><tr><td class=subg><img alt="" width=1 height=6></td></tr></table></body></html>
I was thinking Qt was changing to use a chunked request when the request got larger, but it's not the case. So the current question is: what is a chunked request in openTSDB?

I've finally found the reason it thinks my request is chunked: OpenTSDB internally uses Netty for networking, and if Netty reads a block that doesn't contain the complete request, then it's flagged as chunked, even if there's no Transfer-Encoding header in the request.
There are two possible solutions for this:
adding "tsd.http.request.enable_chunked = true" to the config file, so that it doesn't reject chunked requests and adding "tsd.http.request.max_chunk = " to the config file, and setting it to a sensible size for your requests
If you can change the client, force it to send smaller requests
Please note that enabling chunked requests for public facing servers may not be a good idea, as a malicious client can send unbounded requests, overloading the server.
Credits goes to irc users jaylr and manolama, who provided me the info on irc channel #opentsdb#freenode.net

Testing specific Azure Web Site Instance

I have an Azure Web Site configured to use multiple (2) instances:
I have a service bus that should pass messages (ie Cache Evict) between the instances. I need to test this mechanism.
In a conventional (on premise) system I would point a browser to instance 1 (ie http://myserver1.example.com), perform an action, then point my browser to the other instance (http://myserver2.example.com) to test.
However, in Azure I can't see a way to hit a specific instance. Is it possible? Or is there an alternative way to to run through this test scenario (act on instance 1, ensure instance 2 behaves appropriately)?

Unfortunately, there isn't an official way of doing this. However, you can achieve that by setting a cookie called ARRAffinity on your request.
Try hitting your site from any client (Chrome, Firefox, curl, httpie, etc) and inspect the response headers that you are getting back.
For example in curl you would do
curl -I <siteName>.azurewebsites.net
you would get this
HTTP/1.1 200 OK
Content-Length: 2
Content-Type: text/html
Last-Modified: Wed, 17 Sep 2014 16:57:26 GMT
Accept-Ranges: bytes
ETag: "2ba0757598d2cf1:0"
Server: Microsoft-IIS/8.0
X-Powered-By: ASP.NET
Set-Cookie: ARRAffinity=<very long hash>; Path=/;Domain=<siteName>.azurewebsites.net
Date: Fri, 28 Nov 2014 03:13:07 GMT
what you are interested in is the ARRAFinity if you send couple of request you would notice that hash will keep changing between 2 values that represent your 2 instances.
Set that in your Cookie header on your request will guarantee it going to one of the instances and not the other.
curl --cookie ARRAfinity=<one of the hashes you got> <siteName>.azurewebsites.net

Why do my HTTP/1.0 GET request returns OK but no body content?

I'm making a simple http page-requester in C. It uses sockets to send HTTP/1.0 GET requests to hosts, and parses the answer to effectively download the html file.
However, when i send a request like this:
GET http://stackoverflow.com/questions HTTP/1.0
User-Agent: myRequester/1.0
It returns this
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=utf-8
Date: Mon, 19 Dec 2011 15:28:08 GMT
Content-Length: 54362
Connection: close
But no body content.
Yes, I've put CRLF on the end of every line and a blank line at the end.
I use only one socket through one connection. And i also have to stick to HTTP/1.0.

The most likely explanation is that the server is actually sending a body but you are not reading all of it. Most networking systems do not necessarily return all of the response in one function call, because they see it useful that what data is available immediately is returned immediately, even if more is expected.
The Unix system call recv returns zero when the connection has ended. You should keep calling it until you get zero or an error.

Chunked Transfer Encoding problem with Apache Abdera

I'm using Apache Abdera to POST atom multipart data to my server, and am having some odd problems that I can't pin down.
It looks like an issue with chunked transfer encoding, but I'm insufficiently experienced to be certain. The problem manifests as the server throwing an error indicating that the request I sent it contains only one mime part, not two as required. I attached Wireshark to the interface and captured the conversation, and it went like this:
POST /sss/col-uri/2ee98ea1-f9ad-4f01-9b1c-cfa3c4a6dc3c HTTP/1.1
Host: localhost
Expect: 100-continue
Transfer-Encoding: chunked
Content-Type: multipart/related; boundary="1306399868259";type="application/atom+xml;type=entry"
The server's response:
HTTP/1.1 100 Continue
My client continues:
198
--1306399868259
Content-Type: application/atom+xml;type=entry
Content-Disposition: attachment; name="atom"
<entry xmlns="http://www.w3.org/2005/Atom"><title xmlns="http://purl.org/dc/terms/">Richard Woz Ere</title><bibliographicCitation xmlns="http://purl.org/dc/terms/">this is my citation</bibliographicCitation><content type="application/zip" src="cid:48bd9436-e8b6-4f68-aa83-5c88eda52fd4" /></entry>
0
b0e9
--1306399868259
Content-Type: application/zip
Content-Disposition: attachment; name="payload"; filename="example.zip"
Content-ID: <48bd9436-e8b6-4f68-aa83-5c88eda52fd4>
Packaging: http://purl.org/net/sword/package/SimpleZip
And at this point the server responds with:
HTTP/1.1 400 Bad Request
Date: Thu, 26 May 2011 08:51:08 GMT
Server: Apache/2.2.17 (Unix) mod_ssl/2.2.17 OpenSSL/0.9.8l DAV/2 mod_wsgi/3.3 Python/2.6.1
Connection: close
Transfer-Encoding: chunked
Content-Type: text/xml
Indicating the error (which is well understood). My server goes on to stream a pile of base64 encoded bits onto the output stream, but in the mean time the server is not listening, it has already decided that the request was erroneous.
Unfortunately, I'm not in charge of the HTTP layer - this is all handled by Abdera using Apache httpclient. My code that does this looks like this:
client.execute("POST", url.toString(), new SWORDMultipartRequestEntity(deposit), options);
Here, the SWORDMultipartRequestEntity is a copy of the standard Abdera MultipartRequestEntity class, with a few extra headers thrown in (see, for example, Packaging in the above snippet); the "deposit" argument is just an object holding the atom part and the inputstream.
When attaching a debugger I get to this line of code fine, and then it disappears into a rat hole and then I get this error back.
Any hints or tips? I've pretty much exhausted my angles of attack!
The only thing that stands out for me is that immediately after the atom:entry document, there is a newline with "0" on it alone, which appears to be chunked transfer encoding speak for "I'm finished". Not sure how it got there, or whether it really has any effect. Help much appreciated.
Cheers,
Richard

The lonely 0 may indeed be a problem. My uninformed guess is that it results from some call to flush(), which then writes the whole buffer as another HTTP chunk. Unfortunately at the point where flush is called, the buffer had already been flushed and its size is therefore zero. So the HttpChunkedOutputFilter (or however it is called) should be taught than an empty buffer does not need to be flushed.
[update:] You should set a breakpoint in the ChunkedOutputStream class, especially the flush method. I just looked at its code and it seems to be ok, but maybe I missed something.

Resources