HTTP Packets, Whats Happening? - http
basically, i was wiresharking packets on my PS3 while viewing Motorstorm Leaderboards. The leaderboards are sent to my ps3 in XML format but only after i have been authorised. So can someone please tell me what is happening between these three packets and how i could replicate it in a browser?
Packet 1 From my PS3 to Sony Servers
POST /ranking_view/func/get_player_rank HTTP/1.1
Host: ranking-view-a01.u0.np.community.playstation.net
Connection: Keep-Alive
Content-Length: 213
Authorization: Digest username="c7y-ranking01", realm="c7y-ranking", nonce="2SpsV4WABAA=47a2b36030cd94de1190f6b9f05db1bd5584bc2a", uri="/ranking_view/func/get_player_rank", qop="auth", nc="00000001", cnonce="d4eb1eb60ab4efaea1476869d83a6e0b", response="96b55c6e79f84dd41b46eb66bed1c167"
Accept-Encoding: identity
User-Agent: PS3Community-agent/1.0.0 libhttp/1.0.0
<?xml version="1.0" encoding="utf-8"?><ranking platform="ps3" sv="3.15"><titleid>NPWR00012_00</titleid><board>7</board><jid>Panzerborn#a5.gb.np.playstation.net</jid><option message="false" info="false"/></ranking>
Packet 2 Sony Server Response to my PS3
Date: Fri, 26 Feb 2010 19:06:12 GMT
WWW-Authenticate: Digest realm="c7y-ranking", nonce="a3PFl4WABAA=6d375259676ec79641448a8032a795b8e12ccae4", algorithm=MD5, stale=true, qop="auth"
Content-Length: 401
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>
Packet 3 PS3 response to Sony Servers last packet
POST /ranking_view/func/get_player_rank HTTP/1.1
Host: ranking-view-a01.u0.np.community.playstation.net
Connection: Keep-Alive
Authorization: Digest username="c7y-ranking01", realm="c7y-ranking", nonce="a3PFl4WABAA=6d375259676ec79641448a8032a795b8e12ccae4", uri="/ranking_view/func/get_player_rank", qop="auth", nc="00000001", cnonce="58869490a891002d8c56573496274a3a", response="ca3d6f252d4e398b8f751c201a3f8f08"
Accept-Encoding: identity
User-Agent: PS3Community-agent/1.0.0 libhttp/1.0.0
<?xml version="1.0" encoding="utf-8"?><ranking platform="ps3" sv="3.15"><titleid>NPWR00012_00</titleid><board>7</board><jid>Panzerborn#a5.gb.np.playstation.net</jid><option message="false" info="false"/></ranking>
I tried to replicate this in Firefox and tamper headers as well as in PHP cURL but im getting nowhere. I assume it is to do with the nonce, cnonce and responce variables that keep changing >< please help =)
Nonce, cnonce and so on are related to HTTP Digest Authentication, which is an authentication mechanism that enables authentication without sending a password in plain text. So if you want to cheat in your PS3 game, you'll first have to hack that password out of the MD5 hash, I guess.
And it's not called HTTP packets, on layer 7 you would usually say request/response or similar.
The nonce an nonce and cnonce look like hash codes.
One possible defense mechanism against cheaters could be this:
def ps3client_send_score():
score = "bazillion points"
nonce = md5(score + "something you don't know about")
send_to_server(score, nonce)
On the server side:
def get_client_score(score, nonce):
if md5(score+"something you don't know about")==nonce:
accept_score(score)
else:
reject_score_and_ban_the_fool_if_he_continues_this()
So unless you want to spend weeks trying to find the salt deep in your game, forget it.
Related
How to read data sent to app-measurement from firebase
I'm trying to do a network analysis of an Android app to see what information Firebase is collecting. I'm doing a man-in-the-middle using Fiddler and can observe pings to Firebase's app-measurement server. But when I decrypt the content to try to see what's being passed, it looks like the message is encoded or compressed somehow. The messages look like this when opened in a text editor: POST https://app-measurement.com/a HTTP/1.1 Content-Length: 761 User-Agent: Dalvik/2.1.0 (Linux; U; Android 5.1; Google Nexus 6 Build/LMY47D) Host: app-measurement.com Connection: Keep-Alive Accept-Encoding: gzip Content-Type: application/x-www-form-urlencoded öR _c _oauto _r _pfo _sys _uwa _sysu _fΞ¶Þ. " _oauto _et_eΞ¶Þ. D _si¹þ½‚ÌÓ¹¨ _vsÓ랶Þ. ÎÏž¶Þ.Ξ¶Þ._fot €ºÀ¶Þ.Ξ¶Þ._fi ³Ÿ¶Þ._lte œ ²Ÿ¶Þ.(Ξ¶Þ.0Ó랶Þ.BandroidJ5.1RGoogle Nexus 6Zen-us`Ôýÿÿÿÿÿÿÿjmanual_installr<my-app-bundle-name>‚4.6.19166.0322ˆÄwÄwª <text here>°ñÊîüƒÝÏʸÊ'1:849883241272:android:<my-aaid> ò dVZyNR1YbAMøºç’1d5adf7b1442fe22˜ŽºªÜàÙè ð Is anyone familiar with this encoding? How can I decode to view the raw text? I've tried gzip, but that doesn't seem to be it. Thank you!
It's not compressed or really obfuscated in any way. You can see plaintext data right there in the payload if you look carefully. What you're capturing is almost certainly a protobuf in binary form, and you'll have to get a hold of the protobuf definition in order to decode it without putting in the work to reverse engineer it. That's not something Google discloses for API endpoints that are not publicly documented.
What is a chunked request in OpenTSDB?
I'm using Qt4 to post some data points to a OpenTSDB server, which doesn't supports chunked HTTP requests.
The code is basically this:
QNetworkRequest request(m_url);
request.setHeader(QNetworkRequest::ContentTypeHeader, QString("application/json"));
request.setHeader(QNetworkRequest::ContentLengthHeader, jsonRequest.toAscii().size());
m_networkAccessManager.post(request, jsonRequest.toAscii());
jsonRequest is a QString containing the data points. This code is called from time to time to upload data to the server, and it usually works fine. However, sometimes I receive an error from openTSDB stating that "Chunked request not supported.".
This seems to happen when the request gets a little bigger (and by bigger, I mean some KB of data).
edit:
I've done a tcpdump of the request when the problem arises, and in fact it doesn't seen to be chunked:
POST /api/put HTTP/1.1
Content-Type: application/json
Content-Length: 14073
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en,*
User-Agent: Mozilla/5.0
Host: 192.168.xx.xxx:xxxx
[{"metric":"slt.reader.temperature","timestamp":1420736269427,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736280628,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736291637,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736302748,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736313840,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736325011,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736336039,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736347182,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736358210,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736369372,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736380401,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736385286,"value":0,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736385286,"value":10,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736385286,"value":7,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736385287,"value":6,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736385287,"value":13,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736385287,"value":99,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736385287,"value":102,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736385287,"value":93,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.transactionsNotDeciphered","timestamp":1420736385287,"value":0,"tags":{"sltId":"5036","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736391436,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736402608,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736413642,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736424676,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736435823,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736446850,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736458007,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736469060,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736480207,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736491418,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736502620,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736513638,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736524682,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736535712,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736546742,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736557834,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736568858,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736579932,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736590966,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736601993,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736613183,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736624357,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736635387,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736646414,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736657493,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736668624,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736679743,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736685286,"value":0,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736685286,"value":8,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736685286,"value":9,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736685295,"value":5,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736685295,"value":4,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736685295,"value":88,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736685295,"value":130,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736685296,"value":123,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.transactionsNotDeciphered","timestamp":1420736685296,"value":0,"tags":{"sltId":"5036","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736690786,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736701910,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736712968,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736723999,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736735075,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736746106,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736757266,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736768455,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736779473,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736790606,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736801633,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736812713,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736823740,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736834856,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736845958,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736857103,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736868216,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736879292,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736890320,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736901503,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736912608,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736923761,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736934850,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736946033,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736957061,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736968223,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736979256,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736985284,"value":0,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736985285,"value":16,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736985285,"value":9,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tags_read","timestamp":1420736985285,"value":11,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736985285,"value":9,"tags":{"sltId":"5036","readerId":"1","antenna":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736985285,"value":162,"tags":{"sltId":"5036","readerId":"1","antenna":"2","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736985285,"value":166,"tags":{"sltId":"5036","readerId":"1","antenna":"3","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.tag_transactions","timestamp":1420736985285,"value":157,"tags":{"sltId":"5036","readerId":"1","antenna":"4","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.transactionsNotDeciphered","timestamp":1420736985286,"value":0,"tags":{"sltId":"5036","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420736990353,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737001532,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737012658,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737023691,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737034823,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737045906,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737056942,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}},{"metric":"slt.reader.temperature","timestamp":1420737068032,"value":56,"tags":{"sltId":"5036","readerId":"1","host":"xxxxxxxxxxxxxxxxx"}}]HTTP/1.1 400 Bad Request
Content-Length: 1080
Content-Type: text/html; charset=UTF-8
Date: Thu, 08 Jan 2015 17:18:43 GMT
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head><meta http-equiv=content-type content="text/html;charset=utf-8"><title>Bad Request</title>
<style><!--
body{font-family:arial,sans-serif;margin-left:2em}A.l:link{color:#6f6f6f}A.u:link{color:green}.subg{background-color:#e2f4f7}.fwf{font-family:monospace;white-space:pre-wrap}//--></style></head>
<body text=#000000 bgcolor=#ffffff><table border=0 cellpadding=2 cellspacing=0 width=100%><tr><td rowspan=3 width=1% nowrap><b><font color=#c71a32 size=10>T</font><font color=#00a189 size=10>S</font><font color=#1a65b7 size=10>D</font> </b><td> </td></tr><tr><td class=subg><font color=#507e9b><b>Looks like it's your fault this time</b></td></tr><tr><td> </td></tr></table><blockquote><h1>Bad Request</h1>Sorry but your request was rejected as being invalid.<br/><br/>The reason provided was:<blockquote>Chunked request not supported.</blockquote></blockquote><table width=100% cellpadding=0 cellspacing=0><tr><td class=subg><img alt="" width=1 height=6></td></tr></table></body></html>
I was thinking Qt was changing to use a chunked request when the request got larger, but it's not the case. So the current question is: what is a chunked request in openTSDB?
I've finally found the reason it thinks my request is chunked: OpenTSDB internally uses Netty for networking, and if Netty reads a block that doesn't contain the complete request, then it's flagged as chunked, even if there's no Transfer-Encoding header in the request. There are two possible solutions for this: adding "tsd.http.request.enable_chunked = true" to the config file, so that it doesn't reject chunked requests and adding "tsd.http.request.max_chunk = " to the config file, and setting it to a sensible size for your requests If you can change the client, force it to send smaller requests Please note that enabling chunked requests for public facing servers may not be a good idea, as a malicious client can send unbounded requests, overloading the server. Credits goes to irc users jaylr and manolama, who provided me the info on irc channel #opentsdb#freenode.net
Http range header requests entire file
I am working with the c# webserver from codeplex version 1.1. I have implemented the Accept-Range headers and it does work. However when I use wireshark (Version 1.4.1 (SVN Rev 34476 from /trunk-1.4)) to catch the traffic, I see the following: GET /movies/i_am_legend%20dvd/main.m4v HTTP/1.1 Host: 10.100.1.199:8081 Accept: */* Range: bytes=0-1 Accept-Encoding: identity Connection: keep-alive User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl) X-Playback-Session-Id: 9CED81CC-BFAE-4CF6-A477-0EA62B2C652F HTTP/1.1 206 PartialContent Content-Range: bytes 0-1/652965648 Accept-Ranges: bytes ETag: "0daA8D4/wgt4MFvxdNIPLw==" Date: Wed, 13 Jun 2012 09:10:18 GMT Content-Length: 2 Content-Type: video/x-m4v Server: Tiny WebServer Connection: keep-alive .. << 2 bytes data GET /movies/i_am_legend%20dvd/main.m4v HTTP/1.1 Host: 10.100.1.199:8081 Accept: */* Range: bytes=0-652965647 Accept-Encoding: identity Connection: keep-alive User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl) X-Playback-Session-Id: 9CED81CC-BFAE-4CF6-A477-0EA62B2C652F HTTP/1.1 206 PartialContent Content-Range: bytes 0-652965647/652965648 Accept-Ranges: bytes ETag: "0daA8D4/wgt4MFvxdNIPLw==" Date: Wed, 13 Jun 2012 09:10:18 GMT Content-Length: 652965648 Content-Type: video/x-m4v Server: Tiny WebServer Connection: keep-alive The webserver will try to send the entire file ( >600MB), wireshark shows that the entire conversation is 159774 bytes. If I do the same thing with IIS I get similar headers GET /ipod/main.m4v HTTP/1.1 Host: 10.100.1.199 User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl) Accept: */* Range: bytes=0-1 Accept-Encoding: identity X-Playback-Session-Id: C5BBF91D-78AB-42BA-ACE0-D74AB9D845CE Connection: keep-alive HTTP/1.1 206 Partial Content Content-Type: video/x-m4v Last-Modified: Mon, 11 Jun 2012 10:33:41 GMT Accept-Ranges: bytes ETag: "7243cabbd47cd1:0" Server: Microsoft-IIS/7.5 X-Powered-By: ASP.NET Date: Wed, 13 Jun 2012 09:21:03 GMT Content-Length: 2 Content-Range: bytes 0-1/652965648 .. << 2 bytes of data GET /ipod/main.m4v HTTP/1.1 Host: 10.100.1.199 User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl) Accept: */* Range: bytes=0-652965647 Accept-Encoding: identity X-Playback-Session-Id: C5BBF91D-78AB-42BA-ACE0-D74AB9D845CE Connection: keep-alive HTTP/1.1 206 Partial Content Content-Type: video/x-m4v Last-Modified: Mon, 11 Jun 2012 10:33:41 GMT Accept-Ranges: bytes ETag: "7243cabbd47cd1:0" Server: Microsoft-IIS/7.5 X-Powered-By: ASP.NET Date: Wed, 13 Jun 2012 09:21:03 GMT Content-Length: 652965648 Content-Range: bytes 0-652965647/652965648 Wireshark shows that the entire conversation is 175615 bytes. I have searched for more information on the Accept-Range headers, and so far I can only find that the server must send the requested range. But I can't believe that it was meant to use a range request for requesting a huge file in one time. My webserver tries to send the entire file because it has been requested as such, but I see new range requests coming in with more huge ranges like this (only the Range header copied from the request header. The (#time ... ) is the time of wireshark Range: bytes=2162688-652965647 (# time == 1.646204) Range: bytes=4980736-652965647 (# time == 2.754322) Range: bytes=6356992-652965647 (# time == 2.922479) After reading this I have tried to send a shorter range whenever I get the range request for the entire file. But then it does not work at all. I would like to know: Is the range request for the entire file is some kind of bug in iOS (seen it with 4.3.3 as well) I would have expected Range: bytes=0-1 and after the replay something like Range: bytes=0-65535/652965648 Can I somehow gracefully deny this large request and tell the requested that I can deliver a maximum size at once? (I did not find this in the RFC) Is IIS simply aborting this request after certain amount of bytes? EDIT: For number 3: Not IIS but the browser seems to simply aborting (and closing) the connection. After that making a new request. I can't imagine that the Range Request was meant to request the entire file or HUGE parts of the file. EDIT: In iOS7 it seems to have changed. The first range request is still the same (bytes 0-1). After that, I see 2 or 3 range requests as mentioned above, where the last request keeps on transferring bytes for a longer period. However still multiple requests are done.
Is the range request for the entire file is some kind of bug in iOS (seen it with 4.3.3 as well) I would have expected Range: bytes=0-1 and after the replay something like Range: bytes=0-65535/652965648 I don't know whether it is a bug. However, I can think of reasons for a media player to request the entire file in one request. In this way the media player gets a data stream from which it can read all data from start to finish. As soon as the media player have read enough data from the stream it can start playing the media file. It then chooses how much more data it shall buffer in the background while the media is playing. There could be several different approaches to this: Eagerly buffer the entire media file. This is a good strategy when bandwidth is cheap (user is not paying or paying a flat rate for data transfer). It is assumed that the user will want to see/listen to the entire media file. Lazily buffer just enough to avoid lagging. This is a good strategy when bandwidth is expensive (user is paying by the byte). In an ideal setup, the media player wouldn't have to buffer anything at all and instead decode data from the stream while playing the media in real time. However, that would require that the underlying network channel would be super stable and transfer data at the required pace at all times. This is not the case, and therefore the media player will choose to buffer a couple of seconds or minutes ahead. It is important to note that whatever strategy is chosen it could still make sense for the media player to request the entire resource in a single request. However, range requests are vital to the media player when: The connection is aborted (for any reason). The user jumps ahead in the media. (for example, wants to see what's 10 minutes into a movie) The media player can then close the data stream it originally opened and send a range request for the desired position. Can I somehow gracefully deny this large request and tell the requested that I can deliver a maximum size at once? No you cannot. Range requests are initiated by the client/browser and a server that have stated that it support range requests (via Accept-Ranges header) must obey the client and respond with whatever range it requests. What you can do however is to send data with a Transfer-Encoding: chunked header. This will enable your server to control how large chunks of data it shall transmit. However, it is still done over a single HTTP connection.
Why do my HTTP/1.0 GET request returns OK but no body content?
I'm making a simple http page-requester in C. It uses sockets to send HTTP/1.0 GET requests to hosts, and parses the answer to effectively download the html file. However, when i send a request like this: GET http://stackoverflow.com/questions HTTP/1.0 User-Agent: myRequester/1.0 It returns this HTTP/1.1 200 OK Cache-Control: private Content-Type: text/html; charset=utf-8 Date: Mon, 19 Dec 2011 15:28:08 GMT Content-Length: 54362 Connection: close But no body content. Yes, I've put CRLF on the end of every line and a blank line at the end. I use only one socket through one connection. And i also have to stick to HTTP/1.0.
The most likely explanation is that the server is actually sending a body but you are not reading all of it. Most networking systems do not necessarily return all of the response in one function call, because they see it useful that what data is available immediately is returned immediately, even if more is expected. The Unix system call recv returns zero when the connection has ended. You should keep calling it until you get zero or an error.
Web service response encoding issue
I am developing a Web Services based on ASP.Net asmx web service. The server end will response byte[] to client encoded in UTF-8, and client to convert the byte[] to string. My confusion is, the England pound character at server side (I dump just before the Http response is wrote, and the character at server side is correct to be England pound) will be received as ?? from client side. Any ideas what is wrong? I suspect it is encoding issue, but I have no idea how to debug further and any settings (settings from client web service proxy?) which will impact? Here is the header part which I got from Fiddler. HTTP/1.1 200 OK Date: Fri, 20 Feb 2009 16:51:30 GMT Server: Microsoft-IIS/6.0 cache-control: no-cache pragma: no-cache X-Powered-By: ASP.NET X-AspNet-Version: 2.0.50727 Cache-Control: private Content-Type: text/xml Content-Length: 22752 xml version="1.0" encoding="utf-8"
The first thing to do is to sniff what's actually being sent, in terms of the headers, the XML declaration and the bytes forming the text itself.
Fiddler is good as an HTTP proxy, or you could use WireShark to sniff at the network level.
Once you've got those three bits of information (the Content-Type header, the XML declaration and the bytes making up the pound sign) if you update your answer we'll see what we can do. It does sound odd, as usually ASP.NET just gets all of this right.
What does your client side code look like? Is that just the normal .NET web service client code as well?
EDIT: Try to find a binary (hex dump) display in Fiddler so you can find the bytes.
However, I strongly suspect that the problem is merely with dumping the result to the console. Here's a bit of code to use to dump the unicode code points:
static void DumpString (string value)
{
foreach (char c in value)
{
Console.Write ("{0:x4} ", (int)c);
}
Console.WriteLine();
}
I suspect you'll see an 00A3 in the output, which is the Unicode for the pound sign. That means the string has actually reached your client fine - but writing it out to the console is failing.