I am working with the c# webserver from codeplex version 1.1. I have implemented the Accept-Range headers and it does work. However when I use wireshark (Version 1.4.1 (SVN Rev 34476 from /trunk-1.4)) to catch the traffic, I see the following:
GET /movies/i_am_legend%20dvd/main.m4v HTTP/1.1
Host: 10.100.1.199:8081
Accept: */*
Range: bytes=0-1
Accept-Encoding: identity
Connection: keep-alive
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl)
X-Playback-Session-Id: 9CED81CC-BFAE-4CF6-A477-0EA62B2C652F
HTTP/1.1 206 PartialContent
Content-Range: bytes 0-1/652965648
Accept-Ranges: bytes
ETag: "0daA8D4/wgt4MFvxdNIPLw=="
Date: Wed, 13 Jun 2012 09:10:18 GMT
Content-Length: 2
Content-Type: video/x-m4v
Server: Tiny WebServer
Connection: keep-alive
.. << 2 bytes data
GET /movies/i_am_legend%20dvd/main.m4v HTTP/1.1
Host: 10.100.1.199:8081
Accept: */*
Range: bytes=0-652965647
Accept-Encoding: identity
Connection: keep-alive
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl)
X-Playback-Session-Id: 9CED81CC-BFAE-4CF6-A477-0EA62B2C652F
HTTP/1.1 206 PartialContent
Content-Range: bytes 0-652965647/652965648
Accept-Ranges: bytes
ETag: "0daA8D4/wgt4MFvxdNIPLw=="
Date: Wed, 13 Jun 2012 09:10:18 GMT
Content-Length: 652965648
Content-Type: video/x-m4v
Server: Tiny WebServer
Connection: keep-alive
The webserver will try to send the entire file ( >600MB), wireshark shows that the entire conversation is 159774 bytes. If I do the same thing with IIS I get similar headers
GET /ipod/main.m4v HTTP/1.1
Host: 10.100.1.199
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl)
Accept: */*
Range: bytes=0-1
Accept-Encoding: identity
X-Playback-Session-Id: C5BBF91D-78AB-42BA-ACE0-D74AB9D845CE
Connection: keep-alive
HTTP/1.1 206 Partial Content
Content-Type: video/x-m4v
Last-Modified: Mon, 11 Jun 2012 10:33:41 GMT
Accept-Ranges: bytes
ETag: "7243cabbd47cd1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Wed, 13 Jun 2012 09:21:03 GMT
Content-Length: 2
Content-Range: bytes 0-1/652965648
.. << 2 bytes of data
GET /ipod/main.m4v HTTP/1.1
Host: 10.100.1.199
User-Agent: AppleCoreMedia/1.0.0.9B206 (iPad; U; CPU OS 5_1_1 like Mac OS X; nl_nl)
Accept: */*
Range: bytes=0-652965647
Accept-Encoding: identity
X-Playback-Session-Id: C5BBF91D-78AB-42BA-ACE0-D74AB9D845CE
Connection: keep-alive
HTTP/1.1 206 Partial Content
Content-Type: video/x-m4v
Last-Modified: Mon, 11 Jun 2012 10:33:41 GMT
Accept-Ranges: bytes
ETag: "7243cabbd47cd1:0"
Server: Microsoft-IIS/7.5
X-Powered-By: ASP.NET
Date: Wed, 13 Jun 2012 09:21:03 GMT
Content-Length: 652965648
Content-Range: bytes 0-652965647/652965648
Wireshark shows that the entire conversation is 175615 bytes.
I have searched for more information on the Accept-Range headers, and so far I can only find that the server must send the requested range. But I can't believe that it was meant to use a range request for requesting a huge file in one time.
My webserver tries to send the entire file because it has been requested as such, but I see new range requests coming in with more huge ranges like this (only the Range header copied from the request header. The (#time ... ) is the time of wireshark
Range: bytes=2162688-652965647 (# time == 1.646204)
Range: bytes=4980736-652965647 (# time == 2.754322)
Range: bytes=6356992-652965647 (# time == 2.922479)
After reading this I have tried to send a shorter range whenever I get the range request for the entire file. But then it does not work at all.
I would like to know:
Is the range request for the entire file is some kind of bug in iOS (seen it with 4.3.3 as well) I would have expected Range: bytes=0-1 and after the replay something like Range: bytes=0-65535/652965648
Can I somehow gracefully deny this large request and tell the requested that I can deliver a maximum size at once? (I did not find this in the RFC)
Is IIS simply aborting this request after certain amount of bytes?
EDIT: For number 3: Not IIS but the browser seems to simply aborting (and closing) the connection. After that making a new request. I can't imagine that the Range Request was meant to request the entire file or HUGE parts of the file.
EDIT: In iOS7 it seems to have changed. The first range request is still the same (bytes 0-1). After that, I see 2 or 3 range requests as mentioned above, where the last request keeps on transferring bytes for a longer period. However still multiple requests are done.
Is the range request for the entire file is some kind of bug in iOS (seen it with 4.3.3 as well) I would have expected Range: bytes=0-1 and after the replay something like Range: bytes=0-65535/652965648
I don't know whether it is a bug. However, I can think of reasons for a media player to request the entire file in one request. In this way the media player gets a data stream from which it can read all data from start to finish.
As soon as the media player have read enough data from the stream it can start playing the media file. It then chooses how much more data it shall buffer in the background while the media is playing. There could be several different approaches to this:
Eagerly buffer the entire media file. This is a good strategy when bandwidth is cheap (user is not paying or paying a flat rate for data transfer). It is assumed that the user will want to see/listen to the entire media file.
Lazily buffer just enough to avoid lagging. This is a good strategy when bandwidth is expensive (user is paying by the byte).
In an ideal setup, the media player wouldn't have to buffer anything at all and instead decode data from the stream while playing the media in real time. However, that would require that the underlying network channel would be super stable and transfer data at the required pace at all times.
This is not the case, and therefore the media player will choose to buffer a couple of seconds or minutes ahead.
It is important to note that whatever strategy is chosen it could still make sense for the media player to request the entire resource in a single request.
However, range requests are vital to the media player when:
The connection is aborted (for any reason).
The user jumps ahead in the media. (for example, wants to see what's 10 minutes into a movie)
The media player can then close the data stream it originally opened and send a range request for the desired position.
Can I somehow gracefully deny this large request and tell the requested that I can deliver a maximum size at once?
No you cannot. Range requests are initiated by the client/browser and a server that have stated that it support range requests (via Accept-Ranges header) must obey the client and respond with whatever range it requests.
What you can do however is to send data with a Transfer-Encoding: chunked header. This will enable your server to control how large chunks of data it shall transmit. However, it is still done over a single HTTP connection.
Related
I try to use c++ develop a HTTP server on Windows,and when i reponse a HTTP by use WSASend to send out
char response[] =
"HTTP/1.1 200 OK\r\n\
Date: Mon, 27 Jul 2009 12:28:53 GMT\n\r\
Server: Apache/2.2.14 (Win32)\n\r\
Last-Modified: Wed, 22 Jul 2009 19:15:56 GMT\n\r\
Content-Length: 88\n\r\
Content-Type: text/html\n\r\
Connection: Closed\n\r\n\r\
<html><body><h1>Hello, World!</h1></body></html>"
Althrougn the browser did show Hello, World! when i type in 127.0.0.1,but the browser just keep show loading sigh as if the pages not yet load complete.And the browser's console never show the response message.Why?
Is there some format issue with my response message?
Content-Length: 88\n\r\
....
Connection: Closed\n\r\n\r\
There are several problems with your code. All over your code you use \n\r instead of \r\n. Therefore the response is invalid HTTP. And the Content-length header must reflect the actual length of the body: <html><body><h1>Hello, World!</h1></body></html> has 48 bytes and not 88 bytes as your code claims. Apart from that it must be Connection: Close instead of Connection: Closed.
Note that HTTP is way more complex than you think. If you really need to implement it yourself instead of using established libraries please study the actual standard (that's what standards are for!) instead of fiddling around until it seems to work. Otherwise it might work only within your specific environment and with a specific browser and you'll get strange problems later.
Background
I am building a custom HTTP parser in C++/CX using sockets. As such, I have full control over the entire HTTP request and response.
Request
GET /posts/html-android-app?referrer=rss HTTP/1.1
Host: mixturatech.com
Connection: close
Response
HTTP/1.1 200 OK
Date: Thu, 30 Apr 2015 04:44:59 GMT
Server: Apache
X-Powered-By: PHP/5.2.17
Access-Control-Allow-Origin: *
Cache-Control: public
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
6a2f
<!DOCTYPE html>
[trimmed document content]
</html>
0
Additional Data
If I navigate to the webpage with Chrome, WireShark captures the same data that I am seeing (with extraneous characters), yet Chrome manages to trim that content out. (I am looking at Chrome's data in the Network tab in Developer Tools.)
I do not see this problem on every site I retrieve, but the problem, if it exists, seems to be sitewide.
Questions
What is up with the 6a2f and 0 preceding and following the document?
Is this an encoding issue?
Is there some way that I can positively identify, without hardcoding boundaries for the document, such as it must start with < and end with >, where the actual content lies?
Will those characters, if they exist in a page, always be limited to length 4 and 1 respectively?
This is "chunked transfer encoding". Read http://greenbytes.de/tech/webdav/rfc7230.html#chunked.encoding.
I have an Azure Web Site configured to use multiple (2) instances:
I have a service bus that should pass messages (ie Cache Evict) between the instances. I need to test this mechanism.
In a conventional (on premise) system I would point a browser to instance 1 (ie http://myserver1.example.com), perform an action, then point my browser to the other instance (http://myserver2.example.com) to test.
However, in Azure I can't see a way to hit a specific instance. Is it possible? Or is there an alternative way to to run through this test scenario (act on instance 1, ensure instance 2 behaves appropriately)?
Unfortunately, there isn't an official way of doing this. However, you can achieve that by setting a cookie called ARRAffinity on your request.
Try hitting your site from any client (Chrome, Firefox, curl, httpie, etc) and inspect the response headers that you are getting back.
For example in curl you would do
curl -I <siteName>.azurewebsites.net
you would get this
HTTP/1.1 200 OK
Content-Length: 2
Content-Type: text/html
Last-Modified: Wed, 17 Sep 2014 16:57:26 GMT
Accept-Ranges: bytes
ETag: "2ba0757598d2cf1:0"
Server: Microsoft-IIS/8.0
X-Powered-By: ASP.NET
Set-Cookie: ARRAffinity=<very long hash>; Path=/;Domain=<siteName>.azurewebsites.net
Date: Fri, 28 Nov 2014 03:13:07 GMT
what you are interested in is the ARRAFinity if you send couple of request you would notice that hash will keep changing between 2 values that represent your 2 instances.
Set that in your Cookie header on your request will guarantee it going to one of the instances and not the other.
curl --cookie ARRAfinity=<one of the hashes you got> <siteName>.azurewebsites.net
I'm making a simple http page-requester in C. It uses sockets to send HTTP/1.0 GET requests to hosts, and parses the answer to effectively download the html file.
However, when i send a request like this:
GET http://stackoverflow.com/questions HTTP/1.0
User-Agent: myRequester/1.0
It returns this
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/html; charset=utf-8
Date: Mon, 19 Dec 2011 15:28:08 GMT
Content-Length: 54362
Connection: close
But no body content.
Yes, I've put CRLF on the end of every line and a blank line at the end.
I use only one socket through one connection. And i also have to stick to HTTP/1.0.
The most likely explanation is that the server is actually sending a body but you are not reading all of it. Most networking systems do not necessarily return all of the response in one function call, because they see it useful that what data is available immediately is returned immediately, even if more is expected.
The Unix system call recv returns zero when the connection has ended. You should keep calling it until you get zero or an error.
basically, i was wiresharking packets on my PS3 while viewing Motorstorm Leaderboards. The leaderboards are sent to my ps3 in XML format but only after i have been authorised. So can someone please tell me what is happening between these three packets and how i could replicate it in a browser?
Packet 1 From my PS3 to Sony Servers
POST /ranking_view/func/get_player_rank HTTP/1.1
Host: ranking-view-a01.u0.np.community.playstation.net
Connection: Keep-Alive
Content-Length: 213
Authorization: Digest username="c7y-ranking01", realm="c7y-ranking", nonce="2SpsV4WABAA=47a2b36030cd94de1190f6b9f05db1bd5584bc2a", uri="/ranking_view/func/get_player_rank", qop="auth", nc="00000001", cnonce="d4eb1eb60ab4efaea1476869d83a6e0b", response="96b55c6e79f84dd41b46eb66bed1c167"
Accept-Encoding: identity
User-Agent: PS3Community-agent/1.0.0 libhttp/1.0.0
<?xml version="1.0" encoding="utf-8"?><ranking platform="ps3" sv="3.15"><titleid>NPWR00012_00</titleid><board>7</board><jid>Panzerborn#a5.gb.np.playstation.net</jid><option message="false" info="false"/></ranking>
Packet 2 Sony Server Response to my PS3
Date: Fri, 26 Feb 2010 19:06:12 GMT
WWW-Authenticate: Digest realm="c7y-ranking", nonce="a3PFl4WABAA=6d375259676ec79641448a8032a795b8e12ccae4", algorithm=MD5, stale=true, qop="auth"
Content-Length: 401
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Authorization Required</title>
</head><body>
<h1>Authorization Required</h1>
<p>This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>
Packet 3 PS3 response to Sony Servers last packet
POST /ranking_view/func/get_player_rank HTTP/1.1
Host: ranking-view-a01.u0.np.community.playstation.net
Connection: Keep-Alive
Authorization: Digest username="c7y-ranking01", realm="c7y-ranking", nonce="a3PFl4WABAA=6d375259676ec79641448a8032a795b8e12ccae4", uri="/ranking_view/func/get_player_rank", qop="auth", nc="00000001", cnonce="58869490a891002d8c56573496274a3a", response="ca3d6f252d4e398b8f751c201a3f8f08"
Accept-Encoding: identity
User-Agent: PS3Community-agent/1.0.0 libhttp/1.0.0
<?xml version="1.0" encoding="utf-8"?><ranking platform="ps3" sv="3.15"><titleid>NPWR00012_00</titleid><board>7</board><jid>Panzerborn#a5.gb.np.playstation.net</jid><option message="false" info="false"/></ranking>
I tried to replicate this in Firefox and tamper headers as well as in PHP cURL but im getting nowhere. I assume it is to do with the nonce, cnonce and responce variables that keep changing >< please help =)
Nonce, cnonce and so on are related to HTTP Digest Authentication, which is an authentication mechanism that enables authentication without sending a password in plain text. So if you want to cheat in your PS3 game, you'll first have to hack that password out of the MD5 hash, I guess.
And it's not called HTTP packets, on layer 7 you would usually say request/response or similar.
The nonce an nonce and cnonce look like hash codes.
One possible defense mechanism against cheaters could be this:
def ps3client_send_score():
score = "bazillion points"
nonce = md5(score + "something you don't know about")
send_to_server(score, nonce)
On the server side:
def get_client_score(score, nonce):
if md5(score+"something you don't know about")==nonce:
accept_score(score)
else:
reject_score_and_ban_the_fool_if_he_continues_this()
So unless you want to spend weeks trying to find the salt deep in your game, forget it.