I try to use chunked http request on ODIN-W2 with mbedOS. I have next http request/response:
POST /post HTTP/1.1<CR><LF>
Accept: */*<CR><LF>
Accept-Encoding: gzip<CR><LF>
Accept-Language: uk,en-US;q=0.8,en;q=0.5,ru;q=0.3<CR><LF>
Cache-Control: no-cache<CR><LF>
Connection: Keep-Alive<CR><LF>
Host: httpbin.org<CR><LF>
Transfer-Encoding: chunked<CR><LF>
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like
Gecko<CR><LF>
content-type: application/json<CR><LF>
<CR><LF>
6<CR><LF>
ABCDEF<CR><LF>
0<CR><LF>
<CR><LF>
Status: 411 - Length Required<LF>
<LF>
Headers:<LF>
Date: Tue, 24 Apr 2018 08:01:41 GMT<LF>
Connection: close<LF>
Content-Type: text/html<LF>
Server: meinheld/0.6.1<LF>
Via: 1.1 vegur<LF>
<LF>
Body (92 bytes):<LF>
<html><head><title>Length Required</title></head><body><p>Length Required.
</p></body></html><LF>
I use standard HTTP library from mbedOS.
This is a bug in httpbin, not in the Mbed HTTP library. See https://github.com/requests/httpbin/issues/340. This is also the reason I used reqres.in as the example URL for chunked requests.
Related
I get an Error 416 Range Not Satisfiable response when I make an http request with a byte-range of 0-65536. The byte length of the file requested is only 3356.
Reading the spec on byte-range, it sounds like requesting a range that extends beyond the length of the file is okay, and in this case I would expect the entire file to be fetched. https://httpwg.org/specs/rfc9110.html#byte.ranges
A client can limit the number of bytes requested without knowing the size of the selected representation. If the last-pos value is absent, or if the value is greater than or equal to the current length of the representation data, the byte range is interpreted as the remainder of the representation (i.e., the server replaces the value of last-pos with a value that is one less than the current length of the selected representation).
Am I misreading the spec or is this request truly unsatisfiable? I prefer if my client does not need to know the size of the file before making the request; they could vary widely in size. The request is successful if I limit the byte-range to <= the length of the file.
The file requested is a cloud-optimized-geotiff. It is being served by a vite dev server. And the request is made using OpenLayers and geotiff.js.
Here are the DevTools details of the request and response:
General
---------
Request URL: http://localhost:3000/scenarios/1/1_parcel_fill.tif
Request Method: GET
Status Code: 416 Range Not Satisfiable
Remote Address: [::1]:3000
Referrer Policy: strict-origin-when-cross-origin
Response Headers
--------------------
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Length: 0
Content-Range: bytes */3356
Date: Wed, 21 Sep 2022 21:38:19 GMT
Keep-Alive: timeout=5
Request Headers
-------------------
Accept: */*
Accept-Encoding: identity
Accept-Language: en-US,en;q=0.9
Cache-Control: no-cache
Connection: keep-alive
Host: localhost:3000
Pragma: no-cache
Range: bytes=0-65536
Referer: http://localhost:3000/
sec-ch-ua: "Google Chrome";v="105", "Not)A;Brand";v="8", "Chromium";v="105"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36
I am trying to call a page in PHP with a http_get :
$url = "http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446";
http_get($url, $appelOptions, $appelInfos);
My problem is that it does not work every time.
I installed Wireshark to see what I'm really sending and I found an odd thing. Sometimes, the port is not used for the HTTP request.
When it works, I have :
Hypertext Transfer Protocol
GET http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446 HTTP/1.1\r\n
Request Method: GET
Request URI: http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446
Request Version: HTTP/1.1
User-Agent: PECL::HTTP/1.6.5 (PHP/5.2.4-2ubuntu5.7)\r\n
Host: mysite.fr:9090\r\n
Pragma: no-cache\r\n
Accept: */*\r\n
Proxy-Connection: Keep-Alive\r\n
Keep-Alive: 300\r\n
Connection: keep-alive\r\n
Date: Fri, 15 Jun 2012 16:40:46 +0200\r\n
Accept-Charset: utf-8\r\n
Accept-Encoding: gzip;q=1.0,deflate;q=0.5\r\n
\r\n
And when it's not :
Hypertext Transfer Protocol
GET http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446 HTTP/1.1\r\n
Request Method: GET
Request URI: http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446
Request Version: HTTP/1.1
User-Agent: PECL::HTTP/1.6.5 (PHP/5.2.4-2ubuntu5.7)\r\n
Host: mysite.fr\r\n
Pragma: no-cache\r\n
Accept: */*\r\n
Proxy-Connection: Keep-Alive\r\n
Keep-Alive: 300\r\n
Connection: keep-alive\r\n
Date: Fri, 15 Jun 2012 16:40:34 +0200\r\n
Accept-Charset: utf-8\r\n
Accept-Encoding: gzip;q=1.0,deflate;q=0.5\r\n
\r\n
I tried to call the page with wget and it's always working :
wget http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446
So I'm guessing that my problem id due to Apache config, but I don't know where to look. Could you help me please ?
You will need to set the port in the $appelOptions array.
$appelOptions['port']=9090;
http_get($url, $appelOptions, $appelInfos);
Unfortunately http_get does not seem to respect the :port syntax in the URL
Take a look at this page:
http://www.ptmytrade.com/product.asp?id=61363
It's loading fine (at least here). Now I would like to grab it with wget.
$ wget http://www.ptmytrade.com/product.asp?id=61363 --debug
DEBUG output created by Wget 1.12 on linux-gnu.
--2011-05-21 18:24:51-- http://www.ptmytrade.com/product.asp?id=61363
Resolving www.ptmytrade.com... 205.209.150.134
Caching www.ptmytrade.com => 205.209.150.134
Connecting to www.ptmytrade.com|205.209.150.134|:80... connected.
Created socket 3.
Releasing 0x0890e260 (new refcount 1).
---request begin---
GET /product.asp?id=61363 HTTP/1.0
User-Agent: Wget/1.12 (linux-gnu)
Accept: */*
Host: www.ptmytrade.com
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 500 Internal Server Error
Connection: keep-alive
Date: Sat, 21 May 2011 16:24:56 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Length: 471822
Content-Type: text/html
Set-Cookie: ASPSESSIONIDSCACCAQA=FOCCMJODFHHMOKNKPAIHJCIL; path=/
Cache-control: private
---response end---
500 Internal Server Error
Stored cookie www.ptmytrade.com -1 (ANY) / <session> <insecure> [expiry none] ASPSESSIONIDSCACCAQA FOCCMJODFHHMOKNKPAIHJCIL
Registered socket 3 for persistent reuse.
Disabling further reuse of socket 3.
Closed fd 3
2011-05-21 18:24:57 ERROR 500: Internal Server Error.
OK, so I check the headers when fetching the page using my browser (using Live HTTP Headers add-on):
http://www.ptmytrade.com/product.asp?id=61361
GET /product.asp?id=61361 HTTP/1.1
Host: www.ptmytrade.com
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:2.0) Gecko/20100101 Firefox/4.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Cookie: ASPSESSIONIDSCACBBRA=AMPBLLNDGMFLNPNCPEBPNNLB; ASPSESSIONIDSCACCAQA=FJNBMJODLHHJNDHPFBIEEPEM
HTTP/1.1 500 Internal Server Error
Date: Sat, 21 May 2011 16:20:46 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Length: 471822
Content-Type: text/html
Cache-Control: private
----------------------------------------------------------
http://www.ptmytrade.com/images/index_117.jpg
GET /images/index_117.jpg HTTP/1.1
Host: www.ptmytrade.com
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:2.0) Gecko/20100101 Firefox/4.0
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://www.ptmytrade.com/product.asp?id=61361
Cookie: ASPSESSIONIDSCACBBRA=AMPBLLNDGMFLNPNCPEBPNNLB; ASPSESSIONIDSCACCAQA=FJNBMJODLHHJNDHPFBIEEPEM
HTTP/1.1 404 Not Found
Content-Length: 1635
Content-Type: text/html
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Sat, 21 May 2011 16:20:48 GMT
I'm not sure what's going on here. The page displays just fine, but I'm getting the 500 error code in the header.
The problem was solved by using curl (which was also getting a 500, but fetched the page just fine) instead, but I'm curious what's going here.
Hi everybody I had this huge problem too, I don't know why, but the solution was add this:
wget -U "Opera 11.0" "http://your_link" -O out.csv
I found it on
[Curl and wget return error 500 for helloworld.php on new install but browser is fine
Using this option will fix the issue:
--content-on-error
If this is set to on, wget will not skip the content when the
server responds with a http status code that indicates error.
So the command looks like this:
wget --content-on-error "https://stackoverflow.com"
NOTE: It's important to put the URL inside double-quotes, otherwise, wget will get stuck on Redirecting output to ‘wget-log’..
Or as stated in the comments and by OP, use curl instead.
But I should note that curl cannot download whole webpages (css, js, images etc.) because it cannot parse HTML. Source and Taken from.
It's a bug in the webpage. The HTTP status is indeed seemingly incorrectly set to HTTP 500. Firefox/Firebug also confirms this. Basically, you're facing a HTTP 500 error page with "normal" content.
Report it to the site admin.
Try enclosing it in quotes:
wget "http://www.ptmytrade.com/product.asp?id=61363"
instead of:
wget http://www.ptmytrade.com/product.asp?id=61363
I am using NetStream, NetConnection and Video object to play an mp4 file which is hosted over a web server using http.
The mp4 file URL is for example: http://xx.xx.xx.xx/file.mp4
This is an AIR application and the relevant code is pasted below:
var url:String = <some http url>;
connect_nc = new NetConnection();
connect_nc.connect(null);
stream_ns = new NetStream(connect_nc);
var ns_object:Object = new Object();
ns_object.onPlayStatus = ns_onPlayStatus;
stream_ns.client = ns_object;
videoMP4.attachNetStream(stream_ns);
stream_ns.bufferTime = 1.0 // 1 sec
stream_ns.addEventListener(NetStatusEvent.NET_STATUS, onNetStatusEventHandler);
stream_ns.play(url);
This code works when run on MAC OS X. But it does not work when run on Windows XP. I get the error:
NetStream.Play.StreamNotFound
I also tried playing the URL using VLC player on the same windows XP host. The URL is valid because VLC can play it.
In my particular case, the http URL is hosted by WMP 12 (window media player 12) on Win 7 machine where I am using the media sharing feature of WMP 12.
After further looking into http traffic on wireshark, here is what i found.
After running wireshark on the host running the adobe AIR application, it seems that it is getting a HTTP 406 response from
the server being run by WMP 12.
GET /WMPNSSv4/63903908/1_ezVGREUzQTA4LTdDQzQtNDJFMy1CNDVDLUZEMjA4MDE5OUM4Q30uMC44.mp4 HTTP/1.1
Host: 192.168.0.102:10243
User-Agent: Mozilla/5.0 (Windows; U; en) AppleWebKit/526.9+ (KHTML, like Gecko) AdobeAIR/1.5
Referer: app:/clicker.swf
x-flash-version: 10,0,12,36
Connection: Keep-Alive
Accept: text/xml, application/xml, application/xhtml+xml, text/html;q=0.9, text/plain;q=0.8, text/css, image/png, image/jpeg, image/gif;q=0.8, application/x-shockwave-flash, video/mp4;q=0.9, flv-application/octet-stream;q=0.8, video/x-flv;q=0.7, audio/mp4, application/futuresplash, */*;q=0.5
Response:
HTTP/1.1 406 Not Acceptable
Last-Modified: Mon, 19 Oct 2009 23:21:14 GMT
Server: Microsoft-HTTPAPI/2.0
Accept-Ranges: bytes
TransferMode.DLNA.ORG: Streaming
Date: Tue, 12 Jan 2010 22:52:48 GMT
Connection: close
Content-Length: 0
On MAC:
It receives 200 OK response though, and that is why the video streaming works.
GET /WMPNSSv4/63903908/1_ezVGREUzQTA4LTdDQzQtNDJFMy1CNDVDLUZEMjA4MDE5OUM4Q30uMC44.m p4 HTTP/1.1
Host: 192.168.0.102:10243
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/526.9+ (KHTML, like Gecko) AdobeAIR/1.5.3
Referer: app:/clicker.swf
X-Flash-Version: 10,0,42,34
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: keep-alive
Response:
HTTP/1.1 200 OK
Content-Length: 1524867
Content-Type: video/mp4
Last-Modified: Mon, 19 Oct 2009 23:21:14 GMT
Server: Microsoft-HTTPAPI/2.0
Accept-Ranges: bytes
TransferMode.DLNA.ORG: Streaming
Date: Tue, 12 Jan 2010 22:56:20 GMT
The difference that I can see in the HTTP requests between the Windows XP and MAC version is the Accept: Header. Is the Accept: header value wrong for Windows case because of which WMP 12 rejects
the http request.
If i run the adobe AIR application on Win 7 host, i see the same failure.
Am I using the NetStream object incorrectly or it is a bug in WMP 12 code not being able to parse
the header properly or it is a flex bug where it is generating an incorrect accept: header?
I believe WMP 12 incorrectly handles 'Accept' header in a request. If it contains 'q' (quality) parameter, then WMP ignores this mime-type. And if there are no other suitable mime-types for WMP, it will respond with 406 Not Acceptable error.
I encountered this issues when was trying to display DLNA image in Chrome browser.
I used curl utility to send requests with different headers to figure out what goes wrong.
Request that results in 406 Not Acceptable error:
curl -v -o file.jpg -H "Accept: text/html,*/*,q=0.8" "http://127.0.0.1:10243/WMPNSSv4/3065481158/0_e0I5MzA1MTRELUYwMEEtNEQwRC1CQzg4LTg3NEI5QjQ4MDYyM30uMC5C.jpg"
GET /WMPNSSv4/3065481158/0_e0I5MzA1MTRELUYwMEEtNEQwRC1CQzg4LTg3NEI5QjQ4MDYyM30uMC5C.jpg HTTP/1.1
User-Agent: curl/7.31.0
Host: 127.0.0.1:10243
Accept: text/html,*/*;q=0.8
HTTP/1.1 406 Not Acceptable
Last-Modified: Tue, 21 May 2013 21:01:09 GMT
Server: Microsoft-HTTPAPI/2.0
Accept-Ranges: bytes
TransferMode.DLNA.ORG: Interactive
Date: Fri, 30 Aug 2013 09:10:32 GMT
Connection: close
Content-Length: 0
Successful request:
curl -v -o file.jpg -H "Accept: text/html,*/*" "http://127.0.0.1:10243/WMPNSSv4/3065481158/0_e0I5MzA1MTRELUYwMEEtNEQwRC1CQzg4LTg3NEI5QjQ4MDYyM30uMC5C.jpg"
GET /WMPNSSv4/3065481158/0_e0I5MzA1MTRELUYwMEEtNEQwRC1CQzg4LTg3NEI5QjQ4MDYyM30uMC5C.jpg HTTP/1.1
User-Agent: curl/7.31.0
Host: 127.0.0.1:10243
Accept: text/html,*/*
HTTP/1.1 200 OK
Content-Length: 2394679
Content-Type: image/jpeg
Last-Modified: Tue, 21 May 2013 21:01:09 GMT
Server: Microsoft-HTTPAPI/2.0
Accept-Ranges: bytes
TransferMode.DLNA.ORG: Interactive
Date: Fri, 30 Aug 2013 09:10:40 GMT
How many bytes of data does a typical HTTP get request consume.
For instance if I request a page from the server through a browser how many bytes of data would be sent?
Pretty typical request, 430 bytes:
GET /ga.js HTTP/1.1\r\n
Host: www.google-analytics.com\r\n
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)\r\n
Accept: */*\r\n
Accept-Language: en-us,en;q=0.5\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
Keep-Alive: 300\r\n
Connection: keep-alive\r\n
Referer: http://stackoverflow.com/\r\n
If-Modified-Since: Mon, 31 Aug 2009 17:13:58 GMT\r\n
\r\n
\r\n
Request with a long query string and a small cookie 657 bytes)
GET /pixel;r=978178957;fpan=0;fpa=1241112640-44259546-69321280;ns=0;url=http%3A%2F%2Fstackoverflow.com%2F;ref=;ce=1;je=1;sr=1920x1200x32;dg=E5912-W-MO-5;dst=1;et=1252061014745;tzo=-120;a=p-c1rF4kxgLUzNc HTTP/1.1\r\n
Host: pixel.quantserve.com\r\n
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.2) Gecko/20090729 Firefox/3.5.2 (.NET CLR 3.5.30729)\r\n
Accept: image/png,image/*;q=0.8,*/*;q=0.5\r\n
Accept-Language: en-us,en;q=0.5\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\n
Keep-Alive: 300\r\n
Connection: keep-alive\r\n
Referer: http://stackoverflow.com/\r\n
Cookie: uid=1274108650-45267447-66848880; mc=1137458542-57565784-88898864\r\n
\r\n
\r\n
Use Fiddler to intercept the request and see for yourself.
It varies, especially when it comes to GET queries or POST requests, but I'd estimate it about 0.5—1k.
Requesting a page from the browser, though, may also result in requesting pictures, stylesheets and other referenced content.
Edit: originally I put in the estimation for request+reply.
I would suggest you use a full packet sniffer like wireshark. You would love it :)
Get it here:
http://www.wireshark.org/