I am trying to call a page in PHP with a http_get :
$url = "http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446";
http_get($url, $appelOptions, $appelInfos);
My problem is that it does not work every time.
I installed Wireshark to see what I'm really sending and I found an odd thing. Sometimes, the port is not used for the HTTP request.
When it works, I have :
Hypertext Transfer Protocol
GET http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446 HTTP/1.1\r\n
Request Method: GET
Request URI: http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446
Request Version: HTTP/1.1
User-Agent: PECL::HTTP/1.6.5 (PHP/5.2.4-2ubuntu5.7)\r\n
Host: mysite.fr:9090\r\n
Pragma: no-cache\r\n
Accept: */*\r\n
Proxy-Connection: Keep-Alive\r\n
Keep-Alive: 300\r\n
Connection: keep-alive\r\n
Date: Fri, 15 Jun 2012 16:40:46 +0200\r\n
Accept-Charset: utf-8\r\n
Accept-Encoding: gzip;q=1.0,deflate;q=0.5\r\n
\r\n
And when it's not :
Hypertext Transfer Protocol
GET http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446 HTTP/1.1\r\n
Request Method: GET
Request URI: http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446
Request Version: HTTP/1.1
User-Agent: PECL::HTTP/1.6.5 (PHP/5.2.4-2ubuntu5.7)\r\n
Host: mysite.fr\r\n
Pragma: no-cache\r\n
Accept: */*\r\n
Proxy-Connection: Keep-Alive\r\n
Keep-Alive: 300\r\n
Connection: keep-alive\r\n
Date: Fri, 15 Jun 2012 16:40:34 +0200\r\n
Accept-Charset: utf-8\r\n
Accept-Encoding: gzip;q=1.0,deflate;q=0.5\r\n
\r\n
I tried to call the page with wget and it's always working :
wget http://mysite.fr:9090/neolane-webservice/campagnesclient/Coclico=1135446
So I'm guessing that my problem id due to Apache config, but I don't know where to look. Could you help me please ?
You will need to set the port in the $appelOptions array.
$appelOptions['port']=9090;
http_get($url, $appelOptions, $appelInfos);
Unfortunately http_get does not seem to respect the :port syntax in the URL
Related
I'm trying to get a token from Spotify using the Web API:
My raw request:
POST https://accounts.spotify.com/api/token HTTP/1.1
Authorization: Basic [authsecret]
Content-Type: application/x-www-form-urlencoded
Host: accounts.spotify.com
Content-Length: 490
Expect: 100-continue
Connection: Keep-Alive
grant_type=authorization_code&code=[code]&redirect_url=http%3A%2F%2Flocalhost%3A53575%2FSpotifyLogin.aspx
Where [authsecret] is replaced by the required string (this was broken and gave me a different error, but now I fixed it) and [code] is replaced by the code returned a step earlier by Spotify.
I get the following response:
HTTP/1.1 400 Bad Request
Server: nginx
Date: Thu, 22 Jun 2017 13:50:33 GMT
Content-Type: application/json
Content-Length: 68
Connection: keep-alive
Keep-Alive: timeout=600
{"error":"invalid_grant","error_description":"Invalid redirect URI"}
I must be doing something wrong, but I can't figure it out. Can anyone help me?
I've figured it out. Issue is the parameter name redirect_url. It should be redirect_uri. Small mistake but breaks the request as it should.
hen a server replies to a HTTP GET request why doesn't it specify in say some new header, the GET request for which this response was generated?
GET /counter.gif HTTP/1.1
Host: www.subbu.org
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.5) Gecko/20041107 Firefox/1.0
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
HTTP/1.1 200 OK
Date: Mon, 19 Dec 2011 00:12:28 GMT
Server: Apache
Last-Modified: Thu, 08 Dec 2011 15:29:54 GMT
Accept-Ranges: bytes
Content-Length: 24523
Content-Type: image/png
**Get-request: /counter.gif**
This will help in case the requests are pipelined. The server can send the replies in the order they are generated and the browser can interpret what the reply is for. I know HTTP is stateless but this seems like a simple optimization. Why do you think it might be a problem to implement this?
Like that:
https://datatracker.ietf.org/doc/html/draft-nottingham-http-pipeline-00#section-5
?
If your using apache you may be referring to this functionnality: Trace
When trace is set to extended, it reflect the request body.
Regards
Take a look at this page:
http://www.ptmytrade.com/product.asp?id=61363
It's loading fine (at least here). Now I would like to grab it with wget.
$ wget http://www.ptmytrade.com/product.asp?id=61363 --debug
DEBUG output created by Wget 1.12 on linux-gnu.
--2011-05-21 18:24:51-- http://www.ptmytrade.com/product.asp?id=61363
Resolving www.ptmytrade.com... 205.209.150.134
Caching www.ptmytrade.com => 205.209.150.134
Connecting to www.ptmytrade.com|205.209.150.134|:80... connected.
Created socket 3.
Releasing 0x0890e260 (new refcount 1).
---request begin---
GET /product.asp?id=61363 HTTP/1.0
User-Agent: Wget/1.12 (linux-gnu)
Accept: */*
Host: www.ptmytrade.com
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 500 Internal Server Error
Connection: keep-alive
Date: Sat, 21 May 2011 16:24:56 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Length: 471822
Content-Type: text/html
Set-Cookie: ASPSESSIONIDSCACCAQA=FOCCMJODFHHMOKNKPAIHJCIL; path=/
Cache-control: private
---response end---
500 Internal Server Error
Stored cookie www.ptmytrade.com -1 (ANY) / <session> <insecure> [expiry none] ASPSESSIONIDSCACCAQA FOCCMJODFHHMOKNKPAIHJCIL
Registered socket 3 for persistent reuse.
Disabling further reuse of socket 3.
Closed fd 3
2011-05-21 18:24:57 ERROR 500: Internal Server Error.
OK, so I check the headers when fetching the page using my browser (using Live HTTP Headers add-on):
http://www.ptmytrade.com/product.asp?id=61361
GET /product.asp?id=61361 HTTP/1.1
Host: www.ptmytrade.com
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:2.0) Gecko/20100101 Firefox/4.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Cookie: ASPSESSIONIDSCACBBRA=AMPBLLNDGMFLNPNCPEBPNNLB; ASPSESSIONIDSCACCAQA=FJNBMJODLHHJNDHPFBIEEPEM
HTTP/1.1 500 Internal Server Error
Date: Sat, 21 May 2011 16:20:46 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Length: 471822
Content-Type: text/html
Cache-Control: private
----------------------------------------------------------
http://www.ptmytrade.com/images/index_117.jpg
GET /images/index_117.jpg HTTP/1.1
Host: www.ptmytrade.com
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:2.0) Gecko/20100101 Firefox/4.0
Accept: image/png,image/*;q=0.8,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive
Referer: http://www.ptmytrade.com/product.asp?id=61361
Cookie: ASPSESSIONIDSCACBBRA=AMPBLLNDGMFLNPNCPEBPNNLB; ASPSESSIONIDSCACCAQA=FJNBMJODLHHJNDHPFBIEEPEM
HTTP/1.1 404 Not Found
Content-Length: 1635
Content-Type: text/html
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Sat, 21 May 2011 16:20:48 GMT
I'm not sure what's going on here. The page displays just fine, but I'm getting the 500 error code in the header.
The problem was solved by using curl (which was also getting a 500, but fetched the page just fine) instead, but I'm curious what's going here.
Hi everybody I had this huge problem too, I don't know why, but the solution was add this:
wget -U "Opera 11.0" "http://your_link" -O out.csv
I found it on
[Curl and wget return error 500 for helloworld.php on new install but browser is fine
Using this option will fix the issue:
--content-on-error
If this is set to on, wget will not skip the content when the
server responds with a http status code that indicates error.
So the command looks like this:
wget --content-on-error "https://stackoverflow.com"
NOTE: It's important to put the URL inside double-quotes, otherwise, wget will get stuck on Redirecting output to ‘wget-log’..
Or as stated in the comments and by OP, use curl instead.
But I should note that curl cannot download whole webpages (css, js, images etc.) because it cannot parse HTML. Source and Taken from.
It's a bug in the webpage. The HTTP status is indeed seemingly incorrectly set to HTTP 500. Firefox/Firebug also confirms this. Basically, you're facing a HTTP 500 error page with "normal" content.
Report it to the site admin.
Try enclosing it in quotes:
wget "http://www.ptmytrade.com/product.asp?id=61363"
instead of:
wget http://www.ptmytrade.com/product.asp?id=61363
I am using NetStream, NetConnection and Video object to play an mp4 file which is hosted over a web server using http.
The mp4 file URL is for example: http://xx.xx.xx.xx/file.mp4
This is an AIR application and the relevant code is pasted below:
var url:String = <some http url>;
connect_nc = new NetConnection();
connect_nc.connect(null);
stream_ns = new NetStream(connect_nc);
var ns_object:Object = new Object();
ns_object.onPlayStatus = ns_onPlayStatus;
stream_ns.client = ns_object;
videoMP4.attachNetStream(stream_ns);
stream_ns.bufferTime = 1.0 // 1 sec
stream_ns.addEventListener(NetStatusEvent.NET_STATUS, onNetStatusEventHandler);
stream_ns.play(url);
This code works when run on MAC OS X. But it does not work when run on Windows XP. I get the error:
NetStream.Play.StreamNotFound
I also tried playing the URL using VLC player on the same windows XP host. The URL is valid because VLC can play it.
In my particular case, the http URL is hosted by WMP 12 (window media player 12) on Win 7 machine where I am using the media sharing feature of WMP 12.
After further looking into http traffic on wireshark, here is what i found.
After running wireshark on the host running the adobe AIR application, it seems that it is getting a HTTP 406 response from
the server being run by WMP 12.
GET /WMPNSSv4/63903908/1_ezVGREUzQTA4LTdDQzQtNDJFMy1CNDVDLUZEMjA4MDE5OUM4Q30uMC44.mp4 HTTP/1.1
Host: 192.168.0.102:10243
User-Agent: Mozilla/5.0 (Windows; U; en) AppleWebKit/526.9+ (KHTML, like Gecko) AdobeAIR/1.5
Referer: app:/clicker.swf
x-flash-version: 10,0,12,36
Connection: Keep-Alive
Accept: text/xml, application/xml, application/xhtml+xml, text/html;q=0.9, text/plain;q=0.8, text/css, image/png, image/jpeg, image/gif;q=0.8, application/x-shockwave-flash, video/mp4;q=0.9, flv-application/octet-stream;q=0.8, video/x-flv;q=0.7, audio/mp4, application/futuresplash, */*;q=0.5
Response:
HTTP/1.1 406 Not Acceptable
Last-Modified: Mon, 19 Oct 2009 23:21:14 GMT
Server: Microsoft-HTTPAPI/2.0
Accept-Ranges: bytes
TransferMode.DLNA.ORG: Streaming
Date: Tue, 12 Jan 2010 22:52:48 GMT
Connection: close
Content-Length: 0
On MAC:
It receives 200 OK response though, and that is why the video streaming works.
GET /WMPNSSv4/63903908/1_ezVGREUzQTA4LTdDQzQtNDJFMy1CNDVDLUZEMjA4MDE5OUM4Q30uMC44.m p4 HTTP/1.1
Host: 192.168.0.102:10243
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en) AppleWebKit/526.9+ (KHTML, like Gecko) AdobeAIR/1.5.3
Referer: app:/clicker.swf
X-Flash-Version: 10,0,42,34
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: keep-alive
Response:
HTTP/1.1 200 OK
Content-Length: 1524867
Content-Type: video/mp4
Last-Modified: Mon, 19 Oct 2009 23:21:14 GMT
Server: Microsoft-HTTPAPI/2.0
Accept-Ranges: bytes
TransferMode.DLNA.ORG: Streaming
Date: Tue, 12 Jan 2010 22:56:20 GMT
The difference that I can see in the HTTP requests between the Windows XP and MAC version is the Accept: Header. Is the Accept: header value wrong for Windows case because of which WMP 12 rejects
the http request.
If i run the adobe AIR application on Win 7 host, i see the same failure.
Am I using the NetStream object incorrectly or it is a bug in WMP 12 code not being able to parse
the header properly or it is a flex bug where it is generating an incorrect accept: header?
I believe WMP 12 incorrectly handles 'Accept' header in a request. If it contains 'q' (quality) parameter, then WMP ignores this mime-type. And if there are no other suitable mime-types for WMP, it will respond with 406 Not Acceptable error.
I encountered this issues when was trying to display DLNA image in Chrome browser.
I used curl utility to send requests with different headers to figure out what goes wrong.
Request that results in 406 Not Acceptable error:
curl -v -o file.jpg -H "Accept: text/html,*/*,q=0.8" "http://127.0.0.1:10243/WMPNSSv4/3065481158/0_e0I5MzA1MTRELUYwMEEtNEQwRC1CQzg4LTg3NEI5QjQ4MDYyM30uMC5C.jpg"
GET /WMPNSSv4/3065481158/0_e0I5MzA1MTRELUYwMEEtNEQwRC1CQzg4LTg3NEI5QjQ4MDYyM30uMC5C.jpg HTTP/1.1
User-Agent: curl/7.31.0
Host: 127.0.0.1:10243
Accept: text/html,*/*;q=0.8
HTTP/1.1 406 Not Acceptable
Last-Modified: Tue, 21 May 2013 21:01:09 GMT
Server: Microsoft-HTTPAPI/2.0
Accept-Ranges: bytes
TransferMode.DLNA.ORG: Interactive
Date: Fri, 30 Aug 2013 09:10:32 GMT
Connection: close
Content-Length: 0
Successful request:
curl -v -o file.jpg -H "Accept: text/html,*/*" "http://127.0.0.1:10243/WMPNSSv4/3065481158/0_e0I5MzA1MTRELUYwMEEtNEQwRC1CQzg4LTg3NEI5QjQ4MDYyM30uMC5C.jpg"
GET /WMPNSSv4/3065481158/0_e0I5MzA1MTRELUYwMEEtNEQwRC1CQzg4LTg3NEI5QjQ4MDYyM30uMC5C.jpg HTTP/1.1
User-Agent: curl/7.31.0
Host: 127.0.0.1:10243
Accept: text/html,*/*
HTTP/1.1 200 OK
Content-Length: 2394679
Content-Type: image/jpeg
Last-Modified: Tue, 21 May 2013 21:01:09 GMT
Server: Microsoft-HTTPAPI/2.0
Accept-Ranges: bytes
TransferMode.DLNA.ORG: Interactive
Date: Fri, 30 Aug 2013 09:10:40 GMT
I am working on a simple download application. While making a request for the following file both firefox and my application doesn't get the content-length field. But if i make the request using wget server does send the content-length field. I did change wgets user agent string to test and it still got the content-length field.
Any ideas why this is happening?
wget request
---request begin---
GET /dc-13/video/2005_Defcon_V2-P_Zimmerman-Unveiling_My_Next_Big_Project.mp4 HTTP/1.0
User-Agent: test
Accept: */*
Host: media.defcon.org
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.0 200 OK
Server: lighttpd
Date: Sun, 05 Apr 2009 04:40:08 GMT
Last-Modified: Tue, 23 May 2006 22:18:19 GMT
Content-Type: video/mp4
Content-Length: 104223909
Connection: keep-alive
firefox request
GET /dc-13/video/2005_Defcon_V2-P_Zimmerman-Unveiling_My_Next_Big_Project.mp4 HTTP/1.1
Host: media.defcon.org
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.0.8) Gecko/2009032608 Firefox/3.0.8
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://www.defcon.org/html/links/defcon-media-archives.html
Pragma: no-cache
Cache-Control: no-cache
HTTP/1.x 200 OK
Server: lighttpd
Date: Sun, 05 Apr 2009 05:20:12 GMT
Last-Modified: Tue, 23 May 2006 22:18:19 GMT
Content-Type: video/mp4
Transfer-Encoding: chunked
Update:
Is there a header that I can send that will tell Lighthttpd not to use chunked encoding.My original problem is that I am using urlConnection to grab the file in my java application which automatically sends HTTP 1.1 request.
I would like to know the size of the file so i can update my percentage.
GET
/dc-13/video/2005_Defcon_V2-P_Zimmerman-Unveiling_My_Next_Big_Project.mp4
HTTP/1.1
Firefox is performing an HTTP 1.1 GET request. Lighthttpd understands that the client will support chunked-transfer encoding and returns the content in chunks, with each chunk reporting its own length.
GET
/dc-13/video/2005_Defcon_V2-P_Zimmerman-Unveiling_My_Next_Big_Project.mp4
HTTP/1.0
Wget on the other hand performs an HTTP 1.0 GET request. Lighthttpd, understanding that the client doesn't support HTTP 1.1 (and thus chunked-transfer encoding), returns the content in one single chunk, with the length reported in the response header.
Looks like it's because of the chunked transfer encoding:
Transfer-Encoding: chunked
This will send the video down in chunks, each with its own size. This is defined in HTTP 1.1, which is what Firefox is using, while wget is using HTTP 1.0, which doesn't support chunked transfer encoding, so the server has to send the whole file at once.
I was having the same problem and found a solution regardless of which HTTP version:
First use a HEAD request to the server which correctly responds with just the HTTP header and no contents. This header correctly includes the wanted Content-Length: bytes size for the file to download.
Proceed with the GET request to download the file (the header from the GET response fails to include Content-length).
An Objective-C language example:
NSString *zipURL = #"http://1.bp.blogspot.com/_6-cw84gcURw/TRNb3PDWneI/AAAAAAAAAYM/YFCZP1foTiM/s1600/paragliding1.jpg";
NSURL *url = [NSURL URLWithString:zipURL];
// Configure the HTTP request for HEAD header fetch
NSMutableURLRequest *urlRequest = [NSMutableURLRequest requestWithURL:url];
urlRequest.HTTPMethod = #"HEAD"; // Default is "GET"
// Define response class
__autoreleasing NSHTTPURLResponse *response;
// Send HEAD request to server
NSData *contentsData = [NSURLConnection sendSynchronousRequest:urlRequest returningResponse:&response error:nil];
// Header response field
NSDictionary *headerDeserialized = response.allHeaderFields;
// The contents length
int contents_length = [(NSString*)headerDeserialized[#"Content-Length"] intValue];
//printf("HEAD Response header: %s\n",headerDeserialized.description.UTF8String);
printf("HEAD:\ncontentsData.length: %d\n",contentsData.length);
printf("contents_length = %d\n\n",contents_length);
urlRequest.HTTPMethod = #"GET";
// Send "GET" to download file
contentsData = [NSURLConnection sendSynchronousRequest:urlRequest returningResponse:&response error:nil];
// Header response field
headerDeserialized = response.allHeaderFields;
// The contents length
contents_length = [(NSString*)headerDeserialized[#"Content-Length"] intValue];
printf("GET Response header: %s\n",headerDeserialized.description.UTF8String);
printf("GET:\ncontentsData.length: %d\n",contentsData.length);
printf("contents_length = %d\n",contents_length);
return;
And the output:
HEAD:
contentsData.length: 0
contents_length = 146216
GET:
contentsData.length: 146216
contents_length = 146216
(Note: This example URL does correctly provides the header Content-Length from the GET response, but shows the idea if it failed to)