I tried to make a GET request for the netflix home page with command prompt because the response returned to me was a 302/301 all the time. So I connected to netflix via the following:
telnet signup.netflix.com 80
Then the request I made was
GET / HTTP/1.1
Host: signup.netflix.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-gb,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
As copied exactly from LiveHTTPHeaders when I visit netflix however I removed the Cookie part because I don't know where the browser (FireFox) is getting these values
Netflix responds with
Why don't I get a 200 OK status code? Is it because I'm not sending any cookies?
Its doing a redirect to https://signup.netflix.com/?tcw=2. i.e., it wants you to resend the request with the twc=2 variable in the query string and more importantly, with the cookie it just gave you through the Set-Cookie header.
Related
I'm using Digest Authentication to check if a user knows the ID and PW of the server. Once the client has entered the server because he knows the credentials (the server has matched its hash response with the hash reponse of the client) all his requests have the Authorization header. Is this normal? The authorization header is not sent only once?
Once the client has passed the Digest Challenge all his requests are like the following (all the requests have the Authorization header):
GET /XMLAliasRegDev HTTP/1.1
***Here's the Host***
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0
Accept: */*
Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://Here's the Host/webpage/html/registry.htm
Authorization: Digest username="xxx", realm="xxx", nonce="b463d286b77fba6535adc1902e43377a", uri="/XMLAliasRegDev", response="4bedc10d3fd7f3fb90ab518ffead238b", opaque="eb2cdfdb6ebd0e78c0737bc4d58d0d3c"
Connection: keep-alive
GET /webpage/scripts/regjs.js HTTP/1.1
***Here's the Host***
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0
Accept: */*
Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://Here's the Host/webpage/html/registry.htm
Authorization: Digest username="xxx", realm="xxx", nonce="b463d286b77fba6535adc1902e43377a", uri="/webpage/scripts/regjs.js", response="95cd7035c6abf7666fbdb0068aa69b9a", opaque="eb2cdfdb6ebd0e78c0737bc4d58d0d3c"
Connection: keep-alive
If the server has matched the responses and the server is sending the resources to the client, why the client is sending the authorization header?
TY!
PS: I'm using Arduino as a server.
Is this normal?
Absolutely.
The authorization header is not sent only once?
No, as written in RFC 2069 a valid response contains the challenge response. It is then sent in all requests for protected resources in a realm with digest authentication.
What would cause Internet Explorer to replace the HTTP header
Authorization : Bearer <server-provided-token>
with
Authorization : Negotiate <some token>
when making an AJAX request?
In Internet Explorer, some AJAX requests that are configured to contain the header Authorization: Bearer ... are being sent by Internet Explorer with the header Authorization: Negotiate ... instead.
For example, Fiddler shows that the first two of three requests contain the Authorization : Bearer... header, while the third suddenly contains the Authorization : Negotiate... header. The first two requests are successful, and the third fails because the request can't be properly authenticated.
All of the requests are constructed using the same client-side code, and are made one after another (within the span of a second). I have verified that the Authorization header correctly contains the Bearer token in all three cases up until the point the request is provided to the browser.
Also, I'm not seeing the same behavior in Chrome; it's only occurring in IE.
Request 1
GET http://localhost/myapp/api/User HTTP/1.1
Accept: application/json, text/plain, */*
Authorization: Bearer oEXS5IBu9huepzW6jfh-POMA18AUA8yWZsPfBPZuFf_JJxq-DKIt0JDyPXSiGpmV_cpT8FlL3D1DN-Tv5ZbT73MTuBOd5y75-bsx9fZvOeJgg04JcO0cUajdCH2h5QlMP8TNwgTpHg-TR9FxyPk3Kw6bQ6tQCOkOwIG_FmEJpP89yrOsoYJoCfrAoZ7M4PVcik9F9qtPgXmWwXB2eHDtkls44wITF_yM_rPm5C47OPCvMVTPz30KwoEPi6fHUcL3qHauP-v9uypv2e48TyPHUwLYmNFxyafMhBx4TkovnRcsdLHZiHmSjMq0V9a2Vw70
Referer: http://localhost/client/login.html
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Host: localhost
DNT: 1
Connection: Keep-Alive
Request 2
POST http://localhost/myapp/api/Permissions HTTP/1.1
Referer: http://localhost/client/#/Dashboard
Content-Type: application/json
Authorization: Bearer oEXS5IBu9huepzW6jfh-POMA18AUA8yWZsPfBPZuFf_JJxq-DKIt0JDyPXSiGpmV_cpT8FlL3D1DN-Tv5ZbT73MTuBOd5y75-bsx9fZvOeJgg04JcO0cUajdCH2h5QlMP8TNwgTpHg-TR9FxyPk3Kw6bQ6tQCOkOwIG_FmEJpP89yrOsoYJoCfrAoZ7M4PVcik9F9qtPgXmWwXB2eHDtkls44wITF_yM_rPm5C47OPCvMVTPz30KwoEPi6fHUcL3qHauP-v9uypv2e48TyPHUwLYmNFxyafMhBx4TkovnRcsdLHZiHmSjMq0V9a2Vw70
Accept: application/json, text/plain, */*
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Host: localhost
Content-Length: 1419
DNT: 1
Connection: Keep-Alive
Pragma: no-cache
<Post Data Removed>
Request 3
GET http://localhost/myapp/api/UserPreferences/Dashboard HTTP/1.1
Referer: http://localhost/client/#/Dashboard
Content-Type: application/json
Authorization: Negotiate YHsGBisGAQUFAqBxMG+gMDAuBgorBgEEAYI3AgIKBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICHqI7BDlOVExNU1NQAAEAAACXsgjiBgAGADMAAAALAAsAKAAAAAYBsR0AAAAPVk1ERVZFTlYtU1JTQ0VSSVM=
Accept: application/json, text/plain, */*
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Connection: Keep-Alive
DNT: 1
Host: localhost
The requests are being made via the AngularJS $http service, and the back-end is ASP.NET Web API hosted in IIS.
I have tried fixing this issue with
document.execCommand('ClearAuthenticationCache', 'false');"
But this leads to another issue. That is if I log in using form based authentication in my application then on page refresh the IIS picks up the windows authenticated user and changes the current logged in user to window's authenticated user.
I would like to make a file download resumable using byte-range requests.
The problem is that my existing download action is responding on a POST method and I would like to keep it that way.
But it seems from my early tests that Chrome turns interrupted POST requests for file downloads into GET requests when the user tries to resume and thus the resuming of the download fails.
Am I missing something?
Is this something related to the HTTP specs that only allow GET requests to be resumed?
Or is it simply a design flaw in Chrome (and maybe other browsers as well) that makes it forget the original HTTP method used?
UPDATE:
Here are the request/response data:
Initial POST request:
POST http://localhost:35547/Download?Guid=396b4697-e275-4396-818c-548bf8c0a281 HTTP/1.1
Host: localhost:35547
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Origin: http://localhost:35547
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Referer: http://localhost:35547/File/396b4697-e275-4396-818c-548bf8c0a281
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
Cookie: __RequestVerificationToken=LuPgM05MHrsuyskgfhsrHVUs; ASP.NET_SessionId=gfiulghfuygisghkf; .ASPXAUTH=FGDJHGDHSDFB15AFDE6371CGJHDFGFBHD; fileDownload=true
Initial response (to the request above):
HTTP/1.1 200 OK
Cache-Control: private, s-maxage=0
Content-Type: application/zip
Server: Microsoft-IIS/7.5
X-AspNetMvc-Version: 5.2
Content-Disposition: attachment; filename="FILE-396b4697e2754396818c548bf8c0a281.zip"
X-AspNet-Version: 4.0.30319
Set-Cookie: fileDownload=true; path=/
X-Powered-By: ASP.NET
Date: Wed, 09 Nov 2016 11:13:50 GMT
Content-Length: 1885473
PK.......... ZIP file data .............................................
After the interruption, this is the request that the browser does on resume (notice the GET method used):
GET http://localhost:35547/Download?Guid=396b4697-e275-4396-818c-548bf8c0a281 HTTP/1.1
Host: localhost:35547
Connection: keep-alive
Referer: http://localhost:35547/File/396b4697-e275-4396-818c-548bf8c0a281
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36
Accept-Encoding: gzip, deflate, sdch, br
Accept-Language: en-US,en;q=0.8
Cookie: __RequestVerificationToken=.............
(Some data from security-related cookies have been shortened and altered)
Am I missing something ?
It depends, upon how have you analyzed the behavior of google chrome. Ideal way will be use any proxy or to use packet sniffer such as Wireshark to see what request method, is used by chrome in subsequent request.
Is this something related to the HTTP specs that only allow GET
requests to be resumed?
As of now, there's no mention in spec of HTTP protocol, that only GET requests can be resumed.
Or is it simply a design flaw in Chrome (and maybe other browsers as
well) that makes it forget the original HTTP method used?
Yes, It's the flaw of google chrome. Make sure that you check it on the latest version of Google chrome with all the update patches. Also check it on other browsers.
For more info about HTTP protocol, refer to https://www.ietf.org/rfc/rfc2616.txt.
Refer to following request, for serving partial response : https://en.wikipedia.org/wiki/Byte_serving
Edit
For more updated info regarding HTTP info, refer to :-
https://www.rfc-editor.org/rfc/rfc7230
We have an ASP.NET application that works without any problem in previous version of Internet Explorer 10.
But the same application does not work in Internet Explorer 10.
When I investigated the issue, I found out the references of .asmx web services gets 401-unauthorized error.
The only difference between an problematic request of IE10 and working request of IE9 is : User-Agent string.
Let me show you the difference:
Below request is the problematic request of IE10.
GET http://abc.com/Services/ResourceService.asmx/jsdebug HTTP/1.1
Accept: application/javascript, */*;q=0.8
Referer: http://abc.com/def/ghi.aspx
Accept-Language: tr-TR
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
Connection: Keep-Alive
DNT: 1
Host: sgmiis2:8090
Cookie: .ASPXANONYMOUS=XXXASP.NET_SessionId=YYY;ASPXAUTH=ZZZ;
It is getting
401 - Unauthorized: Access is denied due to invalid credentials.
Here is the same application's request via before IE10 .
GET http://abc.com/Services/ResourceService.asmx/jsdebug HTTP/1.1
Accept: application/javascript, */*;q=0.8
Referer: http://abc.com/def/ghi.aspx
Accept-Language: tr-TR
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/6.0;)
Connection: Keep-Alive
DNT: 1
Host: sgmiis2:8090
Cookie: .ASPXANONYMOUS=XXXASP.NET_SessionId=YYY;ASPXAUTH=ZZZ;
But this request is working and getting
HTTP/1.1 200 OK
As you can see the only difference is "MSIE 10.0" and "MSIE 7.0/MSIE 8.0/MSIE 9.0" sections of user-agent header.
I am creating the request via Fiddler's Composer section. So there is no risk of really getting 401 error. (with faulty cookies I mean) Because when I changed the user-agent string only, I can successfully get the response of HTTP 200.
Any idea how can I figure out the issue?
Thanks,
Fatih.
There are so many issues with ie 10.
A hotfix is available for the ASP.NET browser definition files in the Microsoft .NET
please refer http://support.microsoft.com/kb/2600088 for .NET 4.0
Refer http://support.microsoft.com/kb/2608565 for .Net 3.5
Please also check this :-http://connect.microsoft.com/VisualStudio/feedback/details/662275/asp-net-user-agent-sniffing-and-ie10-internet-explorer-10
I hope it will help you.
Despite what the title may suggest, this is not related to the common "preventing double post request" issue.
In my application, I add some data on outgoing HTTP traffic, and with some some websites, I randomly encounter an HTTP POST request that has a double-header, resulting in a server termination, as I assume the server doesn't understand the request and decides to cut me off. As the title says, I'm literally seeing two POST headers in a single request. This only happens when I append some custom HTTP fields to the header. For example, I came across this today when I followed a surveygizmo.com link, as seen in the trace at the bottom of this post.
I cannot understand if it's the browser that's doing something funky because it noticed I've modified some data, or it's something in my LSP application that causes this to happen.
When I debug my application, I only see the intercepted request the first time, which is when I inject the custom data. After that, I don't see the request anywhere except in Wireshark, so it's not like I can remediate the double headers by deleting the redundant data.
Things to note looking at the trace:
The data I'm appending is 'Custom-FieldN:'
Two almost-identical headers
Three double-CRLF's in one single request header (how is that possible?)
The Request:
POST http://www.surveygizmo.com/s3/1212345/Who-Are-You HTTP/1.1
Host: www.surveygizmo.com
Custom-Field1: UserNameBob
Custom-Field2: 2578291789
proxy-connection: keep-alive
Content-Length: 836
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://www.surveygizmo.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryaQraA7ZABICMT6jO
Referer: http://www.surveygizmo.com/s3/1212345/Who-Are-You
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6,ja;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: sg-response-979523-1212345=0%3B1369154430_519ba37e477bc8.35524744%3B1369154430%3BPartial
POST http://www.surveygizmo.com/s3/1212345/Who-Are-You HTTP/1.1
Host: www.surveygizmo.com
Custom-Field1: UserNameBob
Custom-Field2: 2578291789
proxy-connection: keep-alive
Content-Length: 836
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://www.surveygizmo.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryaQraA7ZABICMT6jO
Referer: http://www.surveygizmo.com/s3/1212345/Who-Are-You
accept-encoding: gzip,deflate
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6,ja;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: sg-response-979523-1212345=0%3B1369154430_519ba37e477bc8.35524744%3B1369154430%3BPartial
------WebKitFormBoundaryaQraA7ZABICMT6jO
Content-Disposition: form-data; name="sg_navchoice"