Server does not respond to manual request - http

I'm trying to connect to api.poloniex.com on port 443 (SSL). They are using a WebSocket/Wamp server on that port which is pushing data. I've already tried to connect with Autobahn|JS and it works.
Before the data transmission begins, a Http-Upgrade-Request is sent to the server, which indicates that the client wants to switch from http to websocket. Then the server normally responds with 101 - Switching protocols.
Using the Autobahn|JS library, the request looks like this:
Request:
GET / HTTP/1.1
Host: api.poloniex.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Sec-WebSocket-Version: 13
Origin: ...
Sec-WebSocket-Protocol: wamp.2.json
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: 0n4ibEwIyTMDMhKoxXt+Cg==
Cookie: ......
Connection: keep-alive, Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Response:
HTTP/1.1 101 Switching Protocols
Server: Crossbar/0.10.4
X-Powered-By: AutobahnPython/0.10.4
Upgrade: WebSocket
Connection: Upgrade
Sec-WebSocket-Protocol: wamp.2.json
Sec-WebSocket-Accept: dbaqDVdVW+xtbCa2V0yjtb7yXqI=
I tried to manually send the Request-Header. I used openssl s_client -connect api.poloniex.com:443 to connect with SSL-encryption. Then I sent a slightly modified header (I removed Origin and Cookie):
GET / HTTP/1.1
Host: api.poloniex.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
Sec-WebSocket-Version: 13
Sec-WebSocket-Protocol: wamp.2.json
Sec-WebSocket-Extensions: permessage-deflate
Sec-WebSocket-Key: czW+w3Z+3yZCKVGx6DQ1Rg==
Connection: keep-alive, Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
But then, the server does not answer. It does not even close the connection, just nothin happens at all. I've also tried to leave the Cookie and Origin fields in, but that does also not work.
Edit: Sending the message this way does also not work:
GET / HTTP/1.1\r\nHost: api.poloniex.com\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:43.0) Gecko/20100101 Firefox/43.0\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: de,en-US;q=0.7,en;q=0.3\r\nAccept-Encoding: gzip, deflate\r\nSec-WebSocket-Version: 13\r\nSec-WebSocket-Protocol: wamp.2.json\r\nSec-WebSocket-Extensions: permessage-deflate\r\nSec-WebSocket-Key: czW+w3Z+3yZCKVGx6DQ1Rg==\r\nConnection: keep-alive, Upgrade\r\nPragma: no-cache\r\nCache-Control: no-cache\r\nUpgrade: websocket\r\n\r\n

HTTP requests have a line end of \r\n but often single \n will be accepted. But what you probably missed is the final empty line which is the end of header marker. Thus the server still waits for the end of the header.

Related

Authorization header

I'm using Digest Authentication to check if a user knows the ID and PW of the server. Once the client has entered the server because he knows the credentials (the server has matched its hash response with the hash reponse of the client) all his requests have the Authorization header. Is this normal? The authorization header is not sent only once?
Once the client has passed the Digest Challenge all his requests are like the following (all the requests have the Authorization header):
GET /XMLAliasRegDev HTTP/1.1
***Here's the Host***
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0
Accept: */*
Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://Here's the Host/webpage/html/registry.htm
Authorization: Digest username="xxx", realm="xxx", nonce="b463d286b77fba6535adc1902e43377a", uri="/XMLAliasRegDev", response="4bedc10d3fd7f3fb90ab518ffead238b", opaque="eb2cdfdb6ebd0e78c0737bc4d58d0d3c"
Connection: keep-alive
GET /webpage/scripts/regjs.js HTTP/1.1
***Here's the Host***
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0
Accept: */*
Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Referer: http://Here's the Host/webpage/html/registry.htm
Authorization: Digest username="xxx", realm="xxx", nonce="b463d286b77fba6535adc1902e43377a", uri="/webpage/scripts/regjs.js", response="95cd7035c6abf7666fbdb0068aa69b9a", opaque="eb2cdfdb6ebd0e78c0737bc4d58d0d3c"
Connection: keep-alive
If the server has matched the responses and the server is sending the resources to the client, why the client is sending the authorization header?
TY!
PS: I'm using Arduino as a server.
Is this normal?
Absolutely.
The authorization header is not sent only once?
No, as written in RFC 2069 a valid response contains the challenge response. It is then sent in all requests for protected resources in a realm with digest authentication.

Internet Explorer 11 replaces Authorization header || Not even working after using document.execCommand('ClearAuthenticationCache', 'false');

What would cause Internet Explorer to replace the HTTP header
Authorization : Bearer <server-provided-token>
with
Authorization : Negotiate <some token>
when making an AJAX request?
In Internet Explorer, some AJAX requests that are configured to contain the header Authorization: Bearer ... are being sent by Internet Explorer with the header Authorization: Negotiate ... instead.
For example, Fiddler shows that the first two of three requests contain the Authorization : Bearer... header, while the third suddenly contains the Authorization : Negotiate... header. The first two requests are successful, and the third fails because the request can't be properly authenticated.
All of the requests are constructed using the same client-side code, and are made one after another (within the span of a second). I have verified that the Authorization header correctly contains the Bearer token in all three cases up until the point the request is provided to the browser.
Also, I'm not seeing the same behavior in Chrome; it's only occurring in IE.
Request 1
GET http://localhost/myapp/api/User HTTP/1.1
Accept: application/json, text/plain, */*
Authorization: Bearer oEXS5IBu9huepzW6jfh-POMA18AUA8yWZsPfBPZuFf_JJxq-DKIt0JDyPXSiGpmV_cpT8FlL3D1DN-Tv5ZbT73MTuBOd5y75-bsx9fZvOeJgg04JcO0cUajdCH2h5QlMP8TNwgTpHg-TR9FxyPk3Kw6bQ6tQCOkOwIG_FmEJpP89yrOsoYJoCfrAoZ7M4PVcik9F9qtPgXmWwXB2eHDtkls44wITF_yM_rPm5C47OPCvMVTPz30KwoEPi6fHUcL3qHauP-v9uypv2e48TyPHUwLYmNFxyafMhBx4TkovnRcsdLHZiHmSjMq0V9a2Vw70
Referer: http://localhost/client/login.html
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Host: localhost
DNT: 1
Connection: Keep-Alive
Request 2
POST http://localhost/myapp/api/Permissions HTTP/1.1
Referer: http://localhost/client/#/Dashboard
Content-Type: application/json
Authorization: Bearer oEXS5IBu9huepzW6jfh-POMA18AUA8yWZsPfBPZuFf_JJxq-DKIt0JDyPXSiGpmV_cpT8FlL3D1DN-Tv5ZbT73MTuBOd5y75-bsx9fZvOeJgg04JcO0cUajdCH2h5QlMP8TNwgTpHg-TR9FxyPk3Kw6bQ6tQCOkOwIG_FmEJpP89yrOsoYJoCfrAoZ7M4PVcik9F9qtPgXmWwXB2eHDtkls44wITF_yM_rPm5C47OPCvMVTPz30KwoEPi6fHUcL3qHauP-v9uypv2e48TyPHUwLYmNFxyafMhBx4TkovnRcsdLHZiHmSjMq0V9a2Vw70
Accept: application/json, text/plain, */*
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Host: localhost
Content-Length: 1419
DNT: 1
Connection: Keep-Alive
Pragma: no-cache
<Post Data Removed>
Request 3
GET http://localhost/myapp/api/UserPreferences/Dashboard HTTP/1.1
Referer: http://localhost/client/#/Dashboard
Content-Type: application/json
Authorization: Negotiate YHsGBisGAQUFAqBxMG+gMDAuBgorBgEEAYI3AgIKBgkqhkiC9xIBAgIGCSqGSIb3EgECAgYKKwYBBAGCNwICHqI7BDlOVExNU1NQAAEAAACXsgjiBgAGADMAAAALAAsAKAAAAAYBsR0AAAAPVk1ERVZFTlYtU1JTQ0VSSVM=
Accept: application/json, text/plain, */*
Accept-Language: en-US
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko
Connection: Keep-Alive
DNT: 1
Host: localhost
The requests are being made via the AngularJS $http service, and the back-end is ASP.NET Web API hosted in IIS.
I have tried fixing this issue with
document.execCommand('ClearAuthenticationCache', 'false');"
But this leads to another issue. That is if I log in using form based authentication in my application then on page refresh the IIS picks up the windows authenticated user and changes the current logged in user to window's authenticated user.

HTTP Request to IceCast and Response

There's an HTTP request to the IceCast 2.3.2-kh29 server MP3 stream http://*:*/.mp3 and response (some data are **ed):
GET /*.mp3 HTTP/1.1
Host: ***:*
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
HTTP/1.0 200 OK
Content-Type: audio/mpeg
icy-br: 192
ice-audio-info: bitrate=192;samplerate=44100;channels=2
icy-description: MP3 192 Kbps
icy-genre: *
icy-name: *
icy-pub: 1
icy-url: http://*
Server: Icecast 2.3.2-kh29
Cache-Control: no-cache
Expires: Mon, 26 Jul 1997 05:00:00 GMT
Pragma: no-cache
The Content-Length is not specified in the response. Also keep-alive is missing, despite the request. Is it normal situation? The protocol specifies that Content-Length must be in this case. The stream starts playing. Can be somehow that not all headers are shown in LiveHTTPHeaders this way? Or the stream is a special case, when used some sort of artificial made Content-Length?
HTTP 1.0 does not require that a content length is specified. SHOUTcast/Icecast servers will not specify the content length because the streams are live and there is no pre-determined length.

Two HTTP POST headers in a request after modifying the header

Despite what the title may suggest, this is not related to the common "preventing double post request" issue.
In my application, I add some data on outgoing HTTP traffic, and with some some websites, I randomly encounter an HTTP POST request that has a double-header, resulting in a server termination, as I assume the server doesn't understand the request and decides to cut me off. As the title says, I'm literally seeing two POST headers in a single request. This only happens when I append some custom HTTP fields to the header. For example, I came across this today when I followed a surveygizmo.com link, as seen in the trace at the bottom of this post.
I cannot understand if it's the browser that's doing something funky because it noticed I've modified some data, or it's something in my LSP application that causes this to happen.
When I debug my application, I only see the intercepted request the first time, which is when I inject the custom data. After that, I don't see the request anywhere except in Wireshark, so it's not like I can remediate the double headers by deleting the redundant data.
Things to note looking at the trace:
The data I'm appending is 'Custom-FieldN:'
Two almost-identical headers
Three double-CRLF's in one single request header (how is that possible?)
The Request:
POST http://www.surveygizmo.com/s3/1212345/Who-Are-You HTTP/1.1
Host: www.surveygizmo.com
Custom-Field1: UserNameBob
Custom-Field2: 2578291789
proxy-connection: keep-alive
Content-Length: 836
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://www.surveygizmo.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryaQraA7ZABICMT6jO
Referer: http://www.surveygizmo.com/s3/1212345/Who-Are-You
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6,ja;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: sg-response-979523-1212345=0%3B1369154430_519ba37e477bc8.35524744%3B1369154430%3BPartial
POST http://www.surveygizmo.com/s3/1212345/Who-Are-You HTTP/1.1
Host: www.surveygizmo.com
Custom-Field1: UserNameBob
Custom-Field2: 2578291789
proxy-connection: keep-alive
Content-Length: 836
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://www.surveygizmo.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryaQraA7ZABICMT6jO
Referer: http://www.surveygizmo.com/s3/1212345/Who-Are-You
accept-encoding: gzip,deflate
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6,ja;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: sg-response-979523-1212345=0%3B1369154430_519ba37e477bc8.35524744%3B1369154430%3BPartial
------WebKitFormBoundaryaQraA7ZABICMT6jO
Content-Disposition: form-data; name="sg_navchoice"

Http Request not reaching the IIS module

During development of an IIS module for basic authentication, I stocked to a problem. The module is working fine when browsing the pages, but when calling web-services it seems that request does not reach the module and some in-the-middle module takes control of request.
using fiddler, I found out when Content-type in http request header is set to application/json that in the middle module/handler is triggered. so following request does not work:
when working fine, the server should ask client to send the user credentials by setting the WWW-Authenticate header in response
GET /WebServices/service.asmx/someMethod?param=test HTTP/1.1
Host: localhost
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Content-Type: application/json
asdfasdf
asdfasdfasdf
response: notice the jsonerror header in response
HTTP/1.1 401 Unauthorized
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/7.5
jsonerror: true
X-Powered-By: ASP.NET
Date: Mon, 11 Mar 2013 23:49:02 GMT
Content-Length: 105
{"Message":"Authentication failed.","StackTrace":null,"ExceptionType":"System.In
validOperationException"}
where this one works fine: notice that there is no content-type
GET /WebServices/service.asmx/someMethod?param=test HTTP/1.1
Host: localhost
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
asdfasdf
asdfasdfasdf
and the correct response is: notice the WWW-Authenticate header in response
HTTP/1.1 401 Unauthorized
Location: http://localhost/WebServices/service.asmx/someMethod?param=test
Server: Microsoft-IIS/7.5
WWW-Authenticate: Basic
X-Powered-By: ASP.NET
Date: Mon, 11 Mar 2013 23:59:48 GMT
Content-Length: 0
Well, that in-the-middle module was ScriptModule where we had both 3.5 and 4.0 version being added in the config. inspecting them through dotpeek, I found that the script module checks request's content-type against being application/json and then tries to handle the request as a REST request or webservice call.
By removing them, nothing special happened. I assume that they are to be used when script manager or Microsoft Specific AJAX services are used. You can find more about it in
ASP.Net Ajax Programming Tricks

Resources