I am getting a HTTP 404 when sending a request to a web service on localhost:
http://localhost/my-ws/Test
In Firefox I get a HTTP 200 OK:
GET /my-ws/Test HTTP/1.1
Host: localhost
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20100101 Firefox/13.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cache-Control: max-age=0
In Opera:
GET /my-ws/Test HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; en; rv:2.0) Gecko/20100101 Firefox/4.0 Opera 12.00
Host: localhost
Accept: text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/webp, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1
Accept-Language: en,de-DE;q=0.9,de;q=0.8
Accept-Encoding: gzip, deflate
Pragma: no-cache
Cache-Control: no-cache
Connection: Keep-Alive
What is wrong here? What parameters to provide to RCurl to get the same result as in Firefox?
Related
when investigating network behavior, I usually use postman for sending HTTP requests, however
I need the option to send a raw HTTP request (via clear text), or at least only the headers, and it seems that postman does not support to edit your request via clear HTTP text. (buy the way the opposite is possible, you can read the raw http text of the requests you constructed in postman but you can't edit them)
for example:
Accept: text/javascript, text/html, application/xml, text/xml, */*
Accept-Encoding: gzip, deflate, br
Accept-Language: he-IL,he;q=0.9,en-US;q=0.8,en;q=0.7
Cache-Control: no-cache
Connection: keep-alive
Content-Length: 21114
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Cookie: csrftoken=0alLaljTasofjCWZv7gcmukXuz6gMxfzlWpV691hzZZ1hTBcdVJ3mH8ozRDnO6hu; tk_or=%22%22; tk_lr=%22%22; session_id_12211=ff6a58b0baf98005748ce5a3c6a732aef33b750f; splunkweb_csrf_token_12211=10024448868272708216; token_key=10024448868272708216; experience_id=4852e1c6-726b-1ab3-bafa-f0a735d3f708; splunkd_12211=NjcrwAj_TLgz5JalVh2HTynLdbp_CPnfHFKi8qmsODiH40HI2urbPvAvJ9uvDKKoM3nATXEkS6dGytD0TvfiOtAUGJhk7Od25on_gJcZrQwcePQZ8HQaCmGScm^RXmOdDa^KVvN
Host: localhost:12211
Origin: http://localhost:12211
Pragma: no-cache
sec-ch-ua: "Chromium";v="106", "Google Chrome";v="106", "Not;A=Brand";v="99"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36
X-Requested-With: XMLHttpRequest
X-Splunk-Form-Key: 10024448898272708216
does postman allow editing the raw HTML? if not, there is other tool that can?
I'm trying to upload a file but I get an error message like the following:
https://sapco.nyc3.digitaloceanspaces.com/b3vcchphzgj5m8p6ld51yk7867uu?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=QTHCA5KKQUHAKMMATAIP%2F20200503%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20200503T214158Z&X-Amz-Expires=10800&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=7b77a0f1551a262586980b709fdc44a2bc173ab6ae7279385e831493b1d13e53
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>BadDigest</Code>
<BucketName>sapco</BucketName>
<RequestId>tx000000000000013462bc9-005eaf36fb-3518e03-nyc3a</RequestId>
<HostId>3518e03-nyc3a-nyc</HostId>
</Error>
How can I debug this further?
So far I have tried:
Resetting my Access and Secret keys.
At first I thought this was related to PWA-related work, but I migrated to an earlier branch and have the same issues.
Tried different files: each have the same error.
Happens both in prod. and locally.
Update 1: This randomly started working again on production. However it's still broken locally.
With the same file I have the 2 paths for the PUT request that leads to the error above.
https://sapco.nyc3.digitaloceanspaces.com/11eego5a6r9b4tslx7cex4p9x45u?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=QTHCA5KKQUHAKMMATAIP%2F20200504%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20200504T005319Z&X-Amz-Expires=10800&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=8d2037f7370eb137facc9d813fe35ed34e055313af06cd66819a72d886dfb018
https://sapco.nyc3.digitaloceanspaces.com/z4vc7ujtvid0akqfn4uou46407zl?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=QTHCA5KKQUHAKMMATAIP%2F20200504%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20200504T005405Z&X-Amz-Expires=10800&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=b4b28cebe56a9b6c12ddfb2cc335b84080a3bfc5e34e2c66e19001230f8b7512
The issue must be related to the way X-Amz-Signature is computed.
Digging further with bundle open activestorage I can see it's roughly here. https://cutt.ly/6yjc7u1
I verified the Content-Length and Content-MD5 are both the same (vs local and prod). (123803 and ujNHxwCuwZ1mak927GUX3g== respectively).
Update 2: I tried this in Firefox with the same image and no problem locally. There must be something fishy going on with the cache. I then tried an Incognito window and that also seemed to work. Finally, I did a hard refresh in Chrome and now I've unblocked myself. Didn't quite figure out what was going on but leaving a final piece of information for anyone else:
Chrome Request Headers (Not Working, 400 Error)
PUT /lw5lufemkgb7ww83pdc56qg2gb0j?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=QTHCA5KKQUHAKMMATAIP%2F20200504%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20200504T013900Z&X-Amz-Expires=10800&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=1ea88bf8550d9bab67b5bca3aa97f7b15f1a44e117dd4f5cea0744c898f70684 HTTP/1.1
Host: sapco.nyc3.digitaloceanspaces.com
Connection: keep-alive
Content-Length: 0
Pragma: no-cache
Cache-Control: no-cache
Accept: */*
DNT: 1
Content-MD5: ujNHxwCuwZ1mak927GUX3g==
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36
Content-Type: image/jpeg
Origin: http://localhost:3000
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: http://localhost:3000/
Accept-Encoding: gzip, deflate, br
Accept-Language: en
Firefox Request Headers (Works)
Host: sapco.nyc3.digitaloceanspaces.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:75.0) Gecko/20100101 Firefox/75.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: image/png
Content-Length: 254924
Referer: http://localhost:3000/
Content-MD5: z0lzYqq/S1TYxKqL0rJMPw==
Origin: http://localhost:3000
DNT: 1
Connection: keep-alive
Chrome Request Headers (Worked)
Host: sapco.nyc3.digitaloceanspaces.com
Connection: keep-alive
Content-Length: 123803
Pragma: no-cache
Cache-Control: no-cache
DNT: 1
Content-MD5: ujNHxwCuwZ1mak927GUX3g==
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36
Content-Type: image/jpeg
Accept: */*
Origin: http://localhost:3000
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: http://localhost:3000/
Accept-Encoding: gzip, deflate, br
Accept-Language: en
Hard refreshing resolved the issue.
My (working) code looks as follows:
<script type="text/javascript" src="../AAAA/1111.js"></script>
<script type="text/javascript" src="BBBB/2222.js"></script>
The headers I see on the server side are:
GET /AAAA/1111.js HTTP/1.1
Host: localhost:8000
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept: */*
Referer: http://localhost:8000/admin
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: Idea-864a62cc=6c435764-3873-4567-a197-140cd7e7fac1
GET /BBBB/2222.js HTTP/1.1
Host: localhost:8000
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept: */*
Referer: http://localhost:8000/admin
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: Idea-864a62cc=6c435764-3873-4567-a197-140cd7e7fac1
How can I tell the path (on the server side) since they both look the same? One is below the referer and the other is in a sibling directory to the referer, yet I can't tell from the HTTP header. The ".." info seems to be lost yet the server gets it right? How does it know?
Thanks!
Blake McBride
That's because the browser resolves relative references. See Section 5 of RFC 3986 for details.
So no, you can't tell these apart on the server.
Over here I have http GET request.
GET http://www.uw-team.org/hm3next/loguj.php HTTP/1.1
Host: www.uw-team.org
Proxy-Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip, deflate, sdch
Accept-Language: pl-PL,pl;q=0.8,en-US;q=0.6,en;q=0.4
I want to convert this request from GET to POST method and add some parameters in request body. So I changed first line from
GET http://www.uw-team.org/hm3next/loguj.php HTTP/1.1
to
POST http://www.uw-team.org/hm3next/loguj.php HTTP/1.1
and added request body:
...
Accept-Encoding: gzip, deflate, sdch
Accept-Language: pl-PL,pl;q=0.8,en-US;q=0.6,en;q=0.4
param1=val¶m2=val2
What I have to change/add else?
For that request body, add:
Content-Type: application/x-www-form-urlencoded
That is all you need.
See more information about POST method
and application/x-www-form-urlencoded.
We have recently enabled compression in IIS6 for our ASP.NET applications (some 2.0 some 4.0) and have found that GZip is only being performed when the request is made over HTTPS.
This problem is being seen on all servers in our web farm. I have also checked that the same problem exists regardless of the browser used.
I've searched for over an hour and can't find anything on this, so hoping someone may be able to help. Examples follow
HTTP Request:
GET /about.htm HTTP/1.1
Host: www.website.com
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.76 Safari/537.36
Referer: http://www.google.co.uk
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
HTTP Response (not GZipping):
HTTP/1.1 200 OK
Cache-Control: private
Content-Length: 65568
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Date: Thu, 26 Sep 2013 09:26:46 GMT
HTTPS Request:
GET /about.htm HTTP/1.1
Host: www.website.com
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.76 Safari/537.36
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
HTTPS Response (GZip working):
HTTP/1.1 200 OK
Cache-Control: private
Date: Thu, 26 Sep 2013 09:29:39 GMT
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Content-Encoding: gzip
Vary: Accept-Encoding
Transfer-Encoding: chunked
UPDATE
I've run a request trace in IIS - the offending requests have the following line
IISCompression DYNAMIC_COMPRESSION_NOT_SUCCESS "NO_MATCHING_SCHEME"