Given the below header:
GET /resources/css/jquery-ui.css HTTP/1.1
Host: site.com
Connection: keep-alive
Cache-Control: no-cache
Pragma: no-cache
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1
Accept: text/css,*/*;q=0.1
Referer: http://site.com/
What part of the header gets parsed by rewrite engines? (I mostly concern about Nginx)
The GET part, that is, /resources/css/jquery-ui.css in your example.
Related
I'm trying to upload a file but I get an error message like the following:
https://sapco.nyc3.digitaloceanspaces.com/b3vcchphzgj5m8p6ld51yk7867uu?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=QTHCA5KKQUHAKMMATAIP%2F20200503%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20200503T214158Z&X-Amz-Expires=10800&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=7b77a0f1551a262586980b709fdc44a2bc173ab6ae7279385e831493b1d13e53
<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code>BadDigest</Code>
<BucketName>sapco</BucketName>
<RequestId>tx000000000000013462bc9-005eaf36fb-3518e03-nyc3a</RequestId>
<HostId>3518e03-nyc3a-nyc</HostId>
</Error>
How can I debug this further?
So far I have tried:
Resetting my Access and Secret keys.
At first I thought this was related to PWA-related work, but I migrated to an earlier branch and have the same issues.
Tried different files: each have the same error.
Happens both in prod. and locally.
Update 1: This randomly started working again on production. However it's still broken locally.
With the same file I have the 2 paths for the PUT request that leads to the error above.
https://sapco.nyc3.digitaloceanspaces.com/11eego5a6r9b4tslx7cex4p9x45u?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=QTHCA5KKQUHAKMMATAIP%2F20200504%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20200504T005319Z&X-Amz-Expires=10800&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=8d2037f7370eb137facc9d813fe35ed34e055313af06cd66819a72d886dfb018
https://sapco.nyc3.digitaloceanspaces.com/z4vc7ujtvid0akqfn4uou46407zl?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=QTHCA5KKQUHAKMMATAIP%2F20200504%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20200504T005405Z&X-Amz-Expires=10800&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=b4b28cebe56a9b6c12ddfb2cc335b84080a3bfc5e34e2c66e19001230f8b7512
The issue must be related to the way X-Amz-Signature is computed.
Digging further with bundle open activestorage I can see it's roughly here. https://cutt.ly/6yjc7u1
I verified the Content-Length and Content-MD5 are both the same (vs local and prod). (123803 and ujNHxwCuwZ1mak927GUX3g== respectively).
Update 2: I tried this in Firefox with the same image and no problem locally. There must be something fishy going on with the cache. I then tried an Incognito window and that also seemed to work. Finally, I did a hard refresh in Chrome and now I've unblocked myself. Didn't quite figure out what was going on but leaving a final piece of information for anyone else:
Chrome Request Headers (Not Working, 400 Error)
PUT /lw5lufemkgb7ww83pdc56qg2gb0j?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=QTHCA5KKQUHAKMMATAIP%2F20200504%2Fnyc3%2Fs3%2Faws4_request&X-Amz-Date=20200504T013900Z&X-Amz-Expires=10800&X-Amz-SignedHeaders=content-md5%3Bcontent-type%3Bhost&X-Amz-Signature=1ea88bf8550d9bab67b5bca3aa97f7b15f1a44e117dd4f5cea0744c898f70684 HTTP/1.1
Host: sapco.nyc3.digitaloceanspaces.com
Connection: keep-alive
Content-Length: 0
Pragma: no-cache
Cache-Control: no-cache
Accept: */*
DNT: 1
Content-MD5: ujNHxwCuwZ1mak927GUX3g==
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36
Content-Type: image/jpeg
Origin: http://localhost:3000
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: http://localhost:3000/
Accept-Encoding: gzip, deflate, br
Accept-Language: en
Firefox Request Headers (Works)
Host: sapco.nyc3.digitaloceanspaces.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:75.0) Gecko/20100101 Firefox/75.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: image/png
Content-Length: 254924
Referer: http://localhost:3000/
Content-MD5: z0lzYqq/S1TYxKqL0rJMPw==
Origin: http://localhost:3000
DNT: 1
Connection: keep-alive
Chrome Request Headers (Worked)
Host: sapco.nyc3.digitaloceanspaces.com
Connection: keep-alive
Content-Length: 123803
Pragma: no-cache
Cache-Control: no-cache
DNT: 1
Content-MD5: ujNHxwCuwZ1mak927GUX3g==
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36
Content-Type: image/jpeg
Accept: */*
Origin: http://localhost:3000
Sec-Fetch-Site: cross-site
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: http://localhost:3000/
Accept-Encoding: gzip, deflate, br
Accept-Language: en
Hard refreshing resolved the issue.
My (working) code looks as follows:
<script type="text/javascript" src="../AAAA/1111.js"></script>
<script type="text/javascript" src="BBBB/2222.js"></script>
The headers I see on the server side are:
GET /AAAA/1111.js HTTP/1.1
Host: localhost:8000
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept: */*
Referer: http://localhost:8000/admin
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: Idea-864a62cc=6c435764-3873-4567-a197-140cd7e7fac1
GET /BBBB/2222.js HTTP/1.1
Host: localhost:8000
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept: */*
Referer: http://localhost:8000/admin
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9
Cookie: Idea-864a62cc=6c435764-3873-4567-a197-140cd7e7fac1
How can I tell the path (on the server side) since they both look the same? One is below the referer and the other is in a sibling directory to the referer, yet I can't tell from the HTTP header. The ".." info seems to be lost yet the server gets it right? How does it know?
Thanks!
Blake McBride
That's because the browser resolves relative references. See Section 5 of RFC 3986 for details.
So no, you can't tell these apart on the server.
To send an HTTP request through a socket to a server, i would do something like this:
GET / HTTP/1.0
Host: www.example.com
User-agent: SomeBot
...
How would you go about programmatically defining an HTTPS request? I'm not looking for any programming language specific answer, something that teaches me the essence of HTTPS.
My research:
When i goto https://www.google.co.in/webhp?tab=ww&ei=sg7MUvKgGoX_rAeKxoGIDg&ved=0CBQQ1S4 this is what i get:
GET https://www.google.co.in/webhp?tab=ww&ei=sg7MUvKgGoX_rAeKxoGIDg&ved=0CBQQ1S4 HTTP/1.1
:host: www.google.co.in
x-chrome-variations: COy1yQEIlLbJAQiftskBCKS2yQEIqLbJAQiptskBCL62yQEI8YPKAQ==
accept-encoding: gzip,deflate,sdch
accept-language: en-US,en;q=0.8
user-agent: Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36
:path: /webhp?tab=ww&ei=sg7MUvKgGoX_rAeKxoGIDg&ved=0CBQQ1S4
accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
:version: HTTP/1.1
cache-control: max-age=0
cookie: PREF=ID=c032cbb31701d0d8:U=3a8fed312bb2ee57:FF=0:LD=en:TM=1374381891:LM=1376055657:S=BooLSkeTxOsbOYls; NID=67=HDIT9zwo-KKhljgRnJMz4u_5L_qpj3FvsN9Y47dWZmByQRS4N8QYs64IcEjFYphs6YpbrmvgsejwaL5YwxzbkY_qYaKU7wBfDA9N955NznF7IIyeHxcQ5UX8Dm999AElAKdkyswNbwUx1WJZo5vEuIaqC4Hdw4AkjsdwmFjY4ujPiEAj72z93QpCLleM-NXOK8N5YWn8DqiteGrEZUQ3FdPK3vkfDet_GF3CcBnkiYWxXON6R8Kum8BWaJGtm9h5dA; SID=DQAAANQAAAB2hOHWGXo76aWm_lgruhW0NH_zbU26rK7YMM_uiyMRvIBoyiEb3Gn_j2AhtmM4v6a74DinFMAOIjq5N4g4JcAAXaMEXz1dUz8MVup_nt1udNM0hpvybeWPxE1xK8rvdL2ra9moRW58jRzzA0HdpmkrH_t2ZIQ7GhqJlxp6lOS_jfvmeeb3REYFp6Q08hRYvCRDmhYFQ7NSt_Ua_3EWu4d_o125kvZ0x0bwm7JDKEcO3S-b6SJ4KnAGIWYjQKPdirgIFEUm1vApvIr4hoa4Z01rBt9YTmhwdEG5KvJmjusPkQ; HSID=AoEbPqSO97tEXhBOd; SSID=Abkp9uP00vi4wX19_; APISID=idPvNkfOQ-W9vefw/AjRgJIuDHZMDnME-B; SAPISID=5UH5pOlPn4c_31En/AcwfUulAqos_McwmH
:scheme: https
referer: https://www.google.co.in/
:method: GET
I see that it is pretty understandable, yet slightly alien...
I made an HTML page with some JavaScript that works fine on my laptop on Linux, but fails when I upload it on a Windows server. I think it is a header issue so I am not writing the Ajax code, but the request headers :
Ajax Request :
GET /index-1.html HTTP/1.1
Host: mydomain.com
Connection: keep-alive
Accept: undefined
X-Requested-With: XMLHttpRequest
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1553.0 Safari/537.36 SUSE/30.0.1553.0
Referer: http://mydomain.com/
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Server Answer :
HTTP/1.1 406 Not Acceptable
Content-Type: text/html
Server: Microsoft-IIS/7.0
X-Powered-By: ASP.NET
Date: Thu, 19 Sep 2013 19:40:44 GMT
Content-Length: 1346
What should I set the headers to?
If you actually read the description of status code 406 you'll quickly find out that the problem is likely caused by "accept: undefined".
Despite what the title may suggest, this is not related to the common "preventing double post request" issue.
In my application, I add some data on outgoing HTTP traffic, and with some some websites, I randomly encounter an HTTP POST request that has a double-header, resulting in a server termination, as I assume the server doesn't understand the request and decides to cut me off. As the title says, I'm literally seeing two POST headers in a single request. This only happens when I append some custom HTTP fields to the header. For example, I came across this today when I followed a surveygizmo.com link, as seen in the trace at the bottom of this post.
I cannot understand if it's the browser that's doing something funky because it noticed I've modified some data, or it's something in my LSP application that causes this to happen.
When I debug my application, I only see the intercepted request the first time, which is when I inject the custom data. After that, I don't see the request anywhere except in Wireshark, so it's not like I can remediate the double headers by deleting the redundant data.
Things to note looking at the trace:
The data I'm appending is 'Custom-FieldN:'
Two almost-identical headers
Three double-CRLF's in one single request header (how is that possible?)
The Request:
POST http://www.surveygizmo.com/s3/1212345/Who-Are-You HTTP/1.1
Host: www.surveygizmo.com
Custom-Field1: UserNameBob
Custom-Field2: 2578291789
proxy-connection: keep-alive
Content-Length: 836
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://www.surveygizmo.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryaQraA7ZABICMT6jO
Referer: http://www.surveygizmo.com/s3/1212345/Who-Are-You
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6,ja;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: sg-response-979523-1212345=0%3B1369154430_519ba37e477bc8.35524744%3B1369154430%3BPartial
POST http://www.surveygizmo.com/s3/1212345/Who-Are-You HTTP/1.1
Host: www.surveygizmo.com
Custom-Field1: UserNameBob
Custom-Field2: 2578291789
proxy-connection: keep-alive
Content-Length: 836
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Origin: http://www.surveygizmo.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryaQraA7ZABICMT6jO
Referer: http://www.surveygizmo.com/s3/1212345/Who-Are-You
accept-encoding: gzip,deflate
Accept-Language: en-GB,en;q=0.8,en-US;q=0.6,ja;q=0.4
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3
Cookie: sg-response-979523-1212345=0%3B1369154430_519ba37e477bc8.35524744%3B1369154430%3BPartial
------WebKitFormBoundaryaQraA7ZABICMT6jO
Content-Disposition: form-data; name="sg_navchoice"