I run a webstore on Magento ver. 2.1.8 eCommerce platform.
Recently, I came across a issue where front-end of my website was using a completely different colour skin and layout and couldn't figure out why this is happening.
Purging the Varnish cache solved the issue for some time as the website would randomly changed the skin again after some time. I had no idea how Varnish was caching the content that should never be displayed.
I also use Acunetix as a Server Side Scanner which scans my website every week for any malicious code or gaps in security. I found one of the POST request that it sent:
POST /themesettings/index/paneltool/ HTTP/1.1
Content-Length: 1193
Content-Type: multipart/form-data; boundary=-----Boundary_JTWCAHJSKP
Referer: https://www.domain.co.uk/
Cookie: PHPSESSID=sadfq345r234324dfasd; mage-messages= vespaneltool=a%3A6%3A%7Bs%3A52%3A%22ves_themesettings_general%2Fgeneral_settings%2Fdirection%22%3Bs%3A3%3A%22rtl%22%3Bs%3A49%3A%22ves_themesettings_general%2Fgeneral_settings%2Flayout%22%3Bs%3A8%3A%22boxed-lg%22%3Bs%3A52%3A%22ves_themesettings_general%2Fgeneral_settings%2Fmax_width%22%3Bs%3A5%3A%22960px%22%3Bs%3A59%3A%22ves_themesettings_general%2Fgeneral_settings%2Fmax_width_custom%22%3Bs%3A8%3A%22gqwtkdks%22%3Bs%3A47%3A%22ves_themesettings_general%2Fgeneral_settings%2Fskin%22%3Bs%3A8%3A%22blue.css%22%3Bs%3A55%3A%22ves_themesettings_header%2Fgeneral_settings%2Fheader_layout%22%3Bs%3A13%3A%22default.phtml%22%3B%7D; _vwo_uuid_v2=EC7CC959823F97596222AB508A6BB8BE|53a815cb661ea346311131469aaeb1c2; PHPSESSID=oaibesqi4980brc3udl1gdrfb0
Host: www.domain.co.uk
Connection: Keep-alive
Accept-Encoding: gzip,deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21
Accept: */*
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="btn-save"
Apply
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/direction]"
rtl
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/layout]"
boxed-md
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/max_width]"
1024px
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/max_width_custom]"
12345'"\'\");|]*%00{%0d%0a<%00>%bf%27'ð©
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/skin]"
aquamarine.css
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_header/general_settings/header_layout]"
default2.phtml
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="vespanel"
1
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="vesreset"
0
-------Boundary_JTWCAHJSKP--
The above is the configuration of the wrong designed that was cached by Varnish. E.g. aquamarine.css is the wrong CSS file cached, it should be red.css.
Is it possible that after this post request, Varnish cached this settings? Also, what is the best way to overcome this problem? Should I create a rule in Acunetix to avoid this URL? Or create a NginX configuration to block access to it?
Try something like the following to just force Varnish to not attempt to use (or store into) the cache for POST requests:
sub vcl_recv {
if (req.method == 'POST') {
set req.hash_always_miss = true;
}
}
Related
If the post is using "multipart/form-data" content-type, and each part could be a file or other content type.
If I want to use GZIP, should the GZIP apply to the entire post body of all parts altogether or could it be possible to choose some file use gzip content-encoding while some file don't.
Is there any standard for it or only common practice ?
Thanks
For example, can I add Content-Encoding:gzip at below 'file1' part
Host: localhost:8081
Connection: keep-alive
Content-Length: 317
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36
Cache-Control: no-cache
Origin: chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop
Postman-Token: 7143164d-0da5-0e1d-112e-91f2a21c22c2
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryPZAv0gGlJrA4ABu2
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7
------WebKitFormBoundaryPZAv0gGlJrA4ABu2
Content-Disposition: form-data; name="key1"
value1
------WebKitFormBoundaryPZAv0gGlJrA4ABu2
Content-Disposition: form-data; name="file1"; filename="sample_file.txt"
Content-Type: text/plain
Content-Encoding: gzip ---------------------------IS IT OK TO ADD GZIP HERE?
This is a sample file content!
------WebKitFormBoundaryPZAv0gGlJrA4ABu2--
If you specify "Content-Encoding: gzip" in the HTTP response header section, it will apply to the full response.
I don't believe there's a way to apply compression to individual parts.
Either apply Content-Encoding compression to the whole message (end-to-end), or have the payload compressed on the transport layer, Transfer-Encoding (potentially hop-by-hop). What you can not do is compression on each part.
The multipart/form-data media type does not support any MIME header
fields in parts other than Content-Type, Content-Disposition, and (in
limited circumstances) Content-Transfer-Encoding. Other header
fields MUST NOT be included and MUST be ignored.
Emphasis mine. Source: https://www.rfc-editor.org/rfc/rfc7578#section-4.8
I would like to make a file download resumable using byte-range requests.
The problem is that my existing download action is responding on a POST method and I would like to keep it that way.
But it seems from my early tests that Chrome turns interrupted POST requests for file downloads into GET requests when the user tries to resume and thus the resuming of the download fails.
Am I missing something?
Is this something related to the HTTP specs that only allow GET requests to be resumed?
Or is it simply a design flaw in Chrome (and maybe other browsers as well) that makes it forget the original HTTP method used?
UPDATE:
Here are the request/response data:
Initial POST request:
POST http://localhost:35547/Download?Guid=396b4697-e275-4396-818c-548bf8c0a281 HTTP/1.1
Host: localhost:35547
Connection: keep-alive
Content-Length: 0
Cache-Control: max-age=0
Origin: http://localhost:35547
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36
Content-Type: application/x-www-form-urlencoded
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Referer: http://localhost:35547/File/396b4697-e275-4396-818c-548bf8c0a281
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
Cookie: __RequestVerificationToken=LuPgM05MHrsuyskgfhsrHVUs; ASP.NET_SessionId=gfiulghfuygisghkf; .ASPXAUTH=FGDJHGDHSDFB15AFDE6371CGJHDFGFBHD; fileDownload=true
Initial response (to the request above):
HTTP/1.1 200 OK
Cache-Control: private, s-maxage=0
Content-Type: application/zip
Server: Microsoft-IIS/7.5
X-AspNetMvc-Version: 5.2
Content-Disposition: attachment; filename="FILE-396b4697e2754396818c548bf8c0a281.zip"
X-AspNet-Version: 4.0.30319
Set-Cookie: fileDownload=true; path=/
X-Powered-By: ASP.NET
Date: Wed, 09 Nov 2016 11:13:50 GMT
Content-Length: 1885473
PK.......... ZIP file data .............................................
After the interruption, this is the request that the browser does on resume (notice the GET method used):
GET http://localhost:35547/Download?Guid=396b4697-e275-4396-818c-548bf8c0a281 HTTP/1.1
Host: localhost:35547
Connection: keep-alive
Referer: http://localhost:35547/File/396b4697-e275-4396-818c-548bf8c0a281
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36
Accept-Encoding: gzip, deflate, sdch, br
Accept-Language: en-US,en;q=0.8
Cookie: __RequestVerificationToken=.............
(Some data from security-related cookies have been shortened and altered)
Am I missing something ?
It depends, upon how have you analyzed the behavior of google chrome. Ideal way will be use any proxy or to use packet sniffer such as Wireshark to see what request method, is used by chrome in subsequent request.
Is this something related to the HTTP specs that only allow GET
requests to be resumed?
As of now, there's no mention in spec of HTTP protocol, that only GET requests can be resumed.
Or is it simply a design flaw in Chrome (and maybe other browsers as
well) that makes it forget the original HTTP method used?
Yes, It's the flaw of google chrome. Make sure that you check it on the latest version of Google chrome with all the update patches. Also check it on other browsers.
For more info about HTTP protocol, refer to https://www.ietf.org/rfc/rfc2616.txt.
Refer to following request, for serving partial response : https://en.wikipedia.org/wiki/Byte_serving
Edit
For more updated info regarding HTTP info, refer to :-
https://www.rfc-editor.org/rfc/rfc7230
I'm having a problem setting a content-type of application/json header on my post request.
saveUpdates(alltabs: AllTabs): Observable<Response> {
let api = this.host + this.routes.save;
let headers = new Headers();
headers.append('Content-Type', 'application/json');
return this._http.post(api, JSON.stringify(alltabs), { headers: headers })
.map((response: Response) => <Response>response.json())
.do(data => console.log("saveUpdates(): " + data))
.catch(this.handleError);
}
Request Headers:
OPTIONS /api/productsave HTTP/1.1
Host: wbtest:92
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache
Access-Control-Request-Method: POST
Origin: http://localhost:3000
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36
Access-Control-Request-Headers: content-type
Accept: */*
Referer: http://localhost:3000/product/60000010080
Accept-Encoding: gzip, deflate, sdch
Accept-Language: en-US,en;q=0.8
Response Headers:
HTTP/1.1 405 Method Not Allowed
Cache-Control: no-cache
Pragma: no-cache
Allow: POST
Content-Type: application/json; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Access-Control-Allow-Origin: *
Date: Tue, 14 Jun 2016 15:16:15 GMT
Content-Length: 76
As you can see, my request has two unexpected headers added "Access-Control-Request-Headers" and "Access-Control-Request-Method". This seems to suggest an issue with CORS (Cross-Origin Resource Sharing). However, the web.conf file on the API server has been working and the response headers states "Access-Control-Allow-Origin: *".
Any idea what could be wrong?
UPDATE:
The above code is correct - the problem is with the Sever code not being configured to handle preflight requests. In my case, the .NET Web API 2 application was not configured to allow CORS.
With CORS, you have two kinds of requests. As a matter of fact, the CORS specification distinguishes two distinct use cases:
Simple requests. This use case applies if we use HTTP GET, HEAD and POST methods. In the case of POST methods, only content types with the following values are supported: text/plain, application/x-www-form-urlencoded and multipart/form-data.
Preflighted requests. When the "simple requests" use case doesn't apply, a first request (with the HTTP OPTIONS method) is made to check what can be done in the context of cross-domain requests.
It seems that your server isn't configured to support preflighted request. The reason for the 405 status code (405 Method Not Allowed).
See this article for more details:
http://restlet.com/blog/2015/12/15/understanding-and-using-cors/
I am attempting to post to a page served by IIS6 and I am adding the following header:
Transfer-Encoding: chunked
When I do this, I get a 404 error: The system cannot find the file specified.. If I make the same request to the page without this header, it definitely exists.
Here are the headers - the only difference between the two requests is the chunking...
POST http://hostname/appname/index.html HTTP/1.1
Host: hostname
Connection: keep-alive
Transfer-Encoding: chunked
Cache-Control: max-age=0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Content-Type: application/x-www-form-urlencoded
Referer: http://referrer/
And the body:
4
xxxx
0
The IIS metabase has AspEnableChunkedEncoding set to true at the W3SVC level and it is not overridden at the site level.
Why do I get a 404 error?
Update: I notice on the response for the unsuccessful request that the:
x-powered-by: asp.net
...header is absent. I don't know if that helps - it is there if I don't set the header in the request.
Is URLScan installed? It could be blocking the header
This question already has answers here:
How does HTTP file upload work?
(5 answers)
Closed 8 years ago.
I'm working on an iPhone app that makes a multipart HTTP request with multiple image files.
It looks like what's happening, on the server side, is that one of the images is getting parsed properly, but the other two files are not.
Can anybody post a sample HTTP multipart request that contains multiple image files?
Well, note that the request contains binary data, so I'm not posting the request as such - instead, I've converted every non-printable-ascii character into a dot (".").
POST /cgi-bin/qtest HTTP/1.1
Host: aram
User-Agent: Mozilla/5.0 Gecko/2009042316 Firefox/3.0.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://aram/~martind/banner.htm
Content-Type: multipart/form-data; boundary=2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Length: 514
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile1"; filename="r.gif"
Content-Type: image/gif
GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile2"; filename="g.gif"
Content-Type: image/gif
GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile3"; filename="b.gif"
Content-Type: image/gif
GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f--
Note that every line (including the last one) is terminated by a \r\n sequence.
EDIT: I am maintaining a similar, but more in-depth answer at: https://stackoverflow.com/a/28380690/895245
To see exactly what is happening, use nc -l and a user agent like a browser or cURL.
Save the form to an .html file:
<form action="http://localhost:8000" method="post" enctype="multipart/form-data">
<p><input type="text" name="text" value="text default">
<p><input type="file" name="file1">
<p><input type="file" name="file2">
<p><button type="submit">Submit</button>
</form>
Create files to upload:
echo 'Content of a.txt.' > a.txt
echo '<!DOCTYPE html><title>Content of a.html.</title>' > a.html
Run:
nc -l localhost 8000
Open the HTML on your browser, select the files and click on submit and check the terminal.
nc prints the request received. Firefox sent:
POST / HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:29.0) Gecko/20100101 Firefox/29.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: __atuvc=34%7C7; permanent=0; _gitlab_session=226ad8a0be43681acf38c2fab9497240; __profilin=p%3Dt; request_method=GET
Connection: keep-alive
Content-Type: multipart/form-data; boundary=---------------------------9051914041544843365972754266
Content-Length: 554
-----------------------------9051914041544843365972754266
Content-Disposition: form-data; name="text"
text default
-----------------------------9051914041544843365972754266
Content-Disposition: form-data; name="file1"; filename="a.txt"
Content-Type: text/plain
Content of a.txt.
-----------------------------9051914041544843365972754266
Content-Disposition: form-data; name="file2"; filename="a.html"
Content-Type: text/html
<!DOCTYPE html><title>Content of a.html.</title>
-----------------------------9051914041544843365972754266--
Aternativelly, cURL should send the same POST request as your a browser form:
nc -l localhost 8000
curl -F "text=default" -F "file1=#a.txt" -F "file2=#a.html" localhost:8000
You can do multiple tests with:
while true; do printf '' | nc -l localhost 8000; done