This question already has answers here:
How does HTTP file upload work?
(5 answers)
Closed 8 years ago.
I'm working on an iPhone app that makes a multipart HTTP request with multiple image files.
It looks like what's happening, on the server side, is that one of the images is getting parsed properly, but the other two files are not.
Can anybody post a sample HTTP multipart request that contains multiple image files?
Well, note that the request contains binary data, so I'm not posting the request as such - instead, I've converted every non-printable-ascii character into a dot (".").
POST /cgi-bin/qtest HTTP/1.1
Host: aram
User-Agent: Mozilla/5.0 Gecko/2009042316 Firefox/3.0.10
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://aram/~martind/banner.htm
Content-Type: multipart/form-data; boundary=2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Length: 514
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile1"; filename="r.gif"
Content-Type: image/gif
GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile2"; filename="g.gif"
Content-Type: image/gif
GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f
Content-Disposition: form-data; name="datafile3"; filename="b.gif"
Content-Type: image/gif
GIF87a.............,...........D..;
--2a8ae6ad-f4ad-4d9a-a92c-6d217011fe0f--
Note that every line (including the last one) is terminated by a \r\n sequence.
EDIT: I am maintaining a similar, but more in-depth answer at: https://stackoverflow.com/a/28380690/895245
To see exactly what is happening, use nc -l and a user agent like a browser or cURL.
Save the form to an .html file:
<form action="http://localhost:8000" method="post" enctype="multipart/form-data">
<p><input type="text" name="text" value="text default">
<p><input type="file" name="file1">
<p><input type="file" name="file2">
<p><button type="submit">Submit</button>
</form>
Create files to upload:
echo 'Content of a.txt.' > a.txt
echo '<!DOCTYPE html><title>Content of a.html.</title>' > a.html
Run:
nc -l localhost 8000
Open the HTML on your browser, select the files and click on submit and check the terminal.
nc prints the request received. Firefox sent:
POST / HTTP/1.1
Host: localhost:8000
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:29.0) Gecko/20100101 Firefox/29.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: __atuvc=34%7C7; permanent=0; _gitlab_session=226ad8a0be43681acf38c2fab9497240; __profilin=p%3Dt; request_method=GET
Connection: keep-alive
Content-Type: multipart/form-data; boundary=---------------------------9051914041544843365972754266
Content-Length: 554
-----------------------------9051914041544843365972754266
Content-Disposition: form-data; name="text"
text default
-----------------------------9051914041544843365972754266
Content-Disposition: form-data; name="file1"; filename="a.txt"
Content-Type: text/plain
Content of a.txt.
-----------------------------9051914041544843365972754266
Content-Disposition: form-data; name="file2"; filename="a.html"
Content-Type: text/html
<!DOCTYPE html><title>Content of a.html.</title>
-----------------------------9051914041544843365972754266--
Aternativelly, cURL should send the same POST request as your a browser form:
nc -l localhost 8000
curl -F "text=default" -F "file1=#a.txt" -F "file2=#a.html" localhost:8000
You can do multiple tests with:
while true; do printf '' | nc -l localhost 8000; done
Related
I run a webstore on Magento ver. 2.1.8 eCommerce platform.
Recently, I came across a issue where front-end of my website was using a completely different colour skin and layout and couldn't figure out why this is happening.
Purging the Varnish cache solved the issue for some time as the website would randomly changed the skin again after some time. I had no idea how Varnish was caching the content that should never be displayed.
I also use Acunetix as a Server Side Scanner which scans my website every week for any malicious code or gaps in security. I found one of the POST request that it sent:
POST /themesettings/index/paneltool/ HTTP/1.1
Content-Length: 1193
Content-Type: multipart/form-data; boundary=-----Boundary_JTWCAHJSKP
Referer: https://www.domain.co.uk/
Cookie: PHPSESSID=sadfq345r234324dfasd; mage-messages= vespaneltool=a%3A6%3A%7Bs%3A52%3A%22ves_themesettings_general%2Fgeneral_settings%2Fdirection%22%3Bs%3A3%3A%22rtl%22%3Bs%3A49%3A%22ves_themesettings_general%2Fgeneral_settings%2Flayout%22%3Bs%3A8%3A%22boxed-lg%22%3Bs%3A52%3A%22ves_themesettings_general%2Fgeneral_settings%2Fmax_width%22%3Bs%3A5%3A%22960px%22%3Bs%3A59%3A%22ves_themesettings_general%2Fgeneral_settings%2Fmax_width_custom%22%3Bs%3A8%3A%22gqwtkdks%22%3Bs%3A47%3A%22ves_themesettings_general%2Fgeneral_settings%2Fskin%22%3Bs%3A8%3A%22blue.css%22%3Bs%3A55%3A%22ves_themesettings_header%2Fgeneral_settings%2Fheader_layout%22%3Bs%3A13%3A%22default.phtml%22%3B%7D; _vwo_uuid_v2=EC7CC959823F97596222AB508A6BB8BE|53a815cb661ea346311131469aaeb1c2; PHPSESSID=oaibesqi4980brc3udl1gdrfb0
Host: www.domain.co.uk
Connection: Keep-alive
Accept-Encoding: gzip,deflate
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.21 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.21
Accept: */*
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="btn-save"
Apply
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/direction]"
rtl
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/layout]"
boxed-md
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/max_width]"
1024px
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/max_width_custom]"
12345'"\'\");|]*%00{%0d%0a<%00>%bf%27'ð©
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_general/general_settings/skin]"
aquamarine.css
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="userparams[ves_themesettings_header/general_settings/header_layout]"
default2.phtml
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="vespanel"
1
-------Boundary_JTWCAHJSKP
Content-Disposition: form-data; name="vesreset"
0
-------Boundary_JTWCAHJSKP--
The above is the configuration of the wrong designed that was cached by Varnish. E.g. aquamarine.css is the wrong CSS file cached, it should be red.css.
Is it possible that after this post request, Varnish cached this settings? Also, what is the best way to overcome this problem? Should I create a rule in Acunetix to avoid this URL? Or create a NginX configuration to block access to it?
Try something like the following to just force Varnish to not attempt to use (or store into) the cache for POST requests:
sub vcl_recv {
if (req.method == 'POST') {
set req.hash_always_miss = true;
}
}
If the post is using "multipart/form-data" content-type, and each part could be a file or other content type.
If I want to use GZIP, should the GZIP apply to the entire post body of all parts altogether or could it be possible to choose some file use gzip content-encoding while some file don't.
Is there any standard for it or only common practice ?
Thanks
For example, can I add Content-Encoding:gzip at below 'file1' part
Host: localhost:8081
Connection: keep-alive
Content-Length: 317
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36
Cache-Control: no-cache
Origin: chrome-extension://fhbjgbiflinjbdggehcddcbncdddomop
Postman-Token: 7143164d-0da5-0e1d-112e-91f2a21c22c2
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryPZAv0gGlJrA4ABu2
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7
------WebKitFormBoundaryPZAv0gGlJrA4ABu2
Content-Disposition: form-data; name="key1"
value1
------WebKitFormBoundaryPZAv0gGlJrA4ABu2
Content-Disposition: form-data; name="file1"; filename="sample_file.txt"
Content-Type: text/plain
Content-Encoding: gzip ---------------------------IS IT OK TO ADD GZIP HERE?
This is a sample file content!
------WebKitFormBoundaryPZAv0gGlJrA4ABu2--
If you specify "Content-Encoding: gzip" in the HTTP response header section, it will apply to the full response.
I don't believe there's a way to apply compression to individual parts.
Either apply Content-Encoding compression to the whole message (end-to-end), or have the payload compressed on the transport layer, Transfer-Encoding (potentially hop-by-hop). What you can not do is compression on each part.
The multipart/form-data media type does not support any MIME header
fields in parts other than Content-Type, Content-Disposition, and (in
limited circumstances) Content-Transfer-Encoding. Other header
fields MUST NOT be included and MUST be ignored.
Emphasis mine. Source: https://www.rfc-editor.org/rfc/rfc7578#section-4.8
I am trying to upload data to a webserver using curl. I have analyzed the data sent to the server, from a browser, using wireshark, shown below:
POST /cgi-bin/upload.cgi HTTP/1.1
Host: 192.168.1.22
Connection: keep-alive
Content-Length: 2637
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Origin: http://192.168.1.22
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/30.0.1599.101 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundaryn1sckmiVOVfCwUMQ
Referer: http://192.168.1.22/upload.html
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
I looked at the webpage the browser accesses to upload the information, shown below:
<form method="POST" enctype="multipart/form-data" action="http://192.168.1.22/cgi-bin/upload.cgi">
File to upload:
<input type="file" name="theupload"><br>
Should it be updated?<input type="checkbox" name="configuration"><br>
<input type="submit" value="Press"> to upload the file!
</form>
I attempted to use:
curl --referer http://192.168.1.22/upload.html -F theupload=#file1.xml -F config=on http://192.168.1.22:80/cgi-bin/upload.cgi
My output was
POST /cgi-bin/upload.cgi HTTP/1.1
User-Agent: curl/7.33.0
Host: 192.168.1.22
Accept: */*
Referer: http://192.168.1.22/upload.html
Content-Length: 2650
Expect: 100-continue
Content-Type: multipart/form-data; boundary=------------------------fd126312049d1f47
The server returns HTTP/1.1 417 Expectation Failed.
I am just learning curl, and I am unsure of what to do. Do i need to emulate the header of the browser command as closely as possible? or is the boundary what is probably throwing it off?
Thanks for you help.
try to use curl -H "Expect:" ...
I'm sending an http request to a server that requests an image but using range header with several ranges, the request text is:
GET /images/nav_logo102.png HTTP/1.1
Host: www.google.com.eg
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept-Language: en-us,en;q=0.5
Cookie: PREF=ID=8aacc11c670a5a37:U=cba349de64cbf880:FF=0:LD=en:TM=1310392358:LM=1327944471:S=zc8_vfTdF5U3C-XN; NID=56=15iHWUMVXBGpdEkEcMCeu32GUL6GlK3aEB5vmocRT4kdILhwGpe9mn6DqDrnZBYWJzA2g4YzfXP8IP3tH7Hw4CzC6FwDrV3uqgv3XhCqmrWI0TBD52Vs3nbdth5YzkOR; SID=DQAAAMUAAAC5feWAhXsCT9NT6ObU32dOLBTU_KtRzxc7Ug6QWIhQW_bpXNvzQtoddtt-a2BbUefv89ZjwNwgrgCiCse3INYkeCyfR7PVPaoDPEmUWQ_0sHXBgbf5U0JBg0XxT8KZMmY9kLOZfEKGg0UGtfNNk8uJKWwPXWlkCYjGalQFyVinUNwiYqayYHGvnA0vtiftBCgqnpHawkyVWoo9hCk3vfDKrRHSdO-GQZoxbL21VlH2dbsJJrnJBaVlR-y8KTg14eLgYpt2swZJxcVZwHThhaF-; HSID=AJLTtk1qkIE6L4aLe; GZ=Z=1
User-Agent: Mozilla/5.0 (Windows NT 6.0; rv:5.0) Gecko/20100101 Firefox/5.0
Range: bytes=2048-4095,4096-6143,6144-8191,8192-10239,10240-12287,12288-14335,14336-16383,16384-18431,18432-20479,20480-22527,22528-24575,24576-26623,26624-26722
but the server is responding with
HTTP/1.1 200 OK
and sending the whole image
if i try the same thing but with only one range, it works and i get only the requested range.
any idea how the request should be made?
You should add \r\n\r\n after last header.
REQUEST:
GET /images/djsBox_2.jpg HTTP/1.1\r\n
Range: bytes=0-2,10-20\r\n
Host: www.djsoft.net\r\n
Accept: */*\r\n
\r\n
I checked it with WFetch tool, and the response is HTTP/1.1 206 Partial Content\r\n
And the server actually returned 2 parts of data:
Accept-Ranges: bytes\r\n
Content-Length: 196\r\n
Content-Type: multipart/byteranges; boundary=4b9e42cd78baa9\r\n
\r\n
\r\n
--4b9e42cd78baa9\r\n
Content-type: image/jpeg\r\n
Content-range: bytes 0-2/4543\r\n
\r\n
яШя\r\n
--4b9e42cd78baa9\r\n
Content-type: image/jpeg\r\n
Content-range: bytes 10-20/4543\r\n
\r\n
\x000\x001\x001\x001\x000`\x000`\x000\x000я\r\n
--4b9e42cd78baa9--\r\n
I have used the Java Sun Wireless Toolkit 2.5.2 to generate method stubs for my SOAP webservice. I have been using this for a while and it works great. I now need to add support for the French language. When I send up a string like 'pièce' as one of the properties on an object, it turns into 'pi??ce' by the time it is read by the server. Obviously, the encoding is off somewhere, but I can't see anywhere that I can change the way the generated stub encodes the parameters passed to the web service. Any suggestions?
Update: Here is the header and the start of the xml sent from the BlackBerry:
POST /website/service.asmx HTTP/1.1
Connection: close
Via: MDS_4.1.5.26
Content-Length: 2257
Content-Type: text/xml
Content-Language: en-US
Accept: */*
Host: host
User-Agent: RIM JSR172/1.0
SOAPAction: "http://www.test.com/Test"
<?xml version="1.0" encoding="utf-8"?>
Here is the header and the start of the xml sent from the iPhone (which works):
POST /website/service.asmx HTTP/1.1
Connection: keep-alive
Content-Length: 2359
Content-Type: application/soap+xml; charset=utf-8
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: fr-fr
Host: host
User-Agent: wsdl2objc
Soapaction: http://www.test.com/Test
<?xml version="1.0"?>
There is a Unicode BOM? I'd check the packet sniffer. As it may not be the content-type but rather the bytes themselves.