A user is able to upload a file. During the upload the file is scanned. If there is an issue with the file Symfony returns a Response(400) and the rest of the file is not uploaded, saving the user and the host time and bandwidth.
This is done via \Symfony\Component\HttpFoundation\Request::getContent(true)
$resource = $request->getContent(true);
The file is scanned a line at a time using:
fgets($resource);
The resource is also closed before the response is sent to the user:
fclose($resource);
However there is unexpected and strange behaviour happening for some user clients.
For example wget:
wget -4 --no-check-certificate --method PUT --timeout=0 --header 'Authorization: Bearer xxx' --body-file='xxx' 'https://example.com/xxx' --content-on-error -d -O -
Response hangs:
---request begin---
PUT /xxx HTTP/1.1
User-Agent: Wget/1.20.3 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: xxx
Connection: Keep-Alive
Content-Length: 37767602
Authorization: Bearer xxx
---request end---
[writing BODY file xxx ...
It appears that wget does not understand the upload does not need to be completed, is this a header that php is failing to send or a flag required in the wget command?
A similar command in curl works
curl -k --location --request PUT 'https://example.com/xxx' \
--header 'Authorization: Bearer xxx' \
--data-binary '#/xxx'
Response
< Server: Apache/2.4.38 (Debian)
< Vary: Authorization
< X-Robots-Tag: noindex
< Transfer-Encoding: chunked
* HTTP error before end of send, stop sending
<
* Closing connection 0
* TLSv1.3 (OUT), TLS alert, close notify (256):
Related
What I want to do is, is create special fuzz based request using curl for type 1 and another type 2 request which is based on 4 requests simultaneously and separately using curl, there is a command of perl, I also want that to include in the request
This is what I came up with.
My Curl Code Type 1:
curl -X GET -d '
Content-Length: 22
Foo: HTTP/1.1 200 OK
Content-Length: 0
Content-Type: text/html
Last-Modified: ...
Refresh: 0; URL=http://www.google.com/
Bar:
' -i $Local'
My Curl Code Type 2 for 4 requests:
#!/bin/bash
echo -e "[ + ] Enter Local Server
read Local
curl -X POST -d ' / HTTP/1.1
Host: google.com
Connection: Keep-Alive
Content-Length: 49225
perl -e 'print (A x 49255)
-i '$Local'
curl -X POST -d ' / HTTP/1.1
Host: google.com
Connection: Keep-Alive
Content-Length: 33
-i '$Local'
curl -X POST -d ' / HTTP/1.1
xxxx: POST /loader/cmd.exe?/c+dir HTTP/1.1
-i '$Local'
curl -X POST -d ' / HTTP/1.1
Connection: Keep-Alive
It didn't work, I tried lots of method, I am not familiar with this type of requests. I want to include exactly all this, what you're seeing.
Error was:
./curl.sh: line 2: unexpected EOF while looking for matching
./curl.sh: line 27: syntax error: unexpected end of file
I have a simple file on my web server, and when I request it in a browser, it loads without problems:
http://example.server/report.php
But when I request the file with wget from a Raspberry Pi, I get this:
$ wget -d --spider http://example.server/report.php
Setting --spider (spider) to 1
DEBUG output created by Wget 1.18 on linux-gnueabihf.
Reading HSTS entries from /home/pi/.wget-hsts
URI encoding = 'ANSI_X3.4-1968'
converted 'http://example.server/report.php' (ANSI_X3.4-1968) -> 'http://example.server/report.php' (UTF-8)
Converted file name 'report.php' (UTF-8) -> 'report.php' (ANSI_X3.4-1968)
Spider mode enabled. Check if remote file exists.
--2018-06-03 07:29:29-- http://example.server/report.php
Resolving example.server (example.server)... 49.132.206.71
Caching example.server => 49.132.206.71
Connecting to example.server (example.server)|49.132.206.71|:80... connected.
Created socket 3.
Releasing 0x00832548 (new refcount 1).
---request begin---
HEAD /report.php HTTP/1.1
User-Agent: Wget/1.18 (linux-gnueabihf)
Accept: */*
Accept-Encoding: identity
Host: example.server
Connection: Keep-Alive
---request end---
HTTP request sent, awaiting response...
---response begin---
HTTP/1.1 406 Not Acceptable
Date: Fri, 15 Jun 2018 08:25:17 GMT
Server: Apache
Keep-Alive: timeout=3, max=200
Connection: Keep-Alive
Content-Type: text/html; charset=iso-8859-1
---response end---
406 Not Acceptable
Registered socket 3 for persistent reuse.
URI content encoding = 'iso-8859-1'
Remote file does not exist -- broken link!!!
I read somewhere that it might be an encoding problem, so I tried
$ wget -d --spider --header="Accept-encoding: *" http://example.server/report.php
but that gives me the exact same error.
That's because the server you're connecting to serves only to certain User-Agents.
Change the user agent and it works fine:
wget -d --user-agent="Mozilla/5.0 (Windows NT x.y; rv:10.0) Gecko/20100101 Firefox/10.0" http://example.server/report.php
To test something, I want to run a simple web server that:
Will listen for HTTPS POST requests
Print the POST data received to STDOUT (along with other stuff, potentially, so it's fine if it just cats the whole HTTP request)
Is there a quick way to set something like this up? I've tried using OpenSSL's s_server, but it only seems to want to respond to GET requests.
Since s_server does not support POST requests, you should use socat instead of openssl s_server:
# socat -v OPENSSL-LISTEN:443,cert=mycert.pem,key=key.pem,verify=0,fork 'SYSTEM:/bin/echo HTTP/1.1 200 OK;/bin/echo;/bin/echo this-is-the-content-of-the-http-answer'
Here are essential parameters:
fork: to loop for many requests
-v: to display the POST data (and other stuff) to STDOUT
verify=0: do not ask for mutual authentication
Now, here is an example:
We use the following POST request:
% wget -O - --post-data=abcdef --no-check-certificate https://localhost/
[...]
this-is-the-content-of-the-http-answer
We see the following socat output:
# socat -v OPENSSL-LISTEN:443,cert=mycert.crt,key=key.pem,verify=0,fork 'SYSTEM:/bin/echo HTTP/1.1 200 OK;/bin/echo;/bin/echo this-is-the-content-of-the-http-answer'
> 2017/08/05 03:13:04.346890 length=212 from=0 to=211
POST / HTTP/1.1\r
User-Agent: Wget/1.19.1 (freebsd10.3)\r
Accept: */*\r
Accept-Encoding: identity\r
Host: localhost:443\r
Connection: Keep-Alive\r
Content-Type: application/x-www-form-urlencoded\r
Content-Length: 6\r
\r
< 2017/08/05 03:13:04.350299 length=16 from=0 to=15
HTTP/1.1 200 OK
> 2017/08/05 03:13:04.350516 length=6 from=212 to=217
abcdef< 2017/08/05 03:13:04.351549 length=1 from=16 to=16
< 2017/08/05 03:13:04.353019 length=39 from=17 to=55
this-is-the-content-of-the-http-answer
According to the documentation and some similar questions on SO curl should follow a redirect using GET method, unless --post30x is specified as a parameter. However that's the result of my testing
curl -kvv -b /tmp/tmp.BEo6w3GKDq -c /tmp/tmp.BEo6w3GKDq -X POST -H "Accept: application/json" -L https://localhost/api/v1/resource
> POST /api/v1/resource HTTP/1.1
> User-Agent: curl/7.29.0
> Host: localhost
> Cookie: JSESSIONIDSSO=AB59F2FD09D38EDBAACB726CF212EA2E; JSESSIONID=743FD68B520840094B6D283A81CF3CFA
> Accept: application/json
>
< HTTP/1.1 302 Found
< Server: Apache-Coyote/1.1
< Strict-Transport-Security: max-age=15768000; includeSubDomains
< Cache-control: no-cache, no-store
< Pragma: no-cache
< Location: https://testserver.int/api/v1/resource
< Content-Length: 0
< Date: Fri, 27 Jan 2017 08:41:05 GMT
<
> POST /api/v1/resource HTTP/1.1
> User-Agent: curl/7.29.0
> Host: testserver.int
> Cookie: JSESSIONID=1tcxpkul4qyqh1hycpf9insei9
> Accept: application/json
I would expect the second request to actually be using GET instead of POST.
curl's man page says:
When curl follows a redirect and the request is not a plain GET (for
example POST or PUT), it will do the following request with a GET if
the HTTP response was 301, 302, or 303. If the response code was any
other 3xx code, curl will re-send the following request using the same
unmodified method.
You can tell curl to not change the non-GET request method to GET
after a 30x response by using the dedicated options for that:
--post301, --post302 and --post303.
Unfortunatelly that's not what I'm seeing and there is no option for --get30x.
So my question is - how to make curl follow a redirect response (301/302/303) with a GET request to the Location as it is written in the documentation?
I've tested it with curl/7.29.0 as well as curl/7.50.3.
Problem: You are telling curl to do that with your use of -X POST. As the man page section for -X explains this:
The method string you set with -X, --request will be used for all requests, which
if you for example use -L, --location may cause unintended side-effects when curl
doesn't change request method according to the HTTP 30x response codes - and
similar.
Fix: Remove the -X POST from your command line. Use -d "" instead to send an empty post that will adjust accordingly to the proper method after redirect.
More: Explanation and rant in my blog post unnecessary use of curl -X.
This topic has been up quite some times in the community (forums, blog posts etc) and the conclusion is that this should be done making a REST Post call to share and the url /service/modules/create-site
The reason is that some surf specific stuff like the site dashboard are created from the share side.
However, I have been trying this approach from different angles all day, always ending up with a HTTP 200 in the response and no share site created. Quite frustrating.
I'm running this on Alfresco Enterprise 4.2.3.3 (I suspect my problems is due to a recent change)
To strip this down to something that is easy to reproduce, I'm following Martin Bergljungs blog post on the subject (http://www.ixxus.com/blog201203creating-alfresco-share-sites-javascript/), starting with using curl like this:
create a text file with login credentials (login.txt) with the following content (change to appropriate values):
username=admin&password=admin
create a text file with the json to create a site (site_data.json)
{"visibility" : "PUBLIC","title" : "My Test Site","shortName" : "mytestsite",
"description" : "My Test Site created from command line", "sitePreset" : "site-dashboard"}
Get the JSESSIONID by requesting a ticket:
curl -v -d #login.txt -H "Content-Type:application/x-www-form-urlencoded" http://localhost:8081/share/page/dologin
copy the resulting JSESSIONID value into the following curl call:
curl -v -d #site_data.json -H "Cookie:JSESSIONID=<insert your jsessionid>" -H "Content-Type:application/json" -H "Accept:application/json" http://localhost:8081/share/service/modules/create-site
output from curl:
* Hostname was NOT found in DNS cache
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8081 (#0)
> POST /share/service/modules/create-site HTTP/1.1
> User-Agent: curl/7.35.0
> Host: localhost:8081
> Cookie:JSESSIONID=5963B948684F562A278909AF466D2306
> Content-Type:application/json
> Accept:application/json
> Content-Length: 196
>
* upload completely sent off: 196 out of 196 bytes
< HTTP/1.1 200 OK
* Server Apache-Coyote/1.1 is not blacklisted
< Server: Apache-Coyote/1.1
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Content-Language: en-US
< Content-Length: 0
< Date: Tue, 02 Dec 2014 13:57:02 GMT
<
* Connection #0 to host localhost left intact
The latter curl call results in a HTTP 200 as seen above, but a login to share reveals there have been no site created what so ever :(
BTW. I have disabled the CSRF Token Filter.
UPDATE:
I have verified that the above approach works to create a site on Alfresco Enterprise 4.1.5
I have verified that it also fails on Alfresco Community 4.2.e
This is reported as a bug: https://issues.alfresco.com/jira/browse/MNT-11706
UPDATE: Since the question was not clear to a reader I have reformulated it now
UPDATE:
Following Dave Websters answer, I been trying again using the following steps, still with CSRF Token disabled:
Login:
curl -v -d #login.txt -H "Content-Type:application/x-www-form-urlencoded" http://localhost:8081/share/page/dologin
Response:
POST /share/page/dologin HTTP/1.1
User-Agent: curl/7.35.0
Host: localhost:8081
Accept: /
Content-Type:application/x-www-form-urlencoded
Content-Length: 29
* upload completely sent off: 29 out of 29 bytes
< HTTP/1.1 302 Found
* Server Apache-Coyote/1.1 is not blacklisted
< Server: Apache-Coyote/1.1
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Set-Cookie: JSESSIONID=058A52486E4EB12F94D1F95302732616; Path=/share/; HttpOnly
< Set-Cookie: alfLogin=1417618589; Expires=Wed, 10-Dec-2014 14:56:29 GMT; Path=/share
< Set-Cookie: alfUsername3=admin; Expires=Wed, 10-Dec-2014 14:56:29 GMT; Path=/share
< Location: http://localhost:8081/share
< Content-Length: 0
< Date: Wed, 03 Dec 2014 14:56:29 GMT
Took the cookie values and inserted into Daves code (with the csrf-stuff stripped out):
curl 'http://localhost:8081/share/service/modules/create-site' -H 'Cookie: JSESSIONID=058A52486E4EB12F94D1F95302732616; alfLogin=1417618589; alfUsername3=admin;' -H 'Origin: http://localhost:8081' -H 'Accept-Encoding: gzip,deflate,sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: */*' -H 'Referer: http://localhost:8081/share/page/site/erik/dashboard' -H 'X-Requested-With: application/json' -H 'Connection: keep-alive' --data-binary $'{"visibility":"PUBLIC","title":"erik'","shortName":"erik'","description":"This site is auto generated","sitePreset":"site-dashboard"}' --compressed
Still no share site generated though, and still a HTTP 200 Response. No errors in the logs either. This is driving me nuts :(
New Update (It works!):
I have now found out that you will need to "touch" a share webscript after making the login call before calling create-site with a post. I do this by making a get request in between. This somehow needs to be done to initialize the share session.
This is the curl command I use to generate sites programatically. I insert the JSESSIONID, LOGINCOOKIECONTENTS and CSRFTOKEN (twice) contents manually, but getting them programatically should work.
curl 'http://localhost:8081/share/service/modules/create-site' -H 'Cookie: JSESSIONID={JSESSIONID}; alfLogin={LOGINCOOKIECONTENTS}; alfUsername3=admin; Alfresco-CSRFToken={CSRFTOKEN};' -H 'Origin: http://localhost:8081' -H 'Accept-Encoding: gzip,deflate,sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.107 Safari/537.36' -H 'Content-Type: application/json' -H 'Accept: */*' -H 'Referer: http://localhost:8081/share/page/site/auto-gen-0/dashboard' -H 'X-Requested-With: application/json' -H 'Connection: keep-alive' -H 'Alfresco-CSRFToken: {CSRFTOKEN}' --data-binary $'{"visibility":"PUBLIC","title":"auto-gen'$I'","shortName":"auto-gen-'$I'","description":"This site is auto generated","sitePreset":"site-dashboard"}' --compressed
The expected response is:
{
"success": true
}