I am trying to send a file using HTTP from a C++ application (no HTML-boxes). The server keeps answering Code 400/ Bad Request.
To keep it simple, I have changed manually the content of the file to a simple string (later on, I will need to upload real binary files).
the POST request is the following:
POST /post.php HTTP/1.0
Host: posttestserver.com
Accept: */*
Content-Type: multipart/form-data; boundary=BOUNDARY
--BOUNDARY
Content-Disposition: form-data; name="userfile"; filename="example.txt"
Content-Type:text/plain
123ABC
--BOUNDARY--
Connection: close
Any idea what is going on?
Related
I need to interact with a remote HTTP server at the lowest possible level (i.e.: at socket level) because my target is a very small embedded system with no support for higher level libraries (it's a bare-metal uController wit no O.S. at all and talking to a GSM modem via serial line; modem has some support for sockets, but nothing above that).
Basic need is to upload a "file" using POST.
I have all needed Header/Body in place and it "usually works".
Problem is I randomly get a "HTTP/1.1 502 Bad gateway error" response and this is more likely to happen as the size of "file" increases.
I understand this means there's some problem between the reverse proxy frontend (nginx, apparently) and the backends, but I have absolutely no control on those (actually I dont't really know how the atual setup besides what can be gleamed from (light) probing).
My current strategy is to open a plain socket and send the folowing sequence (dots represent binary data):
POST /path/to/websend.php HTTP/1.0
Host: host.domain.tld
User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:33.0) Gecko/20100101 Firefox/33.0
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Type: multipart/form-data; boundary=AaB03x
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Accept: */*
Content-Length: <full_length>
--AaB03x
Content-Disposition: form-data; name="IV"
Content-Type: application/data
Content-Transfer-Encoding: binary
000102030405060708090A0B0C0D0E0F
--AaB03x
Content-Disposition: form-data; name="S_TXT_FILE"; filename="FILENAME_s.txt"
Content-Type: application/data
Content-Transfer-Encoding: binary
..............................................................
..............................................................
...... several 512byte blocks ................................
..............................................................
..............................................................
--AaB03x--
Is there something I could do to enhance reliability?
I already do multiple retries and this actually works, but sometimes I need to retry six or more times to have a positive answer (200 OK).
Note I send exactly the same sequence on rety and it succeeds... eventually.
I need to send two parts because content is encrypted and first part is the neded "Initialization Vector".
I am trying to send a multipart/form-data request to a POST API using Nifi's InvokeHTTP processor. This request takes a json and a file. The request headers and request body in POSTMAN look something like this -
POST /delivery/deliverPackage
User-Agent: PostmanRuntime/7.6.1
Accept: */*
Host: example.hostname:port
accept-encoding: gzip, deflate
content-type: multipart/form-data; boundary=--------------------------161413078116998145311888
content-length: 1115
json={ "destinationProtocol" : "HL7", "destinationFormat": "HL7_V2_ORU", "destinationType": "example", "destinationConnectionParams":{ "URI": "example", "HOST": "example", "PORT": "example" } }file=[object Object]
where the file object contains the file details I am trying to send.
I want to send this multipart/form-data request in nifi. Based on an answer I saw on one of the forums (sorry do not have the link to it), I am trying to create this request body in the content of the flowfile using ReplaceText processor before sending the flowfile to an InvokeHttp processor. The flowfile content looks something like this -
POST /delivery/deliverPackage
User-Agent: curl/7.46.0
Accept: */*
Host: example.hostname:port
accept-encoding: gzip, deflate
content-type: multipart/form-data; boundary=--------------------------161413078116998145311888
content-length: 1115
--------------------------161413078116998145311888
Content-Disposition: form-data; json="{ "destinationProtocol" : "HL7", "destinationFormat": "HL7_V2_ORU", "destinationType": "example", "destinationConnectionParams":{ "URI": "example", "HOST": "example", "PORT": "example" } }"
anonymous
--------------------------161413078116998145311888
Content-Disposition: form-data; name="file"; filename="/path/to/file/in/localsystem.HL7”
Content-Type: text/plain
contents of the file
--------------------------161413078116998145311888--
This doesn't seem to be working though, it doesn't look right to me. I am pretty new to Nifi. Can anyone help me understand what I'm doing wrong or give some insights on how to handle this properly? Thanks!
I have instead tried to use an ExecuteStreamCommand processor to simply run the curl command with command arguments -
-X POST;"https://example.hostname:port/delivery/deliverPackage?json=%7B%20%22destinationProtocol%22%20%3A%20%22HL7%22%2C%20%22destinationFormat%22%3A%20%22HL7_V2_ORU%22%2C%20%22destinationType%22%3A%20%22example%22%2C%20%22destinationConnectionParams%22%3A%7B%20%22URI%22%3A%20%22example%3A%2F%2Fexample%3A15050%22%2C%20%22HOST%22%3A%20%22example%22%2C%20%22PORT%22%3A%20%22example%22%20%7D%20%7D";-H "Content-Type: multipart/form-data";-F "file=#/path/to/file/in/localsystem.HL7";
This works, but I wanted to know how to do it using the InvokeHttp processor. Any help is greatly appreciated! Thank you.
You can do this by using GenerateFlowFile --> InvokeHTTP.
The GenerateFlowFile would create the payload(in your case the content of localsystem.HL7) of your post in the Custom Text filed.
The InvokeHTTP will need to have Content-Type as ${mime.type} - default value and
Then see what the output is after your POST
I have a program that activate an chip for racing results. (Its just a piece of hardware).
I watch with Fiddler (Sniffing program) the in and outgoing traffic from my pc when I connect the chip with my computer.
The program sends the following HTTP Request:
POST http://example.com/index.php HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
Content-Length: 185
Content-Type: application/x-www-form-urlencoded
Host: example.com
Pragma: no-cache
User-Agent: SomeProgram 1.2.3
Data==%0D%0AAjlFNEEw-SOMELONGSECRETKEY-RGAw%3D%3D%0D%0A
I receive the following response:
<?xml version="1.0" encoding="utf-8"?>
<message type="3" result="1" txid="someid" activationdate="" availablecredits="732" firstname="John" lastname="Doe" email="JohnDoe#outlook.com" phonenumber="00123445" notification_email="1" notification_text="1"/>
Is it possible to edit the response so that when the programs check for the availablecredits variable, he get the value 9999 instead of 732.
Im working on a Windows 8 laptop.
Definitely - Fiddler allows you to modify requests and responses by adding rules to FiddlerScript. Citing Fiddler documentation:
To make custom changes to web requests and responses, use
FiddlerScript to add rules to Fiddler's OnBeforeRequest or
OnBeforeResponse function. Which function is appropriate depends on
the objects your code uses: OnBeforeRequest is called before each
request, and OnBeforeResponse is called before each response.
So, all you have to do is to add to OnBeforeResponse the logic for replacing the availablecredits attribute value with any value you desire.
I am attempting to upload a file to Google Drive using the "upload" URL with a type of "multipart". I'm trying to do this without a library and using basic HTTP with a multipart POST. With a body like the following, I am constantly getting the error "Invalid multipart request with 0 mime parts."
The HTTP message looks valid to me. Is there something obvious that I'm missing or doing wrong?
Is there a protocol tester that can verify if my POST body is valid or not?
POST /upload/drive/v2/files?uploadType=multipart HTTP/1.1
Authentiction: Bearer {valid auth_token}
Content-Type: multipart/mixed; boundary="--314159265358979323846"
host: localhost:3004
content-length: 254
Connection: keep-alive
--314159265358979323846
Content-Type: application/json
{"title":"Now","mimeType":"text/plain"}
--314159265358979323846
Content-Type: text/plain
Content-Transfer-Encoding: 8bit
Mon Jun 17 2013 20:59:02 GMT-0400 (EDT)
--314159265358979323846--
(The segments look like they have double newlines. I think this is an artifact of the pasting, they are CRLF pairs in the code and appear as a newline when testing, but I guess this could theoretically be the problem, but I'd like proof.)
boundary attribute on the Content-Type header should not include double dashes. Use the following as your Content-Type:
Content-Type: multipart/mixed; boundary="314159265358979323846"
I can upload a file to my Apache web server using Curl just fine:
echo "[$(date)] file contents." | curl -T - http://WEB-SERVER/upload/sample.put
However, if I put a Squid proxy server in between, then I am not able to:
echo "[$(date)] file contents." | curl -x http://SQUID-PROXY:3128 -T - http://WEB-SERVER/upload/sample.put
Curl reports the following error:
Note: This error response was in HTML format, but I've removed the tags for ease of reading.
ERROR: The requested URL could not be retrieved
ERROR
The requested URL could not be retrieved
While trying to retrieve the URL:
http://WEB-SERVER/upload/sample.put
The following error was encountered:
Unsupported Request Method and Protocol
Squid does not support all request methods for all access protocols.
For example, you can not POST a Gopher request.
Your cache administrator is root.
My squid.conf doesn't seem to be having any ACL/rule that should disallow based on the src or dst IP addresses, or the protocol, or the HTTP method... as I can do an HTTP POST just fine between the same client and the web server, with the same proxy sitting in between.
In case of the failing HTTP PUT case, to see the request and response traffic that was actually occurring, I placed a netcat process in between Curl and Squid, and this is what I saw:
Request:
PUT http://WEB-SERVER/upload/sample.put HTTP/1.1
User-Agent: curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
Host: WEB-SERVER
Pragma: no-cache
Accept: */*
Proxy-Connection: Keep-Alive
Transfer-Encoding: chunked
Expect: 100-continue
Response:
HTTP/1.0 501 Not Implemented
Server: squid/2.6.STABLE21
Date: Sun, 13 May 2012 02:11:39 GMT
Content-Type: text/html
Content-Length: 1078
Expires: Sun, 13 May 2012 02:11:39 GMT
X-Squid-Error: ERR_UNSUP_REQ 0
X-Cache: MISS from SQUID-PROXY-FQDN
X-Cache-Lookup: NONE from SQUID-PROXY-FQDN:3128
Via: 1.0 SQUID-PROXY-FQDN:3128 (squid/2.6.STABLE21)
Proxy-Connection: close
<SNIPPED the HTML error response already shown earlier above>
Note: I have anonymized the IP addresses and server names throughout for readability reasons.
Thanks to Amos Jeffries for answering this on squid-users forum. The issue is basically that Squid before version 3.1 does not implement HTTP 1.1 and thus rejects the chunked transfer encoding.