Correct HTTP response header - http

I am using Safari to test my C++ socket handler, I send a request to the application, in the address bar:
http://127.0.0.1:8124/?{%22module%22:%22mdFileIO%22,%22command%22:%22open%22}
In my application I send a response:
const QString CRLF("\r\n");
QString strContent(strResponse)
,strDtNow(QDateTime::currentDateTime().toUTC().toString("ddd, dd MMM yyyy hh:mm:ss"))
,strHdr = "200 OK" + CRLF
+ "Content-Type: application/json" + CRLF
+ "Content-Length: " + QString::number(strContent.length()) + CRLF
+ "Date: " + strDtNow + " GMT" + CRLF + CRLF;
strResponse = strHdr + strContent;
In the above 'strContent' contains:
{"ack":"ack","module":"mdFileIO","time":"2020-10-05 18:00:19"}
The output for the response looks like this:
200 OK\r\nContent-Type: application/json\r\nContent-Length: 62\r\nDate: Mon, 05 Oct 2020 18:25:59 GMT\r\n\r\n{\"ack\":\"ack\",\"module\":\"mdFileIO\",\"time\":\"2020-10-05 18:25:59\"}
The return from write() is 161. Safari shows the following:

A couple of errors in the header:
The header should have started with "HTTP/1.1 "
The Content-Type should have been: "application/jsonrequest"

Related

Send data from Pico W to InfluxDB via urequest

I want to save the wind measurement i record with my raspberry pico w in the InfluxDB, that is hosted on my Raspberry Pi 4b with http-requests. For that i am using the urequest-library
def send_data_to_influxdb(speed):
data = '"Windmeasure,location=outdoor windspeed=' + str(speed) + ' $(date +%s%N)"'
url = 'http://192.168.178.46:8086/write?db=iobroker'
username = 'user_iobroker'
password = '{pwd}'
url = 'http://' + username + ':' + password + '#192.168.178.46:8086/write?db=iobroker'
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
try:
response = urequests.post(url, headers=headers, data=data.encode())
except Exception as e:
print("Error sending data to InfluxDB: ", e)
else:
if response.status_code != 204:
print("Error sending data to InfluxDB. Response code: ", response.status_code)
else:
print("Data sent successfully to InfluxDB.")
response.close()
I get the following error:
Error sending data to InfluxDB: invalid syntax for integer with base 10
How does it work correctly?

Translate a working curl request into HTTP

I'm using curl to perform a POST request, but I can't assume my target platform to have curl available, so I'm trying to rewrite my curl request in HTTP (which is guaranteed to be available). My knowledge of both curl and HTTP is very limited, so I'm hoping someone can point out what I'm doing wrong.
My curl request (command line):
curl.exe POST https://xxxxxx.ingest.sentry.io/api/xxxxxxx/minidump/?sentry_key=xxxxxxxxxxxxxxxxxxxxxxx -F upload_file_minidump=#"C:\path\Minidump.dmp" -F upload_file_log=#"C:\path\program.log"
A relevant part of curl's output is shown below. This is after connecting to the server and sending it the POST request. The server now lets the client know the first file can be sent, and curl responds first by sending the file's own header, and then the data (clamped here)
<= Recv header, 23 bytes (0x17)
0000: HTTP/1.1 100 Continue
=> Send data, 175 bytes (0xaf)
0000: --------------------------f2a4a742c08bf427
002c: Content-Disposition: form-data; name="upload_file_minidump"; fil
006c: ename="UE4Minidump.dmp"
0085: Content-Type: application/octet-stream
00ad:
=> Send data, 16384 bytes (0x4000)
0000: MDMP..a..... .......m/S`.........................;..............
0040: 8Z......T...=...........`.......8...........T................[..
0080: .........\...........]..........= ..............................
00c0: ....................................aJ.......`......Lw..........
0100: ............T........?..i/S`........ ... ... ............ ......
0140: ............G.M.T. .S.t.a.n.d.a.r.d. .T.i.m.e...................
0180: ................................G.M.T. .D.a.y.l.i.g.h.t. .T.i.m.
01c0: e...................................................1.9.0.4.1...
..etc..
By reading the verbose output of curl, I've created a HTTP request looking like this (c++ code using unreal engine 4 libraries):
TSharedRef<IHttpRequest, ESPMode::ThreadSafe> httpRequest = FHttpModule::Get().CreateRequest();
httpRequest->SetURL(TEXT("https://xxxxxx.ingest.sentry.io/api/xxxxxx/minidump/?sentry_key=xxxxxxxxxxxxxxxxxxxxxxxx"));
httpRequest->SetVerb(TEXT("POST"));
const FString boundary(TEXT("------------------------f2a4a742c08bf427"));
httpRequest->SetHeader(TEXT("Content-Type"), TEXT("multipart/form-data; boundary=") + boundary);
const FString fileName(FPaths::Combine(path, crashToReport.folderName, TEXT("UE4Minidump.dmp")));
ensure(FPaths::FileExists(fileName));
const FString prefixBoundary(TEXT("\r\n--") + boundary + TEXT("\r\n"));
const FString fileHeader(TEXT("Content-Disposition: form-data; name=\"upload_file_minidump\"; filename=\"UE4Minidump.dmp\"\r\nContent-Type: application/octet-stream\r\n\r\n"));
FString fileContents;
FFileHelper::LoadFileToString(fileContents, *fileName);
const FString suffixBoundary(TEXT("\r\n--") + boundary + TEXT("--\r\n"));
const FString content(prefixBoundary + fileHeader + fileContents + suffixBoundary);
httpRequest->SetContentAsString(content);
This works to a degree, the server now accepts this, and will receive the file - however the file ends up being unreadable server-side, leading me to think I'm not sending it in the right format.
What kind of data is expected in a multipart/form-data request?
A thing I notice is that the curl request sends the file's header separately (the first chunk of 175 bytes). I would love some information on how to achieve that!
I finally figured it out. I'm not sure how to report what I was doing wrong, but I think it had to do with what happens under the hood in:
httpRequest->SetContentAsString(..)
, which reliably caused the backend to fail to interpret the binary file I was trying to send. I ended up reading the binary file .. as a binary file:
TArray<uint8> dumpFileData;
FFileHelper::LoadFileToArray(dumpFileData, *FPaths::Combine(path,crashToReport.folderName, TEXT("UE4Minidump.dmp")));
Then send it via the POST request, similarly as before, but adding the complete form data part as binary data:
TSharedRef<IHttpRequest, ESPMode::ThreadSafe> httpRequest = FHttpModule::Get().CreateRequest();
httpRequest->SetURL(TEXT("https://xxxxx.ingest.sentry.io/api/xxxxxx/minidump/?sentry_key=xxxxxxxxxxxxxxxxxxxxxxxxxxx"));
httpRequest->SetVerb(TEXT("POST"));
const FString boundary(TEXT("------------------------bb33b671b1212234"));
httpRequest->SetHeader(TEXT("Content-Type"), TEXT("multipart/form-data; boundary=") + boundary);
httpRequest->SetHeader(TEXT("Accept"), TEXT("*/*"));
httpRequest->SetHeader(TEXT("Expect"), TEXT("100-continue"));
{
const FString prefixBoundary(TEXT("--") + boundary + TEXT("\r\n"));
const FString fileHeader(TEXT("Content-Disposition: form-data; name=\"upload_file_minidump\"; filename=\"UE4Minidump.dmp\"\nContent-Type: application/octet-stream\r\n\r\n"));
const FString suffixBoundary(TEXT("\r\n--") + boundary + TEXT("--\r\n"));
TArray<uint8> CombinedContent;
CombinedContent.Append(FStringToUint8(prefixBoundary + fileHeader));
CombinedContent.Append(dumpFileData);
CombinedContent.Append(FStringToUint8(suffixBoundary));
httpRequest->SetContent(CombinedContent);
}
httpRequest->ProcessRequest();
For completeness, FStringToUint8 is defined as follows:
// Convert FString to UTF8 and put it in a TArray
TArray<uint8> FStringToUint8(const FString& InString)
{
TArray<uint8> OutBytes;
// Handle empty strings
if (InString.Len() > 0)
{
FTCHARToUTF8 Converted(*InString); // Convert to UTF8
OutBytes.Append(reinterpret_cast<const uint8*>(Converted.Get()), Converted.Length());
}
return OutBytes;
}

How can I write HTTP POST request multipart/form-data myself?

I want to send a POST request to server in multipart/form-data contents.
I have to write the request statement myself, without any high-level API
because it has to be sent as an AT command of the modem.
The curl command below is a command to test the server's API, and this is what I want to construct myself.
# This is what I want to construct myself.
curl -X POST $API_ENDPOINT \
-F time=yyyy-mm-dd-hh:mm:ss \
-F event=1 \
-F rssi=31 \
-F battery=80 \
-F filename=test.jpg \
-F files=#test.jpg \ # Send an image from my local file system to server.
-w %{http_code}
echo "\r\n"
And the below is what I sent to my modem.
/* This is form for multipart/form-data. Is it right?
POST $API_ENDPOINT HTTP/1.1
Host: $HOST
Content-Length: $body_length
Content-Type: multipart/form-data; boundary="boundary"
--boundary
Content-Disposition: form-data; name="files"; filename="example.jpg"
Content-Type: image/jpg
--boundary--
*/
// Below C++ code was written for the above POST request.
std::string filename = "example.jpg";
std::string filePath = "example.jpg";
std::string body =
std::string("--boundary\r\n") +
"Content-Disposition: form-data; name=\"files\"; filename=\"" + filePath + "\"\r\n" +
"Content-Type: image/jpg\r\n" +
"\r\n" +
"--boundary--\r\n";
std::string header =
std::string("POST ") + url + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Content-Length: " + std::to_string(body.length()) + "\r\n" +
"Content-Type: multipart/form-data; boundary=\"boundary\"\r\n" +
"\r\n";
std::string data = header + body;
... and this is the server log.
Error: MultipartParser.end(): stream ended unexpectedly: state = PART_DATA
This is my question.
How can I add fields(time, event, rssi, etc) in the request body? The server seems getting each fields as <key: value> format. How can I add it in multipart/form-data?
Is it correct to attach an image file like that? In the curl command, I wrote the path of my local image file (-F files) and the name that this image file will be stored on the server (-F filename). How should I reflect this in my request form?
Thanks brothers
I did it! Right that.
/*
POST $API_ENDPOINT HTTP/1.1
Host: $HOST
Content-Length: $body_length
Content-Type: multipart/form-data; boundary="boundary"
--boundary
Content-Type: text/plain
Content-Disposition: form-data; name="time"
yyyy-mm-dd-hh:mm:ss
--boundary
Content-Type: text/plain
Content-Disposition: form-data; name="event"
1
--boundary
Content-Type: text/plain
Content-Disposition: form-data; name="rssi"
31
--boundary
Content-Type: text/plain
Content-Disposition: form-data; name="battery"
99
--boundary
Content-Type: text/plain
Content-Disposition: form-data; name="filename"
example.jpg
--boundary
Content-Type: image/jpg
Content-Disposition: form-data; name="files"; filename="example.jpg"
....binary bytes of example.jpg...
--boundary--
*/
const int fd = // File descriptor of my serial port
const std::string host = // host
const std::string uri = // uri
const int tryout = 10 // for modem
std::string filename = "1996-03-05.jpg";
std::string filePath = "1996-03-05.jpg";
std::ifstream bin(filePath, std::ios::binary);
std::string imageBin((std::istreambuf_iterator<char>(bin)), std::istreambuf_iterator<char>());
std::string TIMESTAMP = "1996-03-05";
std::string event = "1";
std::string rssi = "31";
std::string battery = "90";
atcmd::__sendATcmd(fd, "AT+QHTTPCFG=\"contextid\",1\r");
atcmd::__readBufferUntil(fd, "\r\nOK\r\n", tryout);
atcmd::__sendATcmd(fd, "AT+QHTTPCFG=\"contenttype\",3\r"); // 3: multipart/form-data
atcmd::__readBufferUntil(fd, "\r\nOK\r\n", tryout);
atcmd::__sendATcmd(fd, "AT+QHTTPCFG=\"requestheader\",1\r");
atcmd::__readBufferUntil(fd, "\r\nOK\r\n", tryout);
const std::string fullUrl = "http://" + host + uri;
atcmd::__sendATcmd(fd, ("AT+QHTTPURL=" + std::to_string(fullUrl.length()) + "\r").c_str());
atcmd::__readBufferUntil(fd, "\r\nCONNECT\r\n", tryout);
atcmd::__sendATcmd(fd, fullUrl.c_str());
atcmd::__readBufferUntil(fd, "\r\nOK\r\n", tryout);
std::string body_fields = (
std::string("--boundary\r\n") +
"Content-Type: text/plain\r\n" +
"Content-Disposition: form/data; name=\"time\"\r\n" +
"\r\n" +
TIMESTAMP + "\r\n" +
"--boundary\r\n" +
"Content-Type: text/plain\r\n" +
"Content-Disposition: form/data; name=\"event\"\r\n" +
"\r\n" +
event + "\r\n" +
"--boundary\r\n" +
"Content-Type: text/plain\r\n" +
"Content-Disposition: form/data; name=\"rssi\"\r\n" +
"\r\n" +
rssi + "\r\n" +
"--boundary\r\n" +
"Content-Type: text/plain\r\n" +
"Content-Disposition: form/data; name=\"battery\"\r\n" +
"\r\n" +
battery + "\r\n" +
"--boundary\r\n" +
"Content-Type: text/plain\r\n" +
"Content-Disposition: form/data; name=\"filename\"\r\n" +
"\r\n" +
filename + "\r\n"
);
std::string body_image = (
std::string("--boundary\r\n") +
"Content-Type: image/jpeg\r\n" +
"Content-Disposition: form-data; name=\"files\"; filename=\"" + filePath + "\"\r\n" +
"\r\n" +
imageBin + "\r\n" +
"--boundary--\r\n"
);
int bodyLen = body_fields.length() + body_image.length();
std::string header = (
std::string("POST ") + uri + " HTTP/1.1\r\n" +
"Host: " + host + "\r\n" +
"Content-Length: " + std::to_string(bodyLen) + "\r\n" +
"Content-Type: multipart/form-data; boundary=\"boundary\"\r\n" +
"\r\n"
);
const std::string dataLen = std::to_string(header.length() + bodyLen);
std::string maxInputBodyTime = "80";
std::string maxResponseTime = "80";
atcmd::__sendATcmd(fd, ("AT+QHTTPPOST="
+ dataLen + ","
+ maxInputBodyTime + ","
+ maxResponseTime + "\r").c_str());
atcmd::__readBufferUntil(fd, "\r\nCONNECT\r\n", tryout);
atcmd::__sendATcmd(fd, header.c_str());
atcmd::__sendATcmd(fd, body_fields.c_str());
atcmd::__sendATcmd(fd, body_image.c_str(), body_image.length());
atcmd::__readBufferUntil(fd, "\r\nOK\r\n", tryout);
atcmd::__sendATcmd(fd, "AT+QHTTPREAD=80\r");
std::cout << atcmd::__readBuffer(fd) << std::endl;

Need some assistance with CyberSource authentication please

I am trying to post a request for "flex/v1/keys" api and i am trying to mirror the examples CyberSource have on the api page. But i keep getting back {"response":{"rmsg":"Authentication Failed"}} .
Please can someone assist me i am clueless and been struggling for a while now.
Code for getting headers sorted.
var crypto = require("crypto-js");
var requestHost = 'apitest.cybersource.com';
var merchantId = 'testrest';
var merchantKeyId = '08c94330-f618-42a3-b09d-e1e43be5efda';
var merchantSecretKey = 'yBJxy6LjM2TmcPGu+GaJrHtkke25fPpUX+UY6/L/1tE=';
var resource = "/flex/v1/keys";
//digest
var payload = '{"encryptionType": "None"}';
var data = crypto.enc.Utf8.parse(payload)
var hash = crypto.SHA256(data)
var base64 = crypto.enc.Base64.stringify(hash);
var digest = "SHA-256=" + base64;
pm.globals.set("digest",digest);
//date
var date = new Date(Date.now()).toUTCString();
pm.globals.set("date",date);
//signature
var signatureHeader = "";
signatureHeader += "keyid=\"" + merchantKeyId + "\"";
signatureHeader += ", algorithm=\"HmacSHA256\"";
var headersForPostMethod = "host date (request-target) digest v-c-merchant-id";
signatureHeader += ", headers=\"" + headersForPostMethod + "\"";
var signatureString = 'host: ' + requestHost;
signatureString += '\ndate: ' + new Date(Date.now()).toUTCString();
signatureString += '\n(request-target): ';
var targetUrlForPost = "post " + resource;
signatureString += targetUrlForPost + '\n';
signatureString += 'digest: SHA-256=' + digest + '\n';
signatureString += 'v-c-merchant-id: ' + merchantId;
var dataSigString = crypto.enc.Utf8.parse(signatureString);
var secKey = crypto.enc.Base64.parse(merchantSecretKey);
var hashHmac = CryptoJS.HmacSHA256(dataSigString, secKey)
var base64hashHmac = CryptoJS.enc.Base64.stringify(hashHmac);
signatureHeader += ", signature=\"" + base64hashHmac + "\"";
pm.globals.set("signature",signatureHeader);
My post request i am trying to send.
curl --location --request POST 'https://apitest.cybersource.com/flex/v1/keys' \
--header 'digest: SHA-256=yF79QR9XHmXEMjhnXRIvsaGie/xoTduWMP8kMOUIyVc=' \
--header 'v-c-merchant-id: testrest' \
--header 'date: Fri, 29 May 2020 15:06:42 GMT' \
--header 'host: apitest.cybersource.com' \
--header 'signature: keyid="08c94330-f618-42a3-b09d-e1e43be5efda", algorithm="HmacSHA256", headers="host date (request-target) digest v-c-merchant-id", signature="lnv5/zeUimcef0Dr3VeOyKgOw/cX8Erdb+qaKuSwuug="' \
--header 'profile-id: 93B32398-AD51-4CC2-A682-EA3E93614EB1' \
--header 'Content-Type: application/json' \
--data-raw '{"encryptionType": "None"}'
RESULTS
POST https://apitest.cybersource.com/flex/v1/keys
Request Headers
digest: SHA-256=yF79QR9XHmXEMjhnXRIvsaGie/xoTduWMP8kMOUIyVc=
v-c-merchant-id: testrest
date: Fri, 29 May 2020 14:57:17 GMT
host: apitest.cybersource.com
signature: keyid="08c94330-f618-42a3-b09d-e1e43be5efda", algorithm="HmacSHA256", headers="host date (request-target) digest v-c-merchant-id", signature="cLFxiYvra8KMBOaTB25Ke+gnQh67/MMn9wr0d8PRSm4="
profile-id: 93B32398-AD51-4CC2-A682-EA3E93614EB1
Content-Type: application/json
User-Agent: PostmanRuntime/7.25.0
Accept: */*
Cache-Control: no-cache
Postman-Token: ef33044d-e745-4014-bdd8-d30f659be760
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Length: 27
Request Body
{"encryptionType": "None"}↵
Response Headers
Strict-Transport-Security: max-age=31536000
v-c-correlation-id: 9e9d8293-b909-4fdf-9ae6-9f992e5dca87
content-type: application/json
v-c-response-time: 1590764238
content-length: 45
Response Body
{"response":{"rmsg":"Authentication Failed"}}
for anyone encountering this issue i have found my fix and it works now with crypto-js authentication.
Silly mistake from me not building the string up properly.
just update the script mentioned in my question with the following.
BROKEN
signatureString += 'digest: SHA-256=' + digest + '\n';
FIX
signatureString += 'digest: ' + digest + '\n';

Split TCP packet into smaller ones in AutoIT

I try to send a simple HTTP GET Request using AutoIT TCPSend() command. The problem is that when I check the traffic using SmartSniff, everything is on the same line without linebreaks.
My code is as follows:
TCPStartup()
$ip = "195.143.118.23"
$port = "80"
Global $tcp = TCPConnect($ip, $port)
TCPSend($tcp, "GET HTTP/1.1")
TCPSend($tcp, "Host: ")
TCPSend($tcp, "Connection: keep-alive")
TCPSend($tcp, "Accept: ")
TCPSend($tcp, "User-Agent: ")
TCPSend($tcp, "Referer: ")
TCPSend($tcp, "Accept-Encoding: ")
TCPSend($tcp, "Accept-Language: ")
TCPSend($tcp, "Cookie: ")
TCPSend($tcp, "Connection: keep-alive")
TCPCloseSocket($tcp)
Use #CRLF at the end of every string
Like
TCPSend($tcp,"Hello Line"&#CRLF)
It would also be better to send it once as one big string and not line by line
And don't close the socket.
If you want to get a respone fron the server, you have to use the function TCPRecv. After this you can close the socket.

Resources