Translate a working curl request into HTTP - http

Question

I'm using curl to perform a POST request, but I can't assume my target platform to have curl available, so I'm trying to rewrite my curl request in HTTP (which is guaranteed to be available). My knowledge of both curl and HTTP is very limited, so I'm hoping someone can point out what I'm doing wrong.
My curl request (command line):
curl.exe POST https://xxxxxx.ingest.sentry.io/api/xxxxxxx/minidump/?sentry_key=xxxxxxxxxxxxxxxxxxxxxxx -F upload_file_minidump=#"C:\path\Minidump.dmp" -F upload_file_log=#"C:\path\program.log"
A relevant part of curl's output is shown below. This is after connecting to the server and sending it the POST request. The server now lets the client know the first file can be sent, and curl responds first by sending the file's own header, and then the data (clamped here)
<= Recv header, 23 bytes (0x17)
0000: HTTP/1.1 100 Continue
=> Send data, 175 bytes (0xaf)
0000: --------------------------f2a4a742c08bf427
002c: Content-Disposition: form-data; name="upload_file_minidump"; fil
006c: ename="UE4Minidump.dmp"
0085: Content-Type: application/octet-stream
00ad:
=> Send data, 16384 bytes (0x4000)
0000: MDMP..a..... .......m/S`.........................;..............
0040: 8Z......T...=...........`.......8...........T................[..
0080: .........\...........]..........= ..............................
00c0: ....................................aJ.......`......Lw..........
0100: ............T........?..i/S`........ ... ... ............ ......
0140: ............G.M.T. .S.t.a.n.d.a.r.d. .T.i.m.e...................
0180: ................................G.M.T. .D.a.y.l.i.g.h.t. .T.i.m.
01c0: e...................................................1.9.0.4.1...
..etc..
By reading the verbose output of curl, I've created a HTTP request looking like this (c++ code using unreal engine 4 libraries):
TSharedRef<IHttpRequest, ESPMode::ThreadSafe> httpRequest = FHttpModule::Get().CreateRequest();
httpRequest->SetURL(TEXT("https://xxxxxx.ingest.sentry.io/api/xxxxxx/minidump/?sentry_key=xxxxxxxxxxxxxxxxxxxxxxxx"));
httpRequest->SetVerb(TEXT("POST"));
const FString boundary(TEXT("------------------------f2a4a742c08bf427"));
httpRequest->SetHeader(TEXT("Content-Type"), TEXT("multipart/form-data; boundary=") + boundary);
const FString fileName(FPaths::Combine(path, crashToReport.folderName, TEXT("UE4Minidump.dmp")));
ensure(FPaths::FileExists(fileName));
const FString prefixBoundary(TEXT("\r\n--") + boundary + TEXT("\r\n"));
const FString fileHeader(TEXT("Content-Disposition: form-data; name=\"upload_file_minidump\"; filename=\"UE4Minidump.dmp\"\r\nContent-Type: application/octet-stream\r\n\r\n"));
FString fileContents;
FFileHelper::LoadFileToString(fileContents, *fileName);
const FString suffixBoundary(TEXT("\r\n--") + boundary + TEXT("--\r\n"));
const FString content(prefixBoundary + fileHeader + fileContents + suffixBoundary);
httpRequest->SetContentAsString(content);
This works to a degree, the server now accepts this, and will receive the file - however the file ends up being unreadable server-side, leading me to think I'm not sending it in the right format.
What kind of data is expected in a multipart/form-data request?
A thing I notice is that the curl request sends the file's header separately (the first chunk of 175 bytes). I would love some information on how to achieve that!

Answer 1

I finally figured it out. I'm not sure how to report what I was doing wrong, but I think it had to do with what happens under the hood in:
httpRequest->SetContentAsString(..)
, which reliably caused the backend to fail to interpret the binary file I was trying to send. I ended up reading the binary file .. as a binary file:
TArray<uint8> dumpFileData;
FFileHelper::LoadFileToArray(dumpFileData, *FPaths::Combine(path,crashToReport.folderName, TEXT("UE4Minidump.dmp")));
Then send it via the POST request, similarly as before, but adding the complete form data part as binary data:
TSharedRef<IHttpRequest, ESPMode::ThreadSafe> httpRequest = FHttpModule::Get().CreateRequest();
httpRequest->SetURL(TEXT("https://xxxxx.ingest.sentry.io/api/xxxxxx/minidump/?sentry_key=xxxxxxxxxxxxxxxxxxxxxxxxxxx"));
httpRequest->SetVerb(TEXT("POST"));
const FString boundary(TEXT("------------------------bb33b671b1212234"));
httpRequest->SetHeader(TEXT("Content-Type"), TEXT("multipart/form-data; boundary=") + boundary);
httpRequest->SetHeader(TEXT("Accept"), TEXT("*/*"));
httpRequest->SetHeader(TEXT("Expect"), TEXT("100-continue"));
{
const FString prefixBoundary(TEXT("--") + boundary + TEXT("\r\n"));
const FString fileHeader(TEXT("Content-Disposition: form-data; name=\"upload_file_minidump\"; filename=\"UE4Minidump.dmp\"\nContent-Type: application/octet-stream\r\n\r\n"));
const FString suffixBoundary(TEXT("\r\n--") + boundary + TEXT("--\r\n"));
TArray<uint8> CombinedContent;
CombinedContent.Append(FStringToUint8(prefixBoundary + fileHeader));
CombinedContent.Append(dumpFileData);
CombinedContent.Append(FStringToUint8(suffixBoundary));
httpRequest->SetContent(CombinedContent);
}
httpRequest->ProcessRequest();
For completeness, FStringToUint8 is defined as follows:
// Convert FString to UTF8 and put it in a TArray
TArray<uint8> FStringToUint8(const FString& InString)
{
TArray<uint8> OutBytes;
// Handle empty strings
if (InString.Len() > 0)
{
FTCHARToUTF8 Converted(*InString); // Convert to UTF8
OutBytes.Append(reinterpret_cast<const uint8*>(Converted.Get()), Converted.Length());
}
return OutBytes;
}