I'm trying to use the HERE route match API, and I'm confused about the file parameter when attempting to use the GET interface. The API for the file parameter states:
Base64 encoded string that contains the (zipped or plain) trace file
content. Zipped can be PKZip (used by zip, WinZip, 7Zip etc) or ZLib
(used by deflate) format. Only used for GET requests. In POST
requests, the file is sent as request body, not Base64 encoded, plain
or zipped. Note: Browsers limit the size of the file to a few KB when
passed in a GET request.
I've chosen to use the GPX format for my waypoints. Questions:
What is the required character encoding of the underlying XML
characters when sending plain or zipped trace file content?
Are there specifications for the compression method (Deflate, Deflate64,
etc.) and compression level?
Are there required specifications for
the type of base64 encoding?
I can definitively answer #3, and that requires URL safe encoding.
Related
Is it legal to send an http file upload request without a filename in content-disposition header?
I know normal browsers do send the filename, but all I could find in the RFC was:
The original local file name may be supplied as well, either as a
'filename' parameter either of the 'content-disposition: form-data'
header or in the case of multiple files in a 'content-disposition:
file' header of the subpart. The client application should make best
effort to supply the file name; if the file name of the client's
operating system is not in US-ASCII, the file name might be
approximated or encoded using the method of RFC 1522. This is a
convenience for those cases where, for example, the uploaded files
might contain references to each other, e.g., a TeX file and its .sty
auxiliary style description.
Which is not very decisive if the 'filename' is mandatory or not..
I am using ruby to send a SOAP request to a very enterprisey bla bla service, so unfortunately I can not attach any samples, there's nobody to send any server-side logs, nobody knows whats wrong on the provider side or how the actual HTTP requests need to look like (except a single XML example I got, but no HTTP headers), the docs are very Microsoft-centric with C# examples and whatnot ("instantiate AbstractFactoryFactory..." and whatnot), long live enterprise software.
But the bottom line is, eventually I took one of their own XMLs from their logs and sent it via HTTP to the endpoint from the WSDL and sent it to their host using the Savon gem raw XML option and got a HTTP 500 error from their host and a bunch of non-ascii binary data inside - literally, no ASCII characters are in the body.
I guessed that maybe Savon does some bad magic or that the XML option is not working as expected and I tried sending the same request via Faraday, but got the same thing,
the HTTP response headers says it's a HTTP response, XML encoded, from an ASP.NET host:
"content-type"=>"text/xml; charset=utf-8",
"server"=>"Microsoft-IIS/7.5",
"x-aspnet-version"=>"2.0.50727",
"x-powered-by"=>"ASP.NET",
but again, a 440 bytes worth of binaries in the response:
method=:post,
body=
"\x1F\x8B\b\x00\x00...
etc.
Am I missing some weird aspect of the SOAP specification and I need to do something to decode this data or has their server gone bonkers from my XML, HTTP headers or something else and I need to ping the provider?
Update 1
I noticed that their original XML had UTF-16 encoding set, so I tried encoding the raw string to UTF-16, then had Savon spew errors at me about bad data, then I updated encoding in the Savon client config. But I still get HTTP 500 error and binaries as response and if I try to log anything Savon reports a bug:
Encoding::CompatibilityError: incompatible encoding regexp match (US-ASCII regexp with UTF-16 string)
from /home/bbozo/.rvm/gems/ruby-2.2.4/gems/savon-2.11.1/lib/savon/log_message.rb:13:in `to_s'
Faraday basically reported the same behavior, an binary blob.
Update 2
I tried piping the encoding to every known encoding, and got nothing, even though the HTTP headers imply the encoding is UTF-8, it obviously isn't
Encoding.name_list.map{ |e_in| [ e_in, ( response.body.dup.force_encoding(e_in).encode('utf-8') rescue 'incompatible' ) ] }
There is nothing that would indicate the encoding in the WSDL files, the API spec doesn't even mention encoding except that the request XMLs need to be UTF-8 encoding, I tried encoding the body, changing the XML encoding definition, HTTP headers, but still I get the same binary blob, with the same heading (\x1F\x8B\b\x00\x00) - so it's not some weird encryption either.
Compression maybe?
I tried with https for good measure and nothing.
Question
Am I missing some weird aspect of the SOAP specification and I need to do something to decode this data or has their server gone bonkers from my XML, HTTP headers or something else and I need to ping the provider?
The response body was compressed! In the end I just gunzipped it and there it was,
How to decompress Gzip string in ruby?
I'm developing a web page with a form which returns a PDF document based on the form data. Currently I use the HTTP response fields
Content-Type: application/pdf
Content-Disposition: attachment; filename="foo.pdf"
However, since the field Content-Disposition is non-standard and doesn't work in all browsers I'm looking for a different approach. Do I have to save the PDF document on the server? What is the modus operandi?
Edit: By "doesn't work in all browsers" I mean that with some browsers the filename is not set to foo.pdf. Dillo, for instance, just sets the default filename (in the download dialog) to the basename of the URL path (plus query string).
Do I have to save the PDF document on the server?
No. As far as the HTTP client is concerned it, the inner workings of the server are completely opaque to it. All it sees is a TCP stream of bytes from the server and how exactly that stream is produced doesn't matter as long as it matches the specified Content-Type.
Just send the PDF right after the HTTP headers and you're done with.
Update due to comment
So if you're wondering how to supply a filename without using a header field: Just augment the URL with it. I.e. something like
http://${DOMAIN}/${PDF_GENERATOR}/${DESIRED_FILENAME}
In the HTTP server add a rewrite rule to simply omit the filename part and redirect to just
http://${DOMAIN}/${PDF_GENERATOR}
The HTTP client does not see that, all it see is some URL ending with a "filename", that it can present the user as a default for saving.
I currently upload to a webservice on an nginx server using the upload module (http://www.grid.net.ru/nginx/upload.en.html) from a custom desktop application doing a simple multipart-form POST that sends a file in one part and a base64 encoded XML with the file's metadata in another part.
The server receives this POST, passes it to my webservice which reads the metadata, processes the file and all is good.
What I want to do now is use the upload module's upload_resumable directive to do the POST in several chunks to minimize disconnection chances and allow resume. I can currently do this following the protocol described here: http://www.grid.net.ru/nginx/resumable_uploads.en.html
One sends byte ranges of the file along with some headers to identify the chunk and the session in several posts and once all the parts have been uploaded, nginx will compose the final POST containing the file name and path and pass it to your upload_pass location (which in my case CGIs to a django app).
However, I am not clear on how one would send a multipart post with this method since the protocol indicates that the body of the POST must be the bytes indicated in the byte range. I need the final post to also contain the XML I wrote about above.
I can think of sending the XML as the first bytes of the body and a header that indicates how many bytes belong to it but that would mean extra handling of the final file to remove that header and the final files are potentially in the GB size range.
Any other ideas?
Since the protocol supported by nginx specifically states that the post should not be multipart I ended up sending the file in the body, and the rest of the parameters encoded in the URL. Not the prettiest URLs but it works.
I'm currently implementing a client application that POST's a file over HTTP and have implemented base64 encoding on the file's data parameter.
However, it appears that when inspecting the traffic between a simple HTML page with a file upload form and the server that no Content-Transfer-Encoding header is sent in the body when describing the file's parameter.
Is this the preferred way of POST'ing a file over HTTP?
No, the preferred way is using multipart/form-data encoding, exactly as you would use with HTML form based file uploads.