After importing an entity (e.g. Car) from JDL studio with one field as a Blob I notice that the generated code looks like this:
this.dataUtils.toBase64(file, (base64Data) => {
car[field] = base64Data;
car[`${field}ContentType`] = file.type;
});
i.e. the file is base64 encoded and subsequently sent with content-type:application/json.
From this post, it seems that the canonical way to upload files to a REST API is using the multipart/form-data (as per W3 recommendation).
For my curiosity, I'm just wondering why base64 / application/json was used as opposed to multipart/form-data.
Thanks,
Related
What is the process by which sinatra's send_file decides what content-type to use?
For example, it seems that it works by the extension of the file passed to send_file, so if it is send_file blah.txt . then when I http to the route, I will get/ the response header will be, content-type: text/plain, so any html in the txt file will be interpreted by the web browser as plain text. Whereas if the file is blah.html then the server will respond with content-type: text/html.(and any html in the file is rendered as such)
And of course the route name is irrelevant so you could go to http://127.0.0.1:4567/zzz.html and it could lead to send_file a.txt and a.txt may contain html tags but since it's a .txt file send_file will cause sinatra to respond with content-type: text/plain and the browser won't render any html sent and will show it as plain text. I may be wrong but that seems to be what my quick tests indicate. Where I tried different routes, different filename extensions(.txt, and .html), sometimes files with html in them sometimes not, seeing whether the browser renders the html or not, and seeing what the content-type header was, with wget -d.
So then my question related to that is, is there a list that sinatra's send_file function uses, that relates file extension to content-type? I would like to see that list. And if not, then what is the process it is using.
Note- I understand there is a way to pass in a content-type Sinatra: How to respond with an image with headers "content-type" => "image/jpeg"
but i'm asking how/ by what method, send_file determines content-type when no content-type is passed in.
This is the send_file method in the Sinatra framework (currently v2.0.5), notice it hands off finding out the content type straight away if none has been set:
if opts[:type] or not response['Content-Type']
content_type opts[:type] || File.extname(path), :default => 'application/octet-stream'
end
The content_type method will either return immediately or hand off to mime_type, which is a delegate of Rack's mime_type method (currently v2.0.7). This uses a well known list of extensions to check against.
def mime_type(ext, fallback='application/octet-stream')
MIME_TYPES.fetch(ext.to_s.downcase, fallback)
end
The list begins on line 49:
MIME_TYPES = {
".123" => "application/vnd.lotus-1-2-3",
".3dml" => "text/vnd.in3d.3dml",
".3g2" => "video/3gpp2",
".3gp" => "video/3gpp",
# <snip>
As you can see from the content_type snippet, the default it falls back on is application/octet-stream.
I am using dropzone.js to upload files to Web API 2 service. Reading multipart stream gives garbled Russian characters. For example, when I upload file with name Русское название - Russian characters it gives ????? ?????? - Russian characters.
I'm sure that dropzone.js works fine and it is just a Web API problem.
Here is GetStream method.
public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
{
// For form data, Content-Disposition header is a requirement
ContentDispositionHeaderValue contentDisposition = headers.ContentDisposition;
if (contentDisposition != null)
{
// We will post process this as form data
_isFormData.Add(String.IsNullOrEmpty(contentDisposition.FileName));
return new MemoryStream();
}
// If no Content-Disposition header was present.
throw new InvalidOperationException(
string.Format("Did not find required '{0}' header field in MIME multipart body part..",
"Content-Disposition"));
}
It is probably bad encoding on js side imo. As you have posted instead of "Русское название - Russian characters" you get "????? ?????? - Russian characters", which means that file is readen with suport of English characters but without russian ones. Try changing encoding to one that supports cyrylic characters.
This might be helpfull: http://en.wikipedia.org/wiki/Character_encoding.
i have written a flex component to allow the user to select an image from the local filesystem and then POST it to a CQ5 DAM.
there are 2 CQ5 instances with which i'm working. the image posts fine to one instance, but not the other. specifically, in the 2nd instance, the renditions are not getting created when using the component.
one difference i've noted is that the working images, when i look at them in crxde, have a jcr:primaryType of dam:Asset. the non-working ones are nt:File.
from Flex, I use URLLoader to POST with a multipart form. the request (in part) looks like this:
POST /content/dam/test/foo.createasset.html HTTP/1.1
Host: xxxxxxxx:4502
Content-type: multipart/form-data; boundary=doudrbitutcfasnbhlpogirdctuxem
--doudrbitutcfasnbhlpogirdctuxem
Content-Disposition: form-data; name="file"
home.png
--doudrbitutcfasnbhlpogirdctuxem
Content-Disposition: form-data; name="Filename"
home.png
--doudrbitutcfasnbhlpogirdctuxem
Content-Disposition: form-data; name="home.png"; filename="home.png"
Content-Type: application/octet-stream
*** image data ***
--doudrbitutcfasnbhlpogirdctuxem
Content-Disposition: form-data; name="Upload"
Submit Query
--doudrbitutcfasnbhlpogirdctuxem--
that does save the image at: /content/dam/test/foo/home.png
i've tried adding a variable to the form:
./jcr:contentType dam:Asset
but that didn't cause the contentType to change. instead, the file didn't show up in CQ5 at all.
i know next to nothing about CQ5. i've seen some (old) examples of code POSTing right to where they want the asset to go, instead of hitting foo.createAsset.html as i've done. i could not get the more-straightforward POST working, and instead used CQ5 DAM to upload and image and captured through Charles, then tried to replicate that.
the CQ5 version that works is 5.5.0.
the version that does not is 5.4.0.
i'm sure that there are other configuration differences as well. in addition, the client is unwilling to upgrade from 5.4.0.
am i on the right track? close?
edit to clarify server setup:
CQ 5.5.0 --> installed locally, this one is an author server. my component works when POSTing to this server. meaning, the uploaded image is marked as dam:Asset and the renditions are generated.
CQ 5.4.0 --> a dev instance used by many. this is an author and publish server. my component does not 100% work when POSTing to this server. however, if i use the DAM admin interface to upload an image, it does properly mark the image as dam:Asset and generate the renditions.
edit #2: WORKING
it turns out that the dev/5.4 instance handles file uploading differently. my multi-part POST code mostly worked, but instead of using createAsset.html, i'm uploading to /tmp/fileupload.
then i issue a 2nd POST, using application/x-www-form-urlencoded, to issue a move command.
for those wishing to do the same, the move code looks like this:
var service:HTTPService = new HTTPService();
var url:String = instanceUrl + "/tmp/fileupload";
service.url = url;
var headerData : Object = new Object();
headerData['Cache-Control'] = 'no-store';
headerData['Authorization'] = getAuthString();
service.headers = headerData;
service.contentType = "application/x-www-form-urlencoded; charset=UTF-8";
service.method = URLRequestMethod.POST;
var urlVar:URLVariables = new URLVariables();
var command:String = "/var/dam/" + destPath + "/" + filename + "#MoveFrom";
var arg:String = "/tmp/fileupload/" + filename;
urlVar[command] = arg;
urlVar["_charset_"] = "utf-8";
var token:AsyncToken = service.send(urlVar);
not knowing CQ5, i can only assume the dev server is set up to run some workflow steps when it receives the #MoveFrom; those are the steps that ensure the uploaded file is of type dam:Asset and that the desired renditions are created.
If uploading from the DAM admin page via a browser works on the 5.4.0 instance, I would suggest analysing the HTTP request that this makes, to reproduce the same request from your Flex client. There's probably a subtle difference between the 5.4.0 and 5.5.0 HTTP APIs that explains this.
as a followup, below are the broad steps i took to get this working.
my overall goal was to write a Flex component that, for a specified VO, allowed the user to upload an image from their local filesystem (i used FileReference for this) into the component, then upload that image to CQ5 and publish it. after it was published, i then read it back into the component to display it.
i won't put the full code solution here, as it's involved and belongs to my client. in addition to my component, i wrote a utility for cq5 DAM operations, and an http service with built-in retries (which ended up being necessary because even though cq will give me a 200 when i request a resource, subsequent operations on that resource may fail, because cq doesn't seem to think it's there). Note that in all retry instances, i have a max retry count. the default value is 10, and default retry interval is 250ms.
please understand i know very little of CQ; most of what i learned was reverse engineering through trying things in the tool and watching Charles. also understand the steps below may be very specific to the install of CQ5 i'm working with.
so here are my overall steps. unless indicated otherwise, all requests are on port 4502:
a destination directory is determined from data in the VO and a POST is issued to create it. this is done with Content-type=application/x-www-form-urlencoded. the url is the full path of the folder i want to create, with no trailing slash.
repeat a GET on the created directory until we get a 200. the url here does have a trailing slash.
the image is POSTed to a temp area, [instance]/tmp/fileupload, as multipart form data. To help with this, i used an MIT-licensed AS class called MultipartURLLoader (https://code.google.com/p/in-spirit/). I used Content-type=multipart/form-data; boundary=[boundary]. CQ seemed very picky about the contents of the form data. mine is set up like this:
file: [name of file]
Filename: [name of file]
[name of file]: [file data]
Upload: Submit Query
another POST is issued, with a move command, to move the image from the temp area to the directory created in step 1. the url is [instance]/tmp/fileupload, and Content-type=application/x-www-form-urlencoded; charset=UTF-8. The form data is set up like this:
/var/dam/[destination_path]/[filename]#MoveFrom: /tmp/fileupload/[filename]
charset: utf-8
repeat step 4 until we get a 200. when new destination folders are indeed created, the first POST to #MoveFrom usually results in a 500, saying the destination folder is not there. perhaps there's another way to ask CQ if the destination is ready? i don't know.
we now need to publish the file, but first we issue a series of GETs on it to ensure it's there, with this url: [instance]/content/dam/[destination]/[filename].assets.json. once it's there, CQ will respond with some JSON that we use next.
check to see if the file has already been published. it may be the case that the user has already uploaded an image with the same name to the same location. the JSON response has a results node, which i check to see if it's 1. if it is, then i look at "pages[0].replication" to see if it has a node called "action". if it does, i see if the value is "ACTIVATE". if it is, it's already published. in every other case, i try to publish it.
POST a command to activate (publish it). the url is [instance]/bin/replicate.json. Content-type=application/x-www-form-urlencoded; charset=UTF-8. The form looks like this:
path: /content/dam/[destination]/[filename]
cmd: Activate
charset: utf-8
for my purposes, i wanted to then retrieve the published image to re-display it in my component. i waited for the 200 from the publish, then tried my GET. the url i used here had no port number, and no trailing slash: [instance:80]/content/dam/[destination]/[filename]. The first call almost always gave me a 404, so i kept trying until i got the 200.
that's it. i hope this is helpful to someone.
note: just saw that "charset" is in italics in the form specification. note that i'm using (underscore)charset(underscore).
I'm sending files from an android app to a asp.net webform using multipart/form-data as the content type. However the Request.files property does not get populated. Reading the Request object I get the following
Request.Params("ALL_HTTP")
"HTTP_CONNECTION:Keep-Alive HTTP_CONTENT_LENGTH:8913 HTTP_CONTENT_TYPE:multipart/form-data;boundary=*********************** HTTP_HOST:192.168.1.2 HTTP_USER_AGENT:Dalvik/1.2.0 (Linux; U; Android 2.2; sdk Build/FRF91) "
The HTTP_CONTENT_LENGTH shows the correct length. I guess I will have to do a binary read and then parse the content and store the file contents. Has anyone done this before or is there a library/class available?
Thanks
How are you writing the files to the request stream? The following rules should be followed when programatically uploading files (binary streams):
1) Write a boundary (it could be anything prefixed by two dashes). Here is an example boundary:
private string boundary = "----" + DateTime.Now.Ticks;
2) Write content disposition in the form:
Content-Disposition: form-data; name="{name}"; filename="{filename}"
3) Write the content type
4) Write an empty line
5) Write the bytes to the request stream
6) Write the end boundary, it marks the end of the request. It should be in the following form:
"--" + boundary + "--"
7) Write an empty line and flush (if needed) the request.
Here is how sample upload request should look inside an Http debugging tool like fiddler:
------634388181001966332
Content-Disposition: form-data; name="files"; filename="cald_3d.JPG"
Content-Type: application/octet-stream
1010101001... (more bytes)
------634388181001966332--
Then, on the server, access the file with Request.Files[name], the same name which you have used when specifying Content Disposition. Good luck :)
Simply put, I'd like someone to be able to click a link, and get a one-time-use pdf. We have the library to create PDF files, so that's not an issue.
We could generate a link to an aspx page, have that page generate the pdf, save the pdf to the filesystem, and then Response.Redirect to the saved pdf. Then we'd somehow have to keep track of and clean up the PDF file.
Since we don't ever need to keep this data, what I'd like to do instead, if possible, is to have the aspx page generate the pdf, and serve it directly back as a response to the original request. Is this possible?
(In our case, we're using C#, and we want to serve a pdf back, but it seems like any solution would probably work for various .NET languages and returned filetypes.)
Assuming you can get a byte[] representing your PDF:
Response.Clear();
Response.ContentType = "application/pdf";
Response.AddHeader("Content-Disposition",
"attachment;filename=\"FileName.pdf\"");
Response.BinaryWrite(yourPdfAsByteArray);
Response.Flush();
Response.End();
Look at how HTTP works. The client (=browser) doesn't rely on extensions, it only wants the server to return some metadata along with the document.
Metadata can be added with Response.AddHeader, and one 'metadata line' consists of Name and Value.
Content-Type is the property you are interested in, and the value is MIME type of the data (study: RFC1945 for HTTP headers, google for MIME type).
For ordinal aspx pages (html, ....) the property is 'text/html' (not so trivial, but for this example it is enough.). If you return JPG image, it can have name 'image.gif', but as long as you send 'image/jpeg' in Content-Type, it is processed as JPG image.
Content-type for pdf is 'application/pdf'.
The browser will act according to default behaviour, for example, with Adobe plugin, it will display the PDF in it's window, if you don't have any plugin for PDF, it should download the file, etc..
Content-Disposition header says, what you should do with the data. If you want explicitly the client to 'download' some HTML/PDF/whatever, and not display it by default, value 'attachment' is what you want. It should have another parameter, (as suggested by Justin Niessner), which is used in case of something like:
http://server/download.aspx?file=11 -> Content-Disposition: attachment;filename=file.jpg says, how the file should be by default named.