Getting attachment ZIP file in REST call response and its content type is application/octet-stream. How to convert it as zip file.
Tried fromBase64, but no luck.
Runtime: 4.3
Try using the Compression Module to decompress the zip payload received.
Example:
<compression:extract>
<compression:compressed>#[payload]</compression:compressed>
<compression:extractor>
<compression:zip-extractor/>
</compression:extractor>
</compression:extract>
Used below code to transform and write zip to local drive.
<ee:transform doc:name="Transform Message" doc:id="da388237-d53a-47aa-83b2-96d1ef04489d" >
<ee:message >
<ee:set-payload ><![CDATA[%dw 2.0
output application/octet-stream
---
payload.content]]></ee:set-payload>
</ee:message>
</ee:transform>
<file:write doc:name="Write" doc:id="1cfd5210-a65a-4ace-9f07-0f575db3a01a" config-ref="File_Config" path="C:\Retrieval.zip"/>
Related
I still don't have a clear picture of practical examples of the chunked header usage, after reading some posts and Wikipedia.
One example I see from Content-Length header versus chunked encoding, is:
On the other hand, if the content length is really unpredictable
beforehand (e.g. when your intent is to zip several files together and
send it as one), then sending it in chunks may be faster than
buffering it in server's memory or writing to local disk file system
first.
So it means that I can send zip files while I am zipping them ? How ?
I've also noticed that if I download a GitHub repo, I am receiving data in chunked. Does GitHub also send files in this way (sending while zipping) ?
A minimal example would be much appreciated. :)
Here is an example using perl (with IO::Compress::Zip module) to send a zipped file on the fly as #deceze pointed to
use IO::Compress::Zip qw(:all);
my #files = ('example.gif', 'example1.png'); # here are some files
my $path = "/home/projects/"; # files location
# here is the header
print "Content-Type: application/zip\n"; # we are going to compress to zip and send it
print "Content-Disposition: attachment; filename=\"zip.zip\"\r\n\r\n"; # zip.zip for example is where we are going to zip data
my $zip = new IO::Compress::Zip;
foreach my $file (#files) {
$zip->newStream(Name => $file, Method => ZIP_CM_STORE); # storing files in zip
open(FILE, "<", "$path/$file");
binmode FILE; # reading file in binary mode
my ($buffer, $data, $n);
while (($n = read FILE,$data, 1024) != 0) { # reading data from file to the end
$zip->print($data); # print the data in binary
}
close(FILE);
}
$zip->close;
As you see in the script so even if you add the zip filename in the header, it doesn't matter, because we are zipping the files and printing it in binary mode right away, so it's not necessary to zip the data and store them then send it to the client, you can directly zip the files and print them without storing it.
In Ruby:
require 'open-uri'
download = open('http://example.com/download.pdf')
IO.copy_stream(download, '~/my_file.pdf')
How to do the same in Crystal?
We can do the following:
require "http/client"
HTTP::Client.get("http://example.org") do |response|
File.write("example.com.html", response.body_io)
end
This writes just the response without any HTTP headers to the file. File.write is also smart enough to not download the entire file into memory first, but to write to the file as it reads chunks from the given IO.
I found something that works:
require "http/request"
require "file"
res = HTTP::Client.get "https://ya.ru"
fl=File.open("ya.html","wb")
res.to_io(fl)
fl.close
I have a ZIP File to be served via Symfony. The controller looks like this:
$headers = [
'Content-Type' => 'application/zip',
'Content-Disposition' => 'attachment; filename="archive.zip"'
];
return new Response(file_get_contents($pathToFile), 201, $headers);
And this one works well. However, if I try to use BinaryFileResponse (as the Documentation recommends), the ZIP File gets corrupted:
$response = new BinaryFileResponse($pathToFile);
$response->setContentDisposition(ResponseHeaderBag::DISPOSITION_ATTACHMENT);
$response->setStatusCode(Response::HTTP_CREATED);
return $response;
The output I get when trying to fix the file with zip -FF archive.zip --out fixed.zip :
zip warning: End record (EOCDR) only 17 bytes - assume truncated
(this command fixes the archive correctly)
Is it a bug or am I doing something wrong?
My setup:
Symfony 2.8.11
PHP 7.0.8
Ubuntu 16.04
nginx 1.10.0
EDIT:
I have made proposed changes, but the problem still exists:
$response = new BinaryFileResponse($pathToFile);
$response->setContentDisposition(ResponseHeaderBag::DISPOSITION_ATTACHMENT, 'archive.zip');
$response->headers->set('Content-Type', 'application/zip');
clearstatcache(false, $pathToFile);
return $response;
EDIT2:
I found one more interesting thing: serving this ZIP file with standard Response (the working code) creates openable file, however running zip -T on it gives:
1 extra byte at beginning or within zipfile
Testing original file gives:
OK
The size of file is less than 1MB.
SOLUTION:
When I opened generated ZIP file in text editor, I found an extra empty line at the beggining of it...
So I've added ob_clean(); before returning Response object and now it works!
No idea where this newline character came from, though...
Since I see you are returning 201 http header I assume file has been created with same request. As per symfony documentation:
If you just created the file during this same request, the file may be sent without any content. This may be due to cached file stats that return zero for the size of the file. To fix this issue, call clearstatcache(false, $file) with the path to the binary file.
I have been assigned a task to retrieve information from files that were processed from raw mass spectrophotometry data (file.mzML).
These files have just ".data" as extension and when I open them, I cannot recognize the language and therefore I cannot load them into R and work on them.
The files with the .data extension are contained in this folder:
ftp://ftp.pride.ebi.ac.uk/pride/data/archive/2015/11/PXD000299/
Could someone take a look at any of the files.data and tell me the language is in (e.g. F010439)?
EDIT: these is are some snippets
Lines 1 to 10
MIME-Version: 1.0 (Generated by Mascot version 1.0)
Content-Type: multipart/mixed; boundary=gc0p4Jq0M2Yt08jU534c0p
--gc0p4Jq0M2Yt08jU534c0p
Content-Type: application/x-Mascot; name="parameters"
LICENSE=Licensed to: INRA Tours, P-F Proteomique Analytique & Fonction.,Nouzilly (0085-0000003524/1), (2 processors).
MP=
NM=
COM=
Lines 120 to 130
NeutralLoss3_master=63.998285
--gc0p4Jq0M2Yt08jU534c0p
Content-Type: application/x-Mascot; name="unimod"
<?xml version="1.0" encoding="UTF-8" ?>
<umod:unimod xmlns:umod="http://www.unimod.org/xmlns/schema/unimod_2" majorVersion="2" minorVersion="0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.unimod.org/xmlns/schema/unimod_2 unimod_2.xsd">
<umod:elements>
<umod:elem avge_mass="1.00794" full_name="Hydrogen" mono_mass="1.007825035" title="H"/>
<umod:elem avge_mass="2.014101779" full_name="Deuterium" mono_mass="2.014101779" title="2H"/>
<umod:elem avge_mass="6.941" full_name="Lithium" mono_mass="7.016003" title="Li"/>
<umod:elem avge_mass="12.0107" full_name="Carbon" mono_mass="12" title="C"/>
It looks like that file is MIME-encoded "multi-part" file that contains (at least) two components files.
You can decode the multi-part using a MIME decoder.
The first component file looks like a simple "name=value" property file
The second component file is XML.
I don't know if this is relevant, but a search for Mascot file format gave me this reference page:
http://www.matrixscience.com/help/data_file_help.html
Also, there were some interesting hits when I searched for parser "x-mascot" and parser mascot. If you can find an existing parser, then you may be able to save yourself implementation effort.
I would like to determine real file extension.
example :
file = "test.fakeExt"
// but the real extention is .exe // for security reason I wish to avoid using it!
How can I do that?
If you want to determine the extension you could use findmimefromdata.
It looks at the first part of the file to determine what type of file it is.
FindMimeFromData function
Sample code
The first two bytes of an .exe file are allways 'MZ'.
So you could read the binary file, and see if the first two bytes are MZ, then you know it's an .exe file...