Streaming a zip file over http in .net with SharpZipLib - asp.net

I'm making a simple download service so a user can download all his images from out site.
To do that i just zip everything to the http stream.
However it seems everything is stored in memory, and the data isn't sent til zip file is complete and the output closed.
I want the service to start sending at once, and not use too much memory.
public void ProcessRequest(HttpContext context)
{
List<string> fileNames = GetFileNames();
context.Response.ContentType = "application/x-zip-compressed";
context.Response.AppendHeader("content-disposition", "attachment; filename=files.zip");
context.Response.ContentEncoding = Encoding.Default;
context.Response.Charset = "";
byte[] buffer = new byte[1024 * 8];
using (ICSharpCode.SharpZipLib.Zip.ZipOutputStream zipOutput = new ICSharpCode.SharpZipLib.Zip.ZipOutputStream(context.Response.OutputStream))
{
foreach (string fileName in fileNames)
{
ICSharpCode.SharpZipLib.Zip.ZipEntry zipEntry = new ICSharpCode.SharpZipLib.Zip.ZipEntry(fileName);
zipOutput.PutNextEntry(zipEntry);
using (var fread = System.IO.File.OpenRead(fileName))
{
ICSharpCode.SharpZipLib.Core.StreamUtils.Copy(fread, zipOutput, buffer);
}
}
zipOutput.Finish();
}
context.Response.Flush();
context.Response.End();
}
I can see the the worker process memory growing while it makes the file, and then releases the memory when its done sending. How do i do this without using too much memory?

Disable response buffering with context.Response.BufferOutput = false; and remove the Flush call from the end of your code.

use Response.BufferOutput = false; at start of ProcessRequest and flush response after each file.

FYI. This is working code to recursively add an entire tree of files, with streaming to browser:
string path = #"c:\files";
Response.Clear();
Response.ContentType = "application/zip";
Response.AddHeader("Content-Disposition", string.Format("attachment; filename=\"{0}\"", "hive.zip"));
Response.BufferOutput = false;
byte[] buffer = new byte[1024 * 1024];
using (ZipOutputStream zo = new ZipOutputStream(Response.OutputStream, 1024 * 1024)) {
zo.SetLevel(0);
DirectoryInfo di = new DirectoryInfo(path);
foreach (string file in Directory.GetFiles(di.FullName, "*.*", SearchOption.AllDirectories)) {
string folder = Path.GetDirectoryName(file);
if (folder.Length > di.FullName.Length) {
folder = folder.Substring(di.FullName.Length).Trim('\\') + #"\";
} else {
folder = string.Empty;
}
zo.PutNextEntry(new ZipEntry(folder + Path.GetFileName(file)));
using (FileStream fs = File.OpenRead(file)) {
ICSharpCode.SharpZipLib.Core.StreamUtils.Copy(fs, zo, buffer);
}
zo.Flush();
Response.Flush();
}
zo.Finish();
}
Response.Flush();

Related

retrieving Binary/Blob files from Microsoft Dynamics Nav with ASP.NET

I am working with a MS Dynamics Nav Database that have a file attachment tables. The files are stored in MS SQL. I am able to pull the files to my desktop with a custom asp.net application that I have built, but when I open the files, they are corrupted. These are PDFs files that are located in the "image" file type column of the database and I have tried to download over 20 files. All of them varies in size and seem to download successfully.
The reason why I suspect these are PDFs files is because the column right next to the binary columns give me the name of the file as in PDF format. I have also tried to renaming the file after I download to different image formats but without any luck when I tried to open it. This is not my first project to retrieve binary files, from MS SQL database. If anyone work on getting files off the Nav database before, please help me. The sample code below I wrote to retrieve files using LINQ to SQL when I give it a specific ID in the browser. Please advice me if you know any sort of compression or encryption in the binary files itself and how to grab the file successfully to read it. Thanks
protected void getFileFromID(string queryid)
{
string Filename = string.Empty;
byte[] bytes;
try
{
DataClassesFilesDataContext dcontext = new DataClassesFilesDataContext();
var myfile = (from file in dcontext.Comptroller_File_Attachments
where file.No_ == queryid
select file).First();
if (myfile.Table_ID.ToString().Length > 0 && myfile.Attachment != null)
{
Filename = myfile.FileName.ToString();
bytes = myfile.Attachment.ToArray();
Response.Clear();
Response.ContentType = "application/octet-stream";
Response.AddHeader("Content-Disposition", "attachment; filename=" + Filename);
Response.BinaryWrite(bytes);
Response.End();
}
else
{
Response.Write("no file exist");
}
}
catch (Exception e)
{
Response.Write(e);
}
}
Well. I figured it out. I read on a blog that 4 bytes was the "magic number" to get rid off. So all you have to do is get rid of 4 bytes from the BLOB bytes array and then decompress it with DeflateStream. The example code I post below is an example where it takes in a byte array and skip the first 4 using LINQ-to-SQL and return the byte and string filename for the 2nd function. It also pass in a queryid string parameter. I am sure the code can be improve more for efficiency purposes. For those who have trouble with this, just give this a try.
//get bytes and remove first 4 bytes from bytes array
protected Tuple<byte[], string> getBytesfromFile(string queryID)
{
byte[] MyFilebytes = null;
string filename = string.Empty;
try
{
DataClassesFilesDataContext dcontext = new DataClassesFilesDataContext();
var myfile = (from file in dcontext.Comptroller_File_Attachments
where file.No_ == queryID
select file).First();
if (myfile.Table_ID.ToString().Length > 0 && myfile.Attachment != null)
{
MyFilebytes = myfile.Attachment.ToArray().Skip(4).ToArray();
filename = myfile.FileName.ToString();
}
else
Response.Write("no byte to return");
}
catch
{
Response.Write("no byte");
}
return Tuple.Create(MyFilebytes, filename);
}
//after getting the remaining bytes (after removing 4 first byte) deflate the byte and then store it in a memory steam and get the result back.
protected void getFile()
{
try
{
string Filename = string.Empty;
byte[] myfile = getBytesfromFile(getQueryID()).Item1;
byte[] result;
using (Stream input = new DeflateStream(new MemoryStream(myfile),
CompressionMode.Decompress))
{
using (MemoryStream output = new MemoryStream())
{
input.CopyTo(output);
result = output.ToArray();
}
}
Filename = getBytesfromFile(getQueryID()).Item2;
Response.Clear();
Response.ContentType = "application/octet-stream";
Response.AddHeader("Content-Disposition", "attachment; filename=" + Filename);
Response.BinaryWrite(result);
Response.End();
}
catch (Exception e)
{
Response.Write(e);
}
}
//pass in file id
protected string getQueryID()
{
QueryID.QueryStringID = Request.QueryString["fileid"];
return QueryID.QueryStringID;
}

Problems with downloading pdf file from web api service

I'm trying to set up a web api service that searches for a .pdf file in a directory and returns the file if it's found.
The controller
public class ProductsController : ApiController
{
[HttpPost]
public HttpResponseMessage Post([FromBody]string certificateId)
{
string fileName = certificateId + ".pdf";
var path = #"C:\Certificates\20487A" + fileName;
//check the directory for pdf matching the certid
if (File.Exists(path))
{
//if there is a match then return the file
HttpResponseMessage result = new HttpResponseMessage(HttpStatusCode.OK);
var stream = new FileStream(path, FileMode.Open);
stream.Position = 0;
result.Content = new StreamContent(stream);
result.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment") { FileName = fileName };
result.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/pdf");
result.Content.Headers.ContentDisposition.FileName = fileName;
return result;
}
else
{
HttpResponseMessage result = new HttpResponseMessage(HttpStatusCode.Gone);
return result;
}
}
}
I'm calling the service with the following code
private void GetCertQueryResponse(string url, string serial)
{
string encodedParameters = "certificateId=" + serial.Replace(" ", "");
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(url);
httpRequest.Method = "POST";
httpRequest.ContentType = "application/x-www-form-urlencoded";
httpRequest.AllowAutoRedirect = false;
byte[] bytedata = Encoding.UTF8.GetBytes(encodedParameters);
httpRequest.ContentLength = bytedata.Length;
Stream requestStream = httpRequest.GetRequestStream();
requestStream.Write(bytedata, 0, bytedata.Length);
requestStream.Close();
HttpWebResponse response = (HttpWebResponse)httpRequest.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
byte[] bytes = null;
using (Stream stream = response.GetResponseStream())
using (MemoryStream ms = new MemoryStream())
{
int count = 0;
do
{
byte[] buf = new byte[1024];
count = stream.Read(buf, 0, 1024);
ms.Write(buf, 0, count);
} while (stream.CanRead && count > 0);
ms.Position = 0;
bytes = ms.ToArray();
}
var filename = serial + ".pdf";
Response.ContentType = "application/pdf";
Response.Headers.Add("Content-Disposition", "attachment; filename=\"" + filename + "\"");
Response.BinaryWrite(bytes);
}
}
This appears to be working in the sense that the download file dialogue is shown with the correct file name and size etc, but the download takes only a couple of seconds (when the file sizes are >30mb) and the files are corrupt when I try to open them.
Any ideas what I'm doing wrong?
Your code looks similar to what Ive used in the past, but below is what I typically use:
Response.AddHeader("content-length", myfile.Length.ToString())
Response.AddHeader("content-disposition", "inline; filename=MyFilename")
Response.AddHeader("Expires", "0")
Response.AddHeader("Pragma", "Cache")
Response.AddHeader("Cache-Control", "private")
Response.ContentType = "application/pdf"
Response.BinaryWrite(finalForm)
I post this for 2 reasons. One, add the content-length header, you may have to indicate how large the file is so the application waits for the whole response.
If that doesn't fix it. Set a breakpoint, does the byte array content the appropriate length (aka, 30 million bytes for a 30 MB file)? Have you used fiddler to see how much content is coming back over the HTTP call?

save multiple image to local from folder in asp.net

ive got a file download issue can you help me for that...
here is the code:
DirectoryInfo directoryInfo = new DirectoryInfo(Server.MapPath(#"/Bailiffs/BailiffFiles/"));
string cukurNumber = string.Empty;
if (txtCukurNumber.Text != string.Empty) {
cukurNumber = txtCukurNumber.Text;
}
FileInfo[] fileInfoEnum = directoryInfo.GetFiles(cukurNumber + "*");
Response.Clear();
Response.AddHeader("Content-Disposition", "attachment;filename=" + txtCukurNumber.Text + ".zip");
Response.ContentType = "application/zip";
using (ZipOutputStream zipStream = new ZipOutputStream(Response.OutputStream)) {
zipStream.SetLevel(9);
byte[] zipBuffer = new byte[4096];
foreach (FileInfo fileInfo in fileInfoEnum) {
string fileFullName = fileInfo.FullName;
ZipEntry zipEntry = new ZipEntry(Path.GetFileName(fileFullName));
zipEntry.DateTime = DateTime.Now;
zipStream.PutNextEntry(zipEntry);
using (FileStream fileStream = File.OpenRead(fileFullName)) {
int sourceBytes = 0;
do {
sourceBytes = fileStream.Read(zipBuffer, 0, zipBuffer.Length);
zipStream.Write(zipBuffer, 0, sourceBytes);
} while (sourceBytes > 0);
}
}
zipStream.Finish();
zipStream.Close();
Response.Flush();
Response.End();
}
this code must be get all image files by filter and save to disk but save file dialog of browser is opening just one time and one bizarre file is saving... where am i doing wrong...
thanks..
Edit : Bizzarre file issue is solved now the main issue is single file saving instead of multiple..
thanks again...
Although you are looping over each file in the directory, once you do your Response.End() on the first iteration of the loop, the response to the user is done. They would only get the first file that is found by the enumerator.
The browser doesn't have a concept of receiving multiple files in the way you are attempting.
You may consider collecting the various image files and putting them together in a ZIP file, and then returning a single ZIP back to the user.
Here is example code that will build a ZIP (using SharZipLib) of the images and reply with a single file called "images.zip"
Include these using statements for SharpZipLib:
using ICSharpCode.SharpZipLib.Core;
using ICSharpCode.SharpZipLib.Zip;
using ICSharpCode.SharpZipLib.Checksums;
Then in the method where you want to stream back the ZIP file:
DirectoryInfo directoryInfo = new DirectoryInfo(Server.MapPath(#"/Bailiffs/BailiffFiles/"));
string cukurNumber = string.Empty;
if (txtCukurNumber.Text != string.Empty) {
cukurNumber = txtCukurNumber.Text;
}
IEnumerable<FileInfo> fileInfoEnum = directoryInfo.EnumerateFiles( cukurNumber + "*" );
Response.Clear();
Response.AddHeader( "Content-Disposition", "attachment;filename=images.zip" );
Response.ContentType = "application/zip";
using( ZipOutputStream zipstream = new ZipOutputStream( Response.OutputStream ) ) {
zipstream.SetLevel( 9 ); // 0-9, 9 being the highest compression
byte[] buffer = new byte[4096];
foreach( FileInfo fileInfo in fileInfoEnum ) {
string file = fileInfo.FullName;
ZipEntry entry = new
ZipEntry( Path.GetFileName( file ) );
entry.DateTime = DateTime.Now;
zipstream.PutNextEntry( entry );
using( FileStream fs = File.OpenRead( file ) ) {
int sourceBytes;
do {
sourceBytes = fs.Read( buffer, 0, buffer.Length );
zipstream.Write( buffer, 0, sourceBytes );
} while( sourceBytes > 0 );
}
}
zipstream.Finish();
zipstream.Close();
}
Response.Flush();
Response.End();

Remote file Download via ASP.NET corrupted file

I am using the code below which I have found on one of the forums to download a file in remote server. it seems it is working. However, the downloaded file is corrupted and I cannot unzip.
Do you have any idea why it is so? or if my approach is wrong, could you suggest me a better way please?
protected void Page_Load(object sender, EventArgs e)
{
string url = "http://server/scripts/isynch.dll?panel=AttachmentDownload&NoteSystem=SyncNotes&NoteType=Ticket&NoteId=1&Field=supp&File=DisplayList%2etxt";
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
req.Credentials = new NetworkCredential("user", "pass");
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
////Initialize the output stream
Response.Clear();
Response.ContentType = "application/octet-stream";
Response.AppendHeader("Content-Disposition:", "attachment; filename=" + "DisplayList.txt");
Response.AppendHeader("Content-Length", resp.ContentLength.ToString());
////Populate the output stream
byte[] ByteBuffer = new byte[resp.ContentLength];
Stream rs = req.GetResponse().GetResponseStream();
rs.Read(ByteBuffer, 0, ByteBuffer.Length);
Response.BinaryWrite(ByteBuffer);
Response.Flush();
///Cleanup
Response.End();
rs.Dispose();
}
First of all, use application/octet-stream as it is the standard content type for downloads.
new byte[resp.ContentLength + 1] will define a buffer which is one byte larger than content type. I believe this is the reason for corruption. Use new byte[resp.ContentLength].
I actually recommend re-writing it and removing memorystream:
const int BufferLength = 4096;
byte[] byteBuffer = new byte[BufferLength];
Stream rs = req.GetResponse().GetResponseStream();
int len = 0;
while ( (len = rs.Read(byteBuffer,0,byteBuffer.Length))>0)
{
if (len < BufferLength)
{
Response.BinaryWrite(byteBuffer.Take(len).ToArray());
}
else
{
Response.BinaryWrite(byteBuffer);
}
Response.Flush();
}
the article on http://support.microsoft.com/default.aspx?scid=kb;en-us;812406 solved my problem. Many thanks to #Aliostad for his effort to help me.

Problem downloading a 25MB file - the 8MB file downloads without problem (ASP.NET)

I have two files at the same location but the big one, when is about to finish download, gives an error (both in IE and Firefox).
I use the following code:
public static void DownloadZipFile (string filename, bool notifyMe)
{
HttpContext context = HttpContext.Current;
HttpServerUtility server = context.Server;
bool ok = false;
try
{
string file = string.Format ("~/contents/licensing/members/downloads/{0}", filename);
string server_file = server.MapPath (file);
HttpResponse response = context.Response;
//response.BufferOutput = false;
response.ContentType = "application/zip";
string value = string.Format ("attachment; filename={0}", filename);
response.AppendHeader ("Content-Disposition", value);
FileInfo f = new FileInfo (server_file);
long size = f.Length;
response.TransmitFile (server_file, 0, size);
response.Flush ();
ok = true;
response.End ();
}
catch (Exception ex)
{
Utilities.Log (ex);
}
finally
{
if (ok && notifyMe)
NotifyDownload (filename);
}
}
Any ideas?
Response.End() calls Response.Flush(). Try removing the Flush call.
The solution to this problem is to add the line:
response.AddHeader("Content-Length",size.ToString());
before the call to TransmitFile ().
The credits go to Jim Schubert (see his comment above).

Resources