I am using asp.net webforms.
I have pdfs in Azure storage that I need to process. I am using PDFJet library in order to do that.
I would like to
stream the pdf without downloading it as I have to process a large number of pdfs.
I am using the following function to stream the pdfs from Azure:
public MemoryStream DownloadToMemoryStream(DTO.BlobUpload b)
{
CloudStorageAccount storageAccount = Conn.SNString(b.OrgID);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(b.Container);
CloudBlockBlob blob = container.GetBlockBlobReference(b.FileName);
var sasToken = blob.GetSharedAccessSignature(new SharedAccessBlobPolicy()
{
Permissions = SharedAccessBlobPermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddMinutes(10),//assuming the blob can be downloaded in 10 miinutes
}, new SharedAccessBlobHeaders()
{
ContentDisposition = "attachment; filename=file-name"
});
using (MemoryStream ms = new MemoryStream())
{
blob.DownloadToStream(ms);
return ms;
}
}
And in the aspx.cs page the following code to read the pdf stream:
BufferedStream pdfScript = new BufferedStream(new FileStream(ScriptPath + Script, FileMode.Open));
SortedDictionary<Int32, PDFobj> objects = pdfFinalScript.Read(pdfScript);
However I get the error message: Cannot access a closed Stream
If I download the pdf to disk, this is the function I use, this work but it is not practical:
blockBlob.DownloadToFile(b.LocalPath + b.FileName, FileMode.Create);
BufferedStream pdfScript = new BufferedStream(new FileStream(ScriptPath + Script, FileMode.Open));
Thank you for your help.
Cannot access a closed Stream
According to error information, it indicates that you need to reset stream position.
Please have a try to reset the stream position before return it.
blob.DownloadToStream(ms);
ms.Position = 0; //add this code
return ms;
Updated:
ms was closed if out of using section. So please have a try to use the following code.
MemoryStream stream = new MemoryStream();
using (MemoryStream ms = new MemoryStream())
{
blob.DownloadToStream(ms);
ms.Position = 0
ms.CopyTo(stream);
stream.Position = 0;
return stream;
}
Related
Here is what I'm trying to accomplish.
I am creating an asp.net MVC application. My restrictions are that I cannot programmatically save anything to the file structure of the server, so I can't save it as a physical file on the host, and then grab it for client download.
I am loading a PDF to a stream, extracting information from the PDF, dynamically building an excel file, and then offering the file for download to the client. My code is below.
// Loads the incoming PDF document to stream
PdfDocument doc = new PdfDocument();
using (var stream = model.BudgetPdfFile.OpenReadStream())
{
doc.LoadFromStream(stream);
}
var pageCount = doc.Pages.Count;
var date = DateTime.Now.ToShortDateString().Replace("/", "-");
// Extracts data from the PDF and separates it by NewLine
SimpleTextExtractionStrategy strategy = new SimpleTextExtractionStrategy();
StringBuilder allText = new StringBuilder();
for (var i = 0; i < pageCount; i++)
{
allText.Append(doc.Pages[i].ExtractText(strategy));
}
var fullDocText = allText.ToString();
List<string> linesList = new List<string>(fullDocText.Split(new[] { Environment.NewLine }, StringSplitOptions.None).ToList());
// generates a comparison list for output data manipulation from static data
var finalList = linesList.BuildFinalList(budgetItems);
// creates a new Spire.PDF.Workbook for the final output excel file
var result = new Workbook();
// checks for whether the submitted budget is for a case in litigation or not and builds the correct excel workbook
if (model.isTrial)
{
result = ExportExcelBudget.TrialBudgetSheet(model, finalList);
}
else
{
result = ExportExcelBudget.PreTrialBudgetSheet(model, finalList);
}
Absolutely everything up to the last section below works perfectly. However, I cannot figure out how to load the workbook into a new stream and then return the file for download.
// saves the final workbook to a stream and offers it for download to the client
Stream outStream = new MemoryStream();
var fileName = "Budget Report_" + model.ClaimNumber + "_" + date + ".xlsx";
var contentType = "application/vnd.ms-excel";
result.SaveToStream(outStream, Spire.Xls.FileFormat.Version2016);
return File(outStream, contentType, fileName);
I've searched and tried multiple different variations but when the application hits the return File(), it returns a null.
I've stepped through execution and the results seem to be there, but it's not passing anything. Any help on what is wrong here would be greatly appreciated.
Stream outStream = new MemoryStream();
var fileName = "Budget Report_" + model.ClaimNumber + "_" + date + ".xlsx";
var contentType = "application/vnd.ms-excel";
result.SaveToStream(outStream, Spire.Xls.FileFormat.Version2016);
**outStream.Position = 0;**
return File(outStream, contentType, fileName);
Had to reset the stream position to 0. Working perfectly now.
I want to compress a binary file in memory using System.IO.Compression.GZipStream. For this, I am using the following method
public byte[] Encrypt()
{
var payload = GetPayload();
Console.WriteLine("[!] Payload Size: {0} bytes", payload.Length);
using (var compressedStream = new MemoryStream(payload))
using (var zipStream = new GZipStream(compressedStream, CompressionMode.Compress))
using (var resultStream = new MemoryStream())
{
zipStream.CopyTo(resultStream);
return resultStream.ToArray();
}
}
But while .CopyTo, I am getting System.NotSupportedException: Stream does not support reading.
You need to "inverse" your logic: create GZipStream over empty MemoryStream and copy your original content into this gzip stream:
using var compressedData = new MemoryStream();
using var gzip = new GZipStream(compressedData);
originalUncompressedStream.CopyTo(gzip); // <- "magic" happens here
gzip.Flush();
// and "rewind" result stream back to beginning (for next reads)
compressedData.Position = 0;
I am developing a process to compress and encrypt a byte array in my desktop application and send it via a WebMethod to my web application, then uncompress/unencrypt it back to a byte array. I am currently attempting to do this with SharpZipLib. The compression of the file seems to be working as expected. I am able to save the file to disk and extract it using 7zip without issue.
The problem I am having is when I receive the byte array on my web server and attempt to extract it.
I use the CompressData method to compress the data on the desktop side.
private byte[] CompressData(byte[] data, string password)
{
MemoryStream input = new MemoryStream(data);
MemoryStream ms = new MemoryStream();
ZipOutputStream os = new ZipOutputStream(ms);
os.SetLevel(9);
if (!string.IsNullOrEmpty(password)) os.Password = password;
ZipEntry entry = new ZipEntry("data")
{
DateTime = DateTime.Now
};
if (!string.IsNullOrEmpty(password)) entry.AESKeySize = 256;
os.PutNextEntry(entry);
StreamUtils.Copy(input, os, new byte[4096]);
os.CloseEntry();
os.IsStreamOwner = false;
os.Close();
ms.Position = 0;
return ms.ToArray();
}
I am using the following code to extract the data on the server end (taken almost verbatim from the SharpZipLib examples):
private byte[] DoRebuildData(byte[] data, string password)
{
MemoryStream inStream = new MemoryStream(data);
MemoryStream outputMemStream = new MemoryStream();
ZipOutputStream zipOut = new ZipOutputStream(outputMemStream)
{
IsStreamOwner = false // False stops the Close also Closing the underlying stream.
};
zipOut.SetLevel(3);
zipOut.Password = password; // optional
RecursiveExtractRebuild(inStream, zipOut);
inStream.Close();
// Must finish the ZipOutputStream to finalise output before using outputMemStream.
zipOut.Close();
outputMemStream.Position = 0;
return outputMemStream.ToArray();
}
// Calls itself recursively if embedded zip
//
private void RecursiveExtractRebuild(Stream str, ZipOutputStream os)
{
ZipFile zipFile = new ZipFile(str)
{
IsStreamOwner = false
};
foreach (ZipEntry zipEntry in zipFile)
{
if (!zipEntry.IsFile)
continue;
String entryFileName = zipEntry.Name; // or Path.GetFileName(zipEntry.Name) to omit folder
// Specify any other filtering here.
Stream zipStream = zipFile.GetInputStream(zipEntry);
// Zips-within-zips are extracted. If you don't want this and wish to keep embedded zips as-is, just delete these 3 lines.
if (entryFileName.EndsWith(".zip", StringComparison.OrdinalIgnoreCase))
{
RecursiveExtractRebuild(zipStream, os);
}
else
{
ZipEntry newEntry = new ZipEntry(entryFileName);
newEntry.DateTime = zipEntry.DateTime;
newEntry.Size = zipEntry.Size;
// Setting the Size will allow the zip to be unpacked by XP's built-in extractor and other older code.
os.PutNextEntry(newEntry);
StreamUtils.Copy(zipStream, os, new byte[4096]);
os.CloseEntry();
}
}
}
The expected result is to get back my original byte array on the server.
On the server, when it comes to the line:
Stream zipStream = zipFile.GetInputStream(zipEntry);
I receive the error 'No password available for AES encrypted stream.'
The only place I see to set a password is in the ZipOutputStream object, and I have checked at runtime, and this is set appropriately.
When unpacking, the password must be assigned to the password-property of the ZipFile-instance, i.e. it must be set in the RecursiveExtractRebuild-method (for this the password has to be added as an additional parameter):
zipFile.Password = password;
as shown in this example.
It should be noted that the current DoRebuildData-method doesn't actually unpack the data, but re-packs it into a new zip. The (optional) line in the DoRebuildData-method:
zipOut.Password = password;
does not specify the password for the unpacking (i.e. for the old zip), but defines the password for the new zip.
I'd like to load an image directly from a URL but without saving it on the server, I want to upload it directly from memory to Amazon S3 server.
This is my code:
Dim wc As New WebClient
Dim fileStream As IO.Stream = wc.OpenRead("http://www.domain.com/image.jpg")
Dim request As New PutObjectRequest()
request.BucketName = "mybucket"
request.Key = "file.jpg"
request.InputStream = fileStream
client.PutObject(request)
The Amazon API gives me the error "Could not determine content length". The stream fileStream ends up as "System.Net.ConnectStream" which I'm not sure if it's correct.
The exact same code works with files from the HttpPostedFile but I need to use it in this way now.
Any ideas how I can convert the stream to become what Amazon API is expecting (with the length intact)?
I had the same problem when I'm using the GetObjectResponse() method and its propertie ResponseStream to copy a file from a folder to another in same bucket. I noted that the AWS SDK (2.3.45) have some faults like a another method called WriteResponseStreamToFile in GetObjectResponse() that simply doesn't work. These lacks of functions needs some workarounds.
I solved the problem openning the file in array of bytes and putting it in a MemoryStream object.
Try this (C# code)
WebClient wc = new WebClient();
Stream fileStream = wc.OpenRead("http://www.domain.com/image.jpg");
byte[] fileBytes = fileStream.ToArrayBytes();
PutObjectRequest request = new PutObjectRequest();
request.BucketName = "mybucket";
request.Key = "file.jpg";
request.InputStream = new MemoryStream(fileBytes);
client.PutObject(request);
The extesion method
public static byte[] ToArrayBytes(this Stream input)
{
byte[] buffer = new byte[16 * 1024];
using (MemoryStream ms = new MemoryStream())
{
int read;
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms.ToArray();
}
}
You can also create a MemoryStream without an array of bytes. But after the first PutObject in S3, the MemoryStream will be discarted. If you need to put others objects, I recommend the first option
WebClient wc = new WebClient();
Stream fileStream = wc.OpenRead("http://www.domain.com/image.jpg");
MemoryStream fileMemoryStream = fileStream.ToMemoryStream();
PutObjectRequest request = new PutObjectRequest();
request.BucketName = "mybucket";
request.Key = "file.jpg";
request.InputStream = fileMemoryStream ;
client.PutObject(request);
The extesion method
public static MemoryStream ToMemoryStream(this Stream input)
{
byte[] buffer = new byte[16 * 1024];
int read;
MemoryStream ms = new MemoryStream();
while ((read = input.Read(buffer, 0, buffer.Length)) > 0)
{
ms.Write(buffer, 0, read);
}
return ms;
}
I had the same problem in a similar scenario.
The reason for the error is that to upload an object the SDK needs to know the whole content length that is going to be uploaded. To be able to obtain stream length it must be seekable, but the stream returned from WebClient is not. To indicate the expected length set Headers.ContentLength in PutObjectRequest. The SDK will use this value if it cannot determine length from the stream object.
To make your code work, obtain content length from the response headers returned by the call made by WebClient. Then set PutObjectRequest.Headers.ContentLength. Of course this relies on the server returned content length value.
Dim wc As New WebClient
Dim fileStream As IO.Stream = wc.OpenRead("http://www.example.com/image.jpg")
Dim contentLength As Long = Long.Parse(client.ResponseHeaders("Content-Length"))
Dim request As New PutObjectRequest()
request.BucketName = "mybucket"
request.Key = "file.jpg"
request.InputStream = fileStream
request.Headers.ContentLength = contentLength
client.PutObject(request)
I came up with a solution that uses UploadPart when the length is not available by any other means, plus this does not load the entire file into memory.
if (args.DocumentContents.CanSeek)
{
PutObjectRequest r = new PutObjectRequest();
r.InputStream = args.DocumentContents;
r.BucketName = s3Id.BucketName;
r.Key = s3Id.ObjectKey;
foreach (var item in args.CustomData)
{
r.Metadata[item.Key] = item.Value;
}
await S3Client.PutObjectAsync(r);
}
else
{
// if stream does not allow seeking, S3 client will throw error:
// Amazon.S3.AmazonS3Exception : Could not determine content length
// as a work around, if cannot use length property, will chunk
// file into sections and use UploadPart, so do not have to load
// entire file into memory as a single MemoryStream.
var r = new InitiateMultipartUploadRequest();
r.BucketName = s3Id.BucketName;
r.Key = s3Id.ObjectKey;
foreach (var item in args.CustomData)
{
r.Metadata[item.Key] = item.Value;
}
var multipartResponse = await S3Client.InitiateMultipartUploadAsync(r);
try
{
var completeRequest = new CompleteMultipartUploadRequest
{
UploadId = multipartResponse.UploadId,
BucketName = s3Id.BucketName,
Key = s3Id.ObjectKey,
};
// just using this size, because it is the max for Azure File Share, but it could be any size
// for S3, even a configured value
const int blockSize = 4194304;
// BinaryReader gives us access to ReadBytes
using (var reader = new BinaryReader(args.DocumentContents))
{
var partCounter = 1;
while (true)
{
byte[] buffer = reader.ReadBytes(blockSize);
if (buffer.Length == 0)
break;
using (MemoryStream uploadChunk = new MemoryStream(buffer))
{
uploadChunk.Position = 0;
var uploadRequest = new UploadPartRequest
{
BucketName = s3Id.BucketName,
Key = s3Id.ObjectKey,
UploadId = multipartResponse.UploadId,
PartNumber = partCounter,
InputStream = uploadChunk,
};
// could call UploadPart on multiple threads, instead of using await, but that would
// cause more data to be loaded into memory, which might be too much
var part2Task = await S3Client.UploadPartAsync(uploadRequest);
completeRequest.AddPartETags(part2Task);
}
partCounter++;
}
var completeResponse = await S3Client.CompleteMultipartUploadAsync(completeRequest);
}
}
catch
{
await S3Client.AbortMultipartUploadAsync(s3Id.BucketName, s3Id.ObjectKey
, multipartResponse.UploadId);
throw;
}
}
I have the following code which takes an improperly saved Image from the database converts it to a Jpeg and returns the Image in a byte array;
public Byte[] GetImageFromDB(int id)
{
var imageData = _repository.GetImage(id);
var newImageData = ConvertCorruptedImage(imageData, id);
return newImageData;
}
private byte[] ConvertCorruptedImage(byte[] imageData, int id)
{
// Save DB Image as a file.
MemoryStream img = new MemoryStream(imageData);
var saveDBImage = Image.FromStream(img);
string originalFileName = #"c:\original_" + id.ToString() + ".jpg";
string newFileName = #"C:\new" + id.ToString() + ".jpg";
// Delete if already Exists
DeleteImageFile(originalFileName);
saveDBImage.Save(originalFileName);
// Read Saved DB Image From Saved File & Save as jpeg
Bitmap bm = new Bitmap(originalFileName);
bm.Save(newFileName , ImageFormat.Jpeg);
// Return Converted JPEG Image
var newImage = ImageToByte(Image.FromFile(newFileName));
//DeleteCreatedImage(newFileName);
//DeleteCreatedImage(originalFileName);
return newImage;
}
private byte[] ImageToByte(Image img)
{
ImageConverter converter = new ImageConverter();
return (byte[])converter.ConvertTo(img, typeof(byte[]));
}
public static void DeleteImageFile(string fileName)
{
FileInfo file = new FileInfo(fileName);
if (file.Exists && !file.IsReadOnly)
{
System.IO.File.Delete(fileName);
}
}
I was wondering if there was a way to do this without saving a file to the hard disk or if i do save it then deleting it once i am done with it.
I've tried adding a delete for each images (check the commented out portion of the ConvertCorruptedImage method) but i keep getting the following error:
The process cannot access the file 'C:\new_xx.jpg' because it is being used by another process.
I really don't want to be saving images to a hard disk.
Thanks in advance
something along the lines of
var image = Image.FromStream(new MemoryStream(imageData));
Bitmap bmp = new Bitmap(image);
MemoryStream outStream = new MemoryStream();
bmp.Save(outStream,ImageFormat.Jpeg);
return outStream.ToArray();
Use the overload of Bitmap.Save that writes to a Stream.
var stream = new MemoryStream();
bm.Save(stream, ImageFormat.Jpeg);
You can load the bitmap directly from your MemoryStream:
Bitmap bm = new Bitmap(imgStream);
You can also save the bitmap to a stream:
MemoryStream newImgStream = new MemoryStream();
bm.Save(newMemoryStream, ImageFormat.Jpeg);