MS VS 2008, ASp.Net 3.5.
On the client side :
client selects start and end dates, report format as Excel, clicks "run report" button
On that click redirected to reportToExcel.aspx, in reportToExcel.aspx.vb in Page_Load event stored procedure is executed to retrieve report data :
oSQLDataReader = oSqlCommand.ExecuteReader()
Then:
Response.ContentType = "application/ms-excel"
Response.AddHeader("Content-Disposition", "attachment; filename=" + MyBase.UserSession.ReportName + ".xls")
Then Response.Write is used to write retrieved report data into Response object in XML format, like
Response.Write("<td>" & FormatColumnValue(oSQLDataReader.GetValue(I), arrColHeader(I + 1).ColumnFormat) & "</td>"), etc. Last callis Response.End().
I know Response.End should not be used, I plan to substitute is with
context.Response.Flush()
context.ApplicationInstance.CompleteRequest()
but I doubt it will improve response time.
Problem: on the client side take 6 mins to receive 32.5 MB of data. This is too long.
How to reduce this time ?
As I understood so far: chuncking is not possible for Excel report and anyway client wants to receive Excel report as one and whole.
In order to use Response.TransferFile : Excel file has to be created first, then zipped to reduce amount of data to download, then downloaded. For this to work Excel should be installed on the server, which is not acceptable in our case.
Deliver data as csv to client is not acceptable: client will have to import it to Excel, which they would not like to do.
Stored procedure executed from SQL management studio shows inconsistent run times: from 12 secs to 4 mins.
So, are there any other ways to reduce report 'delivery' time to the client ?
Thank you for all replies
You do need to get the stored procedure run time down. But that's a whole question and answer in itself.
Your method of writing out the HTML is slower than it needs to be. Essentially, you are doing repeated string concatenations, which are slow. Consider using a StringBuilder to construct the entire document before writing it to the Response stream.
Another option (and perhaps a better one) would be to try something like the free Excel Xml Writer library: http://www.carlosag.net/tools/excelxmlwriter/. I haven't used it, but I've heard good things about it. This would (I believe) let you write your Excel file on the server without needing Excel itself installed.
Related
Current situation
I have ASP.NET web application that render PDF for users using MS Report Viewer. The PDF is rendered with this method:
Byte pdfByte = Byte();
pdfByte = ReportViewer.LocalReport.Render("PDF", Nothing, mimeType, encoding, extensions, stream, warning)
And send to browser as an attachment with response object:
Response.Clear()
Response.ContentType = mimeType
Response.AddHeader("content-disposition", "attachment; filename=myfile." + extension)
Response.BinaryWrite(pdfByte)
Response.Flush()
Response.End()
This work great! The user browser will get the PDF as download-able attachment.
What I am trying to achieve
Render multiple PDF and send all of them separately to user's browser. User will get separate PDF documents. It doesn't matter whether they will get them all at once or one by one.
The problem
The problem is after Response.End() the next line of code is not executed. I have tried to store the pdfByte object in session, looping through it and send them to user's browser with Response object but after the first PDF get sent then it stop.
I have also tried removing Response.End() thinking the code will keep running but still it stop after the first PDF get sent.
Please advice any workaround or tips. Thanks!
You cannot send multiple files (as separate entities) in a single HTTP response (the protocol does not support it. However, what you can do is to archive all files together and send the that single zip (or whatever format you want) to the client.
You can use libraries such as DotNetZip/SharpZipLib to combine (and compress) files together. Based on library API, you may need to save PDF files to disk before adding to zip file. Also do not forget to change your content type appropriate while sending the zip file to client.
Yet another alternative is to provide user with a page having multiple links to download files. It may mean that you either have to store your PDFs for some time so that they can served later (via links) or make link point to a handler that will re-run the report again to get the PDF out of it.
Admittedly the method I'm using doesn't feel very elegant, but here's what I'm doing:
create one IFRAME on the page for each document you want to send to the client (maybe create the IFRAMEs dynamically in server-side code if the number of documents is variable);
create a HttpHandler that generates the PDF documents, depending on a parameter you're passing in through the QueryString, just like you're doing above;
set the src on all IFRAMES to the URL of the HttpHandler with the appropriate parameters attached.
Of course the HttpHandler needs to do implement security logic, if required.
This works quite beautifully: If I want to send 3 documents, I create 3 IFRAMEs, set their src, and the user will see 3 "Save As..." dialogs pop up.
I need to export huge amount of data from ado.net datatable(which i get by db query) to excel.
I tried the following way :
1. Create excel object with workbook/worksheet # server side...and use memory stream to write whole document to client side.
But this gave me "out of Memory exception". bcoz my memory stream was was so huge.
So I replaced this with a new way - as follows :
Writing each row from datatable as a coma seperated string to client side.
So, as and when we get each row ...we can write to client side ..no memory is used.
But by this way we can write to csv file...not to excel...
Does anybody know how to handle this situation.
Can I use silverlight to get data row by row from server, pass it to client side...build excel at client side.?
try spreadsheetgear
OR
smartxls
I'd keep the csv approach but write to a file and not the memory strea. After you've created the file, I'd use TransmitFile to get it to the browser. You can see more about using TransmitFile here.
How should I go about providing download functionality on an asp.net page to download a series of rows from a database table represented as a linq2sql class that only has primitive types for members (ideally into a format that can be easily read by Excel)?
E.g.
public class Customer
{
public int CustomerID;
public string FirstName;
public string LastName;
}
What I have tried so far.
Initially I created a DataTable, added all the Customer data to this table and bound it to a DataGrid, then had a download button that called DataGrid1.RenderControl to an HtmlTextWriter that was then written to the response (with content type "application/vnd.ms-excel") and that worked fine for a small number of customers.
However, now the number of rows in this table is >10,000 and is expected to reach upwards of 100,000, so it is becoming prohibitive to display all this data on the page before the user can click the download button.
So the question is, how can I provide the ability to download all this data without having to display it all on a DataGrid first?
After the user requests the download, you could write the data to a file (.CSV, Excel, XML, etc.) on the server, then send a redirect to the file URL.
I have used the following method on Matt Berseth blog for large record sets.
Export GridView to Excel
If you have issues with the request timing out try increasing the http request time in the web.config
Besides the reasonable suggestion to save the data on server first to a file in one of the answers here, I would like to also point out that there is no reason to use a DataGrid (it’s one of you questions as well). DataGrid is overkill for almost anything. You can just iterate over the records, and save them directly using HtmlTextWriter, TextWriter (or just Response.Write or similar) to the server file or to a client output stream. It seems to me like an obvious answer, so I must be missing something.
Given the number of records, you may run into a number of problems. If you write directly to the client output stream, and buffer all data on server first, it may be a strain on the server. But maybe not; it depends on the amount of memory on the serer, the actual data size and how often people will be downloading the data. This method has the advantage of not blocking a database connection for too long. Alternatively, you can write directly to the client output stream as you iterate. This may block the database connection for too long as it depends on the download speed of the client. But again; it your application is of a small or medium size (in audience) then anything is fine.
You should definitely check out the FileHelpers library. It's a freeware, excellent utility set of classes to handle just this situation - import and export of data, from text files; either delimited (like CSV), or fixed width.
It offer a gazillion of options and ways of doing things, and it's FREE, and it works really well in various projects that I'm using it in. You can export a DataSet, an array, a list of objects - whatever it is you have.
It even has import/export for Excel files, too - so you really get a bunch of choices.
Just start using FileHelpers - it'll save you so much boring typing and stuff, you won't believe it :-)
Marc
Just a word of warning, Excel has a limitation on the number of rows of data - ~65k. CSV will be fine, but if your customers are importing the file into Excel they will encounter that limitation.
Why not allow them to page through the data, perhaps sorting it before paging, and then give them a button to just get everything as a cvs file.
This seems like something that DLinq would do well, both the paging, and writing it out, as it can just fetch one row at a time, so you don't read in all 100k rows before processing them.
So, for cvs, you just need to use a different LINQ query to get all of the rows, then start to save them, separating each cell by a separator, generally a comma or tab. That could be something picked by the user, perhaps.
OK, I think you are talking too many rows to do a DataReader and then loop thru to create the cvs file. The only workable way will be to run:
SQLCMD -S MyInstance -E -d MyDB -i MySelect.sql -o MyOutput.csv -s
For how to run this from ASP.Net code see here. Then once that is done, your ASP.Net page will continue with:
string fileName = "MyOutput.csv";
string filePath = Server.MapPath("~/"+fileName);
Response.Clear();
Response.AppendHeader("content-disposition",
"attachment; filename=" + fileName);
Response.ContentType = "application/octet-stream";
Response.WriteFile(filePath);
Response.Flush();
Response.End();
This will give the user the popup to save the file. If you think more than one of these will happen at a time you will have to adjust this.
So after a bit of research, the solution I ended up trying first was to use a slightly modified version of the code sample from http://www.asp.net/learn/videos/video-449.aspx and format each row value in my DataTable for CSV using the following code to try to avoid potentially problematic text:
private static string FormatForCsv(object value)
{
var stringValue = value == null ? string.Empty : value.ToString();
if (stringValue.Contains("\"")) { stringValue = stringValue.Replace("\"", "\"\""); }
return "\"" + stringValue + "\"";
}
For anyone who is curious about the above, I'm basically surrounding each value in quotes and also escaping any existing quotes by making them double quotes. I.e.
My Dog => "My Dog"
My "Happy" Dog => "My ""Happy"" Dog"
This appears to be doing the trick for now for small numbers of records. I will try it soon with the >10,000 records and see how it goes.
Edit: This solution has worked well in production for thousands of records.
I was wondering what's the best practise for serving a generated big file in classic asp.
We have an application with "export to excel" function that produces 10MB files. The excels are created by just calling a .asp page that has the Response.ContentType set to excel and has an HTML table for the data.
This gives as problem that it takes 4 minutes before the user sees the "Save as..." dialog.
My current solution is to call an .asp page that creates the excel on the server with AJAX and lets the page return the URL of the generated document. Then I can use javascript to display the on the original page.
Is this easy to do with classic asp (creating files on server with some kind of stream) while keeping security in mind? (URL should make people be able to guess the location of other files)
How would I go about handling deleted the generated files overtime? They have to be deleted periodicly as the data changes in realtime.
Thanks.
edit: I realized now that creating the file on the server will probably also take 4 minutes...
I think you are selecting a complex route, when the solution is simple enough (Though I may be missing some requirements)
If you to generate an excel, just call an asp page that do the following:
Response.clear
Response.AddHeader "content-disposition", "attachment; filename=myexcel.xls"
Response.ContentType = "application/excel"
'//write the content of the file
Response.write "...."
Response.end
This will a start a download process in the browser without needing to generate a extra call, javascript or anything
See this question for more info on the format you will choose to generate the excel.
Edit
Since Thomas update the question and the real problem is that the file take 4 minutes to generate, the solution could be:
Offer the user the send the file by email (if this is a workable solution in you server or hosting).
Generate the file async, and let the user know when the file generation is done (with an ajax call, like SO does when other user have added an answer)
To generate the file on the server
'//You should change for a random name or something that makes sense
FileName = "C:\temp\myexcel.xls"
FileNumber = FreeFile
Open FileName For Append As #FileNumber
'//generate the content
TheRow = "...."
Print #FileNumber, TheRow
Close #FileNumber
To delete the temp files generated
I use Empty Temp Folders a freeware app that I run daily on the server to take care of temp files generated. (Again, it depends on you server or hosting)
About security
Generate the files using random numbers or GUIds for a light protection. If the data is sensitive, you will need to download the file from a ASP page, but I think that you will be in the same problem again...(waiting 4 minutes to download)
Read file using FSO.
Set headers for Excel file-type, name according to file read and for download (attachment)
Flush response after headers are set. The client should display "save as" dialogue.
Output FSO to response. Client will download file and see progress bar.
How do you plan to generate the Excel? I hope you don't plan to call Excel to do that, as it is unsupported, and generally won't work well.
You should check to see if there are COM components to generate Excel that you can call from Classic ASP. Alternatively, add one ASP.NET page for the purpose. I know for a fact that there are compoonents that can be called from ASP.NET pages to do this. Worse come to worst, there's an Excel exporter component from Infragistics that works with their UltraWebGrid control to export. The grid need not be visible in order to accomplish this, but styles in the grid translate to styles in the spreadsheet. They also allow you to manipulate the spreadsheet programmatically.
How can I export the data in my webapp to an Excel sheet from ASP.NET (VB.NET,SQL 2005)?
change the contenttype of your ASP.Net page
Response.ContentType = "application/ms-excel"
One of my most popular blogs is how to generate an Excel document from .NET code, using the Excel XML markup (this is not OpenXML, it's standard Excel XML) - http://www.aaron-powell.com/linq-to-xml-to-excel
I also link off to an easier way to do it with VB 9.
Although this is .NET 3.5 code it could easily be done in .NET 2.0 using XmlDocument and creating the nodes that way.
Then it's just a matter to set the right response headers and streaming back in the response.
SpreadsheetGear for .NET will do it. You can find a bunch of live ASP.NET samples with C# & VB.NET source on this page.
Disclaimer: I own SpreadsheetGear LLC
If you can display your data in a GridView control, it inherently supports "right-click-->Export to Excel" without having to write any code whatsoever.
SQL Server Reporting services would be the best way to export data from an application into Excel.
If you dont have access to / dont wan't to use reporting services depending on the data you want to extract / format possibly using a CSV structure instead of Excel may be easiest.
Use the Microsoft.Office.Interop.Excel dlls to create excel files with your data and then provide links to download the files using Hunter Daley's download method...
As a general solution, you may want to consider writing handler (ashx) for exporting -- and pass either the query parameters to recreate the query to generate the data or an identifier to get the data from the cache (if cached). Depending on whether CSV is sufficient for your Excel export you could just format the data and send it back, setting the ContentType as #Hunter suggests or use the primary interop assemblies (which would require Excel on the server) to construct a real Excel spreadsheet and serialize it to the response stream.
I prefer to use a OLEDB connection string.
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\Excel.xls;Extended Properties="Excel 8.0;HDR=Yes;IMEX=1";
Not sure about exporting a page but if you just want to export a dataset or datatable
HttpContext.Current.Response.Clear()
HttpContext.Current.Response.AddHeader("content-disposition", String.Format("attachment; filename={0}", fileName))
HttpContext.Current.Response.ContentType = "application/ms-excel"
Dim sw As StringWriter = New StringWriter
Dim htw As HtmlTextWriter = New HtmlTextWriter(sw)
Dim table As Table = New Table
table.RenderControl(htw)
' render the htmlwriter into the response
HttpContext.Current.Response.Write(sw.ToString)
HttpContext.Current.Response.End()
I use almost the same exact code as CodeKiwi. I would use that if you have a DataTable and want to stream it to the client browser.
If you want a file, you could also do a simple loop through each row/column, create a CSV file and I guess provide a link to the client - you can use a file extension of CSV or XLS. Or if you stream the resulting file to the client it will prompt them if they want to open or save it to disk.
The interops are (well were last time I tried them) great for small datasets, but didn't scale well - horrifically slow for larger datasets.