I am using ZipOutputStream to get files from db, compress to zip and download (zip file around 400mb).
During this, my app pool memory is going up to 1.4gb and after download is complete, its coming down to 1gb when it should come back to like 100 mb or something.
There are only like 10 users using this app and only 1 user using this particular page.
i am calling the dispose method. I aslo tried explictly calling GC.Collect but still no use.
Am i missing anything here?
Thanks in advance.
Dim zipStream = New ZipOutputStream(HttpContext.Current.Response.OutputStream)
Try
da.Fill(ds)
For Each dr As DataRow In ds.Tables(0).Rows
Try
Dim docName As String = ""
strImgID = dr("image_id")
If Not IsDBNull(dr("loan_number")) Then iLoanID = dr("loan_number")
If Not IsDBNull(dr("document_name")) Then docName = dr("document_name")
Dim ext As String = dr("image_type_extension")
Dim strFinalFileName As String = ""
strFinalFileName = docName & "_" & iLoanID & ext
Dim b As Byte() = dr("image_binary")
Dim fileEntry = New ZipEntry(Path.GetFileName(strFinalFileName))
zipStream.PutNextEntry(fileEntry)
zipStream.Write(b, 0, b.Length)
Catch ex As Exception
LogError(ex, iLoanID & "," & strImgID)
AddError(sb, ex, iLoanID & "," & strImgID)
End Try
Next
Catch ex As Exception
Throw
Finally
zipStream.Close()
zipStream.Dispose()
cmd.Connection.Close()
cmd.Connection.Dispose()
End Try
You need to chunk data into the stream rather than allocate all at once.
E.g. (in c#)
byte[] buffer = new byte[4096];
FileStream readFs = File.OpenRead(strFile);
for (int rnt = readFs.Read(buffer, 0, buffer.Length);
rnt > 0;
rnt = readFs.Read(buffer, 0, buffer.Length))
{
zipoutputstream.Write(buffer, 0, rnt);
}
I think this will help with your memory issue. Please comment back if not..
Related
This is baffling me. On a new server (Windows Server 2016) my images seem to be loading in synchronously. The browser is attempting to download them all at the same time, but something is stopping the image from loading until the previous one has finished. The time to first byte increases as each image is attempted to be loaded.
EDIT: I have posted my imagehandler code below, but you can see the problem exists when loading .jpg in its native format (see screenshot about half way down).
EDIT 2: I have disabled http/2 and added a screenshot.
My SVG image handler code (problem exists with image.ashx also):
Dim freshness As TimeSpan = New TimeSpan(1, 0, 0, 0)
Dim now As DateTime = DateTime.Now
Response.Cache.SetExpires(now.Add(freshness))
Response.Cache.SetMaxAge(freshness)
Response.Cache.SetCacheability(HttpCacheability.Public)
Response.Cache.SetValidUntilExpires(True)
Response.Cache.VaryByParams("colour") = True
Response.Cache.VaryByParams("colour2") = True
Response.Cache.VaryByParams("svg") = True
Response.Cache.VaryByParams("rnd") = True
Response.Cache.VaryByHeaders("rnd") = True
Response.Cache.VaryByHeaders("Type") = True
Dim sSVG As String = Request.QueryString("svg").Split("-")(0)
If sSVG = String.Empty Then
Try
sSVG = Request.ServerVariables("SCRIPT_NAME").Split("-")(0)
Catch ex As Exception
sSVG = "svg"
End Try
End If
Dim xDoc As XDocument = XDocument.Load(Server.MapPath("\images\icons\svg\" & sSVG & ".svg"))
Dim sColour As String = "FFFFFF"
Dim sColour2 As String = "FFFFFF"
Dim sXML As String = xDoc.Document.ToString
If Request.QueryString("Colour") <> String.Empty Then
sColour = Request.QueryString("Colour")
Else
Try
sColour = sSVG.Split("-")(1)
Catch ex As Exception
End Try
End If
If Request.QueryString("Colour2") <> String.Empty Then
sColour2 = Request.QueryString("Colour2")
Else
Try
sColour = sSVG.Split("-")(2)
Catch ex As Exception
End Try
End If
sXML = sXML.Replace("COLOUR2", sColour2)
sXML = sXML.Replace("COLOUR", sColour)
Response.Headers.Add("Vary", "Accept-Encoding")
Response.ContentType = "image/svg+xml"
Response.Write(sXML)
HTTP/2 disabled:
I am using this part of code to download file using the onClick event for imageButton inside UpdatePanel, it works fine if I add the imageButton as PostBackTrigger for the updatePanel, is there away to do that with out full postback?
Dim stream As IO.Stream = Nothing
Dim bytesToRead As Integer = 10000
Dim buffer As Byte() = New Byte(bytesToRead - 1) {}
Try
Dim fileReq As Net.HttpWebRequest = CType(Net.HttpWebRequest.Create(FileURL), Net.HttpWebRequest)
Dim fileResp As Net.HttpWebResponse = CType(fileReq.GetResponse(), Net.HttpWebResponse)
If fileReq.ContentLength > 0 Then fileResp.ContentLength = fileReq.ContentLength
stream = fileResp.GetResponseStream()
Dim resp = HttpContext.Current.Response
resp.ContentType = "application/octet-stream"
resp.AddHeader("Content-Disposition", "attachment; filename=""" & FileName & """")
resp.AddHeader("Content-Length", fileResp.ContentLength.ToString())
Dim length As Integer
Do
If resp.IsClientConnected Then
length = stream.Read(buffer, 0, bytesToRead)
resp.OutputStream.Write(buffer, 0, length)
resp.Flush()
buffer = New Byte(bytesToRead - 1) {}
Else
length = -1
End If
Loop While length > 0
Finally
If stream IsNot Nothing Then
stream.Close()
End If
End Try
I found the solution.
#HATCHA's answer on Asp:progress won't end anitmation when file is downloaded from ashx file was great.
what #VDWWD said was wonderful , But not in my case, because I need a button to create the pdf file before starting the download. which means the client side click won't be useful.
ScriptManager.RegisterClientScriptBlock(Me, GetType(Page), "downloadFileExport", "javascript:window.location ='" & ResolveUrl("~/report/download.ashx?url=" & url & "&fName=" & fName) & "';", True)
Reading an JSON document "source.json" thru filestream but how do you get the data from the json file? After that I am trying to append the newly edited json data back on the same file.
Dim pathSource As String = "Server.MapPath('~/source.json')"
Try
Using fs As FileStream = New FileStream(pathSource, _
FileMode.Open, FileAccess.Read)
Dim bytes() As Byte = New Byte((fsSource.Length) - 1) {}
Dim numBytesToRead As Integer = CType(fsSource.Length,Integer)
Dim numBytesRead As Integer = 0
While (numBytesToRead > 0)
Dim n As Integer = fsSource.Read(bytes, numBytesRead, _
numBytesToRead)
If (n = 0) Then
Exit While
End If
numBytesRead = (numBytesRead + n)
numBytesToRead = (numBytesToRead - n)
End While
numBytesToRead = bytes.Length
Dim xmlBuilder = New StringBuilder()
fs.Seek(0, SeekOrigin.Begin)
Dim ms As New MemoryStream()
fs.CopyTo(ms)
xmlBuilder.Append(Encoding.UTF8.GetString(ms.ToArray()))
ms.Flush()
ms.Close()
'???How to access the data from the file "source.json" you just read in???
'Edit the file "source.json" data
?? How to put it into "bytesout" the edited data???
Using fsAppend As FileStream = New FileStream(pathSource, _
FileMode.Append, FileAccess.Write)
fsAppend.Write(bytesout, 0, numBytesToRead)
End Using
End Using
Catch ioEx As FileNotFoundException
Console.WriteLine(ioEx.Message)
End Try
I've been down that path. Try to figure out what you did wrong with newtonsoft. The serializer and deserializer methods are fairly easy to use. You only have to create an object per the Json format and deserializer will populate the object automatically. You just have to have the variables in the class be declared as public property nameofjsonobject as
I am trying to download a webpage as string. Can someone please explain why the following code doesn't work?
Dim URL As String = "http://stackoverflow.com/"
Dim client As WebClient = New WebClient()
Dim data As Stream = client.OpenRead(URL)
Dim reader As StreamReader = New StreamReader(data)
Dim str As String = ""
str = reader.ReadLine()
Do While str.Length > 0
Console.WriteLine(str)
str = reader.ReadLine()
Loop
When I run it it never goes inside the loop.
Give this a shot...
Dim URL As String = "http://stackoverflow.com/"
Dim html As String = New WebClient().DownloadString(URL)
Better Solution (Releases resources that the network stack is using. Also ensure's (hope) the CLR cleans these up when needed.)
Using client As New WebClient()
html = client.DownloadString(URL)
End Using
I have a website where a user can choose to view a video (no copyright) that is on a 3rd party website. From the moment the user choose it, it needs to be downloaded onto my web server before the user can access it. I tried to automatically start the download on the 3rd party website and make a response.redirect with this path, but when the user is watching the video, if the video has only 10 seconds downloaded, any player/browser will considere this video as a ten-seconds video and stop it after 10 seconds.
What would be the best practice to redirect a stream instead of a file?
I found that this piece of code is doing exactly what I want. It downloads a file on a third party website, and as it's streaming it, each chunk is writen in the Response so that it completely obsfuscates the origin. Thus, it is possible to serve any file from any website as if I own it.
Private Sub SendFile(ByVal url As String)
Dim stream As System.IO.Stream = Nothing
Dim bytesToRead As Integer = 10000
Dim buffer() As Byte = New Byte((bytesToRead) - 1) {}
Try
Dim fileReq As System.Net.WebRequest = CType(System.Net.HttpWebRequest.Create(url), System.Net.HttpWebRequest)
Dim fileResp As System.Net.HttpWebResponse = CType(fileReq.GetResponse, System.Net.HttpWebResponse)
If (fileReq.ContentLength > 0) Then
fileResp.ContentLength = fileReq.ContentLength
End If
stream = fileResp.GetResponseStream
Dim resp As System.Web.HttpResponse = HttpContext.Current.Response
resp.ContentType = "application/octet-stream"
resp.AddHeader("Content-Disposition", ("attachment; filename=\""" + ("mp3" + "\""")))
resp.AddHeader("Content-Length", fileResp.ContentLength.ToString)
Dim length As Integer = 1000000
While (length > 0)
If resp.IsClientConnected Then
length = stream.Read(buffer, 0, bytesToRead)
resp.OutputStream.Write(buffer, 0, length)
resp.Flush()
buffer = New Byte((bytesToRead) - 1) {}
Else
length = -1
End If
End While
Catch
stream.Close()
Finally
If (Not (stream) Is Nothing) Then
stream.Close()
End If
End Try
Response.End()
End Sub