I'm fairly new to BizTalk and creating a custom pipeline component. I have seen code in examples that are similar to the following:
public void Disassemble(IPipelineContext pContext, IBaseMessage pInMsg)
{
Stream originalDataStream = pInMsg.BodyPart.GetOriginalDataStream();
StreamReader strReader = new StreamReader(originalDataStream);
string strOriginalData = strReader.ReadToEnd();
byte[] bufferOriginalMessage = new byte[strOriginalData.Length];
bufferOriginalMessage = ASCIIEncoding.Default.GetBytes(strOriginalData);
Stream ms = new MemoryStream();
ms.Write(bufferOriginalMessage, 0, strOriginalD
//other stuff here
ms.Seek(0, SeekOrigin.Begin);
pInMsg.BodyPart.Data = ms;
}
But nowhere in the method is the StreamReader being closed or disposed. The method simply exits.
Normally when using StreamReader and other classes, it is best practice to use a using statement so that the stream is automatically disposed.
Is there a particular reason (perhaps in BizTalk) why you wouldn't dispose this StreamReader?
I have not found any information on this point. Can anyone help?
In general, yes, it's a good practice to close readers and streams you don't need anymore. That said, there might not necessarily be 100% required everytime. For example, closing the reader would close the underlying stream normally, but chances are, something else is probably already aware of the stream and will close it at the right time on it's own.
What is good practice, however, is to add any streams you use in a pipeline component with a lifetime matching that of the message to the resource tracker, so that BizTalk can dispose them automatically when the pipeline execution finishes and the message has been processed.
Related
I am working with a legacy ASP.NET Framework MVC application that is experiencing issues with memory usage such as occasional bursts of OutOfMemory exceptions across a variety of operations. The application is frequently operating with large lists of objects (sometimes 10s to 100s of megabytes), and then serializing them to JSON to return to the client. We are not totally sure what the source of the OutOfMemory exceptions is, but believe a likely candidate is memory fragmentation due to too many large objects going on the Large Object Heap.
We are thinking a quick win is to refactor some of the controller endpoints to serialize their JSON content using a stream writer (as outlined in the JSON.NET Documentation), and to stream the content back to the browser. This won't eliminate the memory load of the data lists prior to serialization, but in theory should cut down the overall amount of data going on to the LOH.
The code is written to send the results in chunks of less than 85kb:
public async Task<ActionResult> MyControllerMethod()
{
var data = GetData();
Response.BufferOutput = false;
Response.ContentType = "application/json";
var serializer = JsonSerializer.Create();
using (var sw = new StreamWriter(Response.OutputStream, Encoding.UTF8, 84999))
{
sw.AutoFlush = false;
serializer.Serialize(sw, data);
await sw.FlushAsync();
}
return new EmptyResult();
}
I am aware of a few downsides with this approach, but don't consider them showstoppers:
More complex to implement a unit test due to the 'EmptyResult' returned by the controller.
I have read there is a small overhead due to a call to PInvoke whenever data is flushed. (In practice I haven't noticed this).
Cannot post-process the content using e.g. an HttpHandler
Cannot set a content-length header which may be useful for the client in some cases.
What other downsides or potential problems exist with this approach?
I've the following code
using(MemoryStream ms = new MemoryStream())
{
//code
return 0;
}
The dispose() method is called at the end of using statement braces } right? Since I return before the end of the using statement, will the MemoryStream object be disposed properly? What happens here?
Yes, Dispose will be called. It's called as soon as the execution leaves the scope of the using block, regardless of what means it took to leave the block, be it the end of execution of the block, a return statement, or an exception.
As #Noldorin correctly points out, using a using block in code gets compiled into try/finally, with Dispose being called in the finally block. For example the following code:
using(MemoryStream ms = new MemoryStream())
{
//code
return 0;
}
effectively becomes:
MemoryStream ms = new MemoryStream();
try
{
// code
return 0;
}
finally
{
ms.Dispose();
}
So, because finally is guaranteed to execute after the try block has finished execution, regardless of its execution path, Dispose is guaranteed to be called, no matter what.
For more information, see this MSDN article.
Addendum:
Just a little caveat to add: because Dispose is guaranteed to be called, it's almost always a good idea to ensure that Dispose never throws an exception when you implement IDisposable. Unfortunately, there are some classes in the core library that do throw in certain circumstances when Dispose is called -- I'm looking at you, WCF Service Reference / Client Proxy! -- and when that happens it can be very difficult to track down the original exception if Dispose was called during an exception stack unwind, since the original exception gets swallowed in favor of the new exception generated by the Dispose call. It can be maddeningly frustrating. Or is that frustratingly maddening? One of the two. Maybe both.
using statements behave exactly like try ... finally blocks, so will always execute on any code exit paths. However, I believe they are subject to the very few and rare situations in which finally blocks are not called. One example that I can remember is if the foreground thread exits while background threads are active: all threads apart from the GC are paused, meaning finally blocks are not run.
Obvious edit: they behave the same apart from the logic that lets them handle IDisposable objects, d'oh.
Bonus content: they can be stacked (where types differ):
using (SqlConnection conn = new SqlConnection("string"))
using (SqlCommand comm = new SqlCommand("", conn))
{
}
And also comma-delimited (where types are the same):
using (SqlCommand comm = new SqlCommand("", conn),
comm2 = new SqlCommand("", conn))
{
}
Your MemoryStream object will be disposed properly, no need to worry about that.
With the using statement, the object will be disposed of regardless of the completion path.
Further reading...
http://aspadvice.com/blogs/name/archive/2008/05/22/Return-Within-a-C_2300_-Using-Statement.aspx
http://csharpfeeds.com/post/8451/Return_Within_a_Csharp_Using_Statement.aspx
Take a look at your code in reflector after you compile it. You'll find that the compiler refactors the code to ensure that dispose is called on the stream.
I'm deserializing a huge JSON (1.4 GB) via a stream, because I don't want to load the whole content into memory in advance just for parsing. That's working fine, but it takes ~80 seconds, so I want to display a progress.
public JObject DeserializeViaStream(string filename)
{
object obj;
var serializer = new JsonSerializer();
using (var sr = new StreamReader(new FileStream(filename, FileMode.Open)))
{
using (var jsonTextReader = new JsonTextReader(sr))
{
obj = serializer.Deserialize(jsonTextReader);
}
}
return (JObject) obj;
}
I have not yet tried but only one idea: I could implement my own stream reader which keep track of the bytes being read and comparing that to the file length.
Is there a built-in option or easier way to do this?
I ended up using the idea I had. Luckily there's already a ProgressStream available by Mel Green (archive.org). The original URL is no longer available.
Please note:
this approach may not work for all situations and with all libraries forever. This is due to the fact that the Seek() operation provides random access, and someone could read the file multiple times.
I can't post the source code here, because it was released under an unclear license.
I have an HtmlTextWriter object (theWriter) being passed into a method. This method is in the middle tier. I'd like to read the contents of the theWriter in debug mode. The method is:
protected override void Render (HtmlTextWriter theWriter) {...}
which inherits from Panel:WebControl.
I've tried
theWriter.Flush();
theWriter.InnerWriter.ToString();
but that only outputs the object type: "System.Web.HttpWriter". I've seen some examples use methods on the Response object. But I don't have access to Response in this layer. Any ideas?
The InnerWriter is a TextWriter-derived class, which writes to a stream. You will have to open that stream and read data from it. Whether you can open and read from that stream is an open question, and depends very much on what type of stream it is.
So to use your example, theWriter.InnerWriter is an object derived from TextWriter. But you don't know what kind, and TextWriter itself doesn't expose the underlying stream.
Now, if InnerWriter is a StreamWriter, then you might be able to write:
var sWriter = theWriter.InnerWriter as StreamWriter;
var stream = sWriter.BaseStream;
var savePosition = stream.Position;
stream.Position = 0;
// now, you can read the stream
// when you're done reading the stream, be sure to reset its position
stream.Position = savePosition;
You have to be very careful, though. If you get the base stream and then open it with a StreamReader, closing the StreamReaderwill close the underlying stream. Then your HtmlTextWriter will throw an exception the next time you try to write to it.
It's also possible that you won't be able to read the stream. If the base stream is a NetworkStream, for example, you can't read it. Or it could be a FileStream that was open for write only. There's no good general way to do this, as it entirely depends not only on the specific TextWriter-derived class, but also on the stream that the TextWriter is writing to.
For example, the HtmlTextWriter could be writing to a StreamWriter, which is connected to a BufferedStream connected to a GZipStream, which finally writes to a MemoryStream.
So, in general, I'd recommend that you look for some other solution to your problem. Unless you know for sure what the underlying stream is, and that you can read it ... and that things won't change on you unexpectedly.
It look desceptively easy to use System.Drawing to create thumbnails in your ASP.NET application. But MSDN tells you:
Classes within the System.Drawing namespace are not supported for use within a Windows or ASP.NET service. Attempting to use these classes from within one of these application types may produce unexpected problems, such as diminished service performance and run-time exceptions.
I'm seeing intermittented 'out of memory' errors within this type of GDI+ code. I'm beginning to suspect this is the cause.
How ARE people doing server side image manipulation? Can anyone recommend any alternative that WON'T blow up my server?
The relevant code below. The exception intermittently happens in System.Drawing.Graphics.DrawImage. I've just inherited this project, so I'd need to check the logs to see how often this is being hit / how often we get an exception...
public byte[] Resize(int newWidth, int newHeight, Image orignalImage)
{
Bitmap bitmap = new Bitmap(newWidth, newHeight);
Graphics g = Graphics.FromImage(bitmap);
g.InterpolationMode = System.Drawing.Drawing2D.InterpolationMode.HighQualityBicubic;
Rectangle r = new Rectangle(0, 0, newWidth, newHeight);
g.DrawImage(orignalImage, r, r.X, r.Y, orignalImage.Width, orignalImage.Height, GraphicsUnit.Pixel);
MemoryStream stream = new MemoryStream();
bitmap.Save(stream, ImageFormat.Jpeg);
// clean up memory leaks
if (bitmap != null)
{
bitmap.Dispose();
bitmap = null;
}
if (g != null)
{
g.Dispose();
g = null;
}
return stream.ToArray();
}
UPDATE: I've searched thru the whole project for anywhere we are using GDI+ and put using() { } around everything that's IDisposable. I haven't seen one 'out of memory' exception since I did this.
Assuming you will be doing "stuff" per request, the issues might be
Processor intensive operation: manipulation of images, which could take time.
In case you are saving the file, it will lead to disk issues.
You can consider using HTTP handlers,
Disposing System.Drawing objects should be a priority(using(){} statement )
Asynchronous Pages can be explored here.
Why don't you set up a separate work server that is exposed through a web service.
I would recommend you put some exception handling code around these operations so that you're guaranteed to dispose of your GDI+ objects. Good practice to close your streams too ... although to my knowledge MemoryStream object is managed so should close itself when GC'd.