I have a legacy code where a transform is applied to an incoming document -- plain XML schema. It is being mapped to request for executing a SQL PROC.
So although it shows error source as Source: Microsoft.BizTalk.Pipeline, on the admin console query hub it is seen as suspended process at orchestration and not at the send port which does the SQL operation of executing the SQL PROC? Can someone explain this? There is no clue in fixing the issue. On reprocess it does work. But this happens very often
Now this transform fails. But interestingly it shows this in event log:
Exception type: COMException
Source: Microsoft.BizTalk.Pipeline
Target Site: Void Read(Byte*, Int32, Int32*)
The following is a stack trace that identifies the location where the
exception occured
at Microsoft.BizTalk.Message.Interop.IStream.Read(Byte* pv, Int32 cb, Int32* pcbRead)
at Microsoft.BizTalk.Message.Interop.StreamViewOfIStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at Microsoft.BizTalk.Streaming.MarkableForwardOnlyEventingReadStream.ReadInternal(Byte[] buffer, Int32 offset, Int32 count)
at Microsoft.BizTalk.Streaming.EventingReadStream.Read(Byte[] buffer, Int32 offset, Int32 count)
at Microsoft.BizTalk.Streaming.StreamEncodingDetector._read(Stream stm, Byte[] buff)
at Microsoft.BizTalk.Streaming.StreamEncodingDetector._detectStreamEncoding(Stream stm)
at Microsoft.BizTalk.Streaming.StreamEncodingDetector.CreateStream(Stream stm, Encoding& encoding)
at Microsoft.BizTalk.Streaming.StreamEncodingDetector.CreateStreamReader(Stream stm)
at Microsoft.XLANGs.Core.UnderlyingPart.CreateXmlReader()
at Microsoft.XLANGs.RuntimeTypes.XmlRewritingContentFactoryState._createStreamOrReader(String& encoding, Boolean wantEncoding, Boolean wantReader)
at Microsoft.XLANGs.Core.Value.GetStream(String& encoding, Boolean wantEncoding)
at Microsoft.XLANGs.Core.Value.RetrieveAs(Type t)
at Microsoft.XLANGs.Core.Part.ProtectedRetrieveAs(Type t)
at Microsoft.XLANGs.Core.Part.RetrieveAs(Type t)
at Microsoft.XLANGs.Core.Part.get_Stream()
at Microsoft.XLANGs.Core.Service.ApplyTransform(Type mapRef, Object[] outParams, Object[] inParams)
at MyProcess.segment1(StopConditions stopOn)
at Microsoft.XLANGs.Core.SegmentScheduler.RunASegment(Segment s, StopConditions stopCond, Exception& exp)
EDIT 1: ADDITIONAL INFO
the map does not use any custom functoid. It just plainly links one field on the source to another field on the destination (destination is a sqlproc schema. i have observed the problem even with smaller files like around 400 kb etc
Related
I want to do some experiments in OpenCL and I want to know possibility to change states during kernel execution from host code using buffer.
I attempted to alter the state of a while loop in the kernel code by modifying the buffer value from within the host code, however the execution is hung.
void my_kernel(
__global bool *in,
__global int *out)
{
int i = get_global_id(0);
while(1) {
if(1 == *in) {
printf("while loop is finished");
break;
}
}
printf("out[0] = %d\n", out[0]);
}
I call second time the function clEnqueueWriteBuffer() to change state of input value.
input[0] = 1;
err = clEnqueueWriteBuffer(commands, input_buffer,
CL_TRUE, 0, sizeof(int), (void*)input,
0, NULL,NULL);
At least for OpenCL 1.x, this is not permitted, and any behaviour you may observe in one implementation cannot be relied upon.
See the NOTE in the OpenCL 1.2 specification, section 5.2.2, Reading, Writing and Copying Buffer Objects:
Calling clEnqueueWriteBuffer to update the latest bits in a region of the buffer object with the ptr argument value set to host_ptr + offset, where host_ptr is a pointer to the memory region specified when the buffer object being written is created with CL_MEM_USE_HOST_PTR, must meet the following requirements in order to avoid undefined behavior:
The host memory region given by (host_ptr + offset, cb) contains the latest bits when the enqueued write command begins execution.
The buffer object or memory objects created from this buffer object are not mapped.
The buffer object or memory objects created from this buffer object are not used by any command-queue until the write command has finished execution.
The final condition is not met by your code, therefore its behaviour is undefined.
I am not certain if the situation is different with OpenCL 2.x's Shared Virtual Memory (SVM) feature, as I have no practical experience using it, perhaps someone else can contribute an answer for that.
environment messages:
(env) λ java -version
openjdk version "1.8.0_242"
OpenJDK Runtime Environment (build 1.8.0_242-xxxxxx_JDK_xxxxx)
OpenJDK 64-Bit Server VM (build 25.242-b08, mixed mode)
/*
* Copyright (c) Google Technologies Co., Ltd. 2021-2021. All rights reserved.
*/
package corejava.v1ch05.practice;
import java.lang.reflect.Field;
import java.util.Random;
public class ChangeString {
private static void printAddress(String message) throws IllegalAccessException, NoSuchFieldException {
Field f = message.getClass().getDeclaredField("value");
f.setAccessible(true);
char[] v = (char[])f.get(message);
System.out.println("message hashcode: " + message.hashCode());
System.out.println("message real identity: " + System.identityHashCode(message));
System.out.println("v real identity: " + System.identityHashCode(v));
System.out.println();
}
private static void change(String message) throws NoSuchFieldException, IllegalAccessException {
System.out.println(System.identityHashCode(message));
Field f = message.getClass().getDeclaredField("value");
System.out.print("Accessible: " + f.isAccessible());
f.setAccessible(true);
char[] v = (char[])f.get(message);
System.out.println(System.identityHashCode(v));
Random random = new Random();
char randomizedCharacter = (char) (random.nextInt(26) + 'a');
v[0] = randomizedCharacter;
System.out.println();
}
public static void main(String[] args) throws NoSuchFieldException, IllegalAccessException {
String s1 = " abcd";
String s2 = " abcd";
String s3 = new String(" abcd");
printAddress(s1);
printAddress(s2);
printAddress(s3);
change(s1);
printAddress(s2);
printAddress(s3);
System.out.print(s1 + " " + s2 + " " + s3);
}
}
The results are as follows:
message hashcode: 32539746
message real identity: 460141958
v real identity: 1163157884
message hashcode: 32539746
message real identity: 460141958
v real identity: 1163157884
message hashcode: 32539746
message real identity: 1956725890
v real identity: 1163157884
460141958
Accessible: false1163157884
message hashcode: 32539746
message real identity: 460141958
v real identity: 1163157884
message hashcode: 32539746
message real identity: 1956725890
v real identity: 1163157884
qabcd qabcd qabcd
As I know, the string constant is stored in the string constant pool.
store location(maybe it's wrong!)：
jdk1.6 Method Area
jdk1.7 heap memory
jdk1.8 local memory
I changed the s1, the s2, s3 reference also changed. It really mixed me up!
do the s1, s2, s3's final value " abcd" has the same memory address?
As I know, the string constant is stored in the string constant pool. store location(maybe it's wrong!)： jdk1.6 Method Area jdk1.7 heap memory jdk1.8 local memory
Yes it’s wrong. As wrong as a statement can be. All objects are stored in the heap memory. Because that’s how heap memory is defined:
The Java Virtual Machine has a heap that is shared among all Java Virtual Machine threads. The heap is the run-time data area from which memory for all class instances and arrays is allocated.
The constant pool is nothing you have to think about. All that matters, is that all string literals of the same contents, i.e. " abcd", refer to the same object.
Moreover, a string literal always refers to the same instance of class String. This is because string literals - or, more generally, strings that are the values of constant expressions (§15.29) - are "interned" so as to share unique instances, as if by execution of the method String.intern (§12.5).
Since they refer to the same object, hacking into the internals to manipulate what is supposed to be immutable, will make the change observable through all references.
The array wrapped by String objects is an implementation detail, so whether creating a new string via new String(" abcd") will create a new array or just let the two equal strings share the same array, is an implementation detail too. If they share the same array, manipulating that array affects both strings.
The “identity hash code” is neither, a “true identity” nor an address. It’s, like the ordinary hash code, a number helping certain data structures, like HashSet, to look up objects efficiently. Since there is no limit on the number of objects an implementation can create, but the hash code is just a 32 bit number, two objects still can have the same identity hash code. Further, objects can be moved around in the memory, but the identity hash code of an object will never change. So, obviously, it can’t be an address.
In your particular output, they two distinct string instances happened to have different identity hash codes and all strings happened to truly refer to the same array, which the behavior after the modification proved better than the hash codes.
var(
file *xlsx.File
sheet *xlsx.Sheet
row *xlsx.Row
cell *xlsx.Cell
)
func addValue(val string) {
cell = row.AddCell()
cell.Value = val
}
and imported from http://github.com/tealeg/xlsx
when ever control comes to this line
cell = row.AddCell()
It is panicking.
error:
panic: runtime error: invalid memory address or nil pointer dereference
Can someone suggest whats going wrong here?
Nil pointer dereference
If to attempt to read or write to address 0x0, the hardware will throw an exception that will be caught by the Go runtime and panic will be throwed. If the panic is not recovered, a stack trace is produced.
Definitely you are trying to operate with nil value pointer.
func addValue(val string) {
var row *xlsx.Row // nil value pointer
var cell *xlsx.Cell
cell = row.AddCell() // Attempt to add Cell to address 0x0.
cell.Value = val
}
Allocate memory first
func new(Type) *Type:
It's a built-in function that allocates memory, but unlike its namesakes in some other languages it does not initialize the memory, it only zeros it. That is, new(T) allocates zeroed storage for a new item of type T and returns its address, a value of type *T. In Go terminology, it returns a pointer to a newly allocated zero value of type T.
Use new function instead of nil pointers:
func addValue(val string) {
row := new(xlsx.Row)
cell := new(xlsx.Cell)
cell = row.AddCell()
cell.Value = val
}
See a blog post about nil pointers
Summary: ASP.Net website with a couple hundred users. Data is exported to Excel files which can be relatively large (~5 MB).
In the pilot phase (just a few users), we are already seeing occasional errors on the server in the exporting method.
Here's the stack trace:
System.Web.HttpUnhandledException: Exception of type 'System.Web.HttpUnhandledException' was thrown. --->
System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at
System.IO.MemoryStream.set_Capacity(Int32 value) at
System.IO.MemoryStream.EnsureCapacity(Int32 value) at
System.IO.MemoryStream.Write(Byte[] buffer, Int32 offset, Int32 count) at
MS.Internal.IO.Packaging.TrackingMemoryStream.Write(Byte[]buffer, Int32 offset, Int32 count) at
MS.Internal.IO.Packaging.SparseMemoryStream.WriteAndCollapseBlocks(Byte[] buffer, Int32 offset, Int32 count) at
MS.Internal.IO.Packaging.SparseMemoryStream.Write(Byte[]buffer, Int32 offset, Int32 count) at
MS.Internal.IO.Packaging.CompressEmulationStream.Write(Byte[]buffer, Int32 offset, Int32 count) at
MS.Internal.IO.Packaging.CompressStream.Write(Byte[]buffer, Int32 offset, Int32 count) at
MS.Internal.IO.Zip.ProgressiveCrcCalculatingStream.Write(Byte[]buffer, Int32 offset, Int32 count) at
MS.Internal.IO.Zip.ZipIOModeEnforcingStream.Write(Byte[]buffer, Int32 offset, Int32 count) at
System.IO.StreamWriter.Flush(BooleanflushStream, Boolean flushEncoder) at
System.IO.StreamWriter.Write(String value) at
System.Xml.XmlTextEncoder.Write(String text) at
System.Xml.XmlTextWriter.WriteString(String text) at
System.Xml.XmlText.WriteTo(XmlWriter w) at
System.Xml.XmlAttribute.WriteContentTo(XmlWriter w) at
System.Xml.XmlAttribute.WriteTo(XmlWriter w) at
System.Xml.XmlElement.WriteTo(XmlWriter w) at
System.Xml.XmlElement.WriteContentTo(XmlWriter w) at
System.Xml.XmlElement.WriteTo(XmlWriter w) at
System.Xml.XmlElement.WriteContentTo(XmlWriter w) at
System.Xml.XmlElement.WriteTo(XmlWriter w) at
System.Xml.XmlElement.WriteContentTo(XmlWriter w) at
System.Xml.XmlElement.WriteTo(XmlWriter w) at
System.Xml.XmlDocument.WriteContentTo(XmlWriter xw) at
System.Xml.XmlDocument.WriteTo(XmlWriter w) at
System.Xml.XmlDocument.Save(Stream outStream) at
OfficeOpenXml.ExcelWorksheet.Save() in
C:\temp\XXXXXXXXXX\ExcelPackage\ExcelWorksheet.cs:line 605 at
OfficeOpenXml.ExcelWorkbook.Save() in
C:\temp\XXXXXXXXXX\ExcelPackage\ExcelWorkbook.cs:line 439 at
OfficeOpenXml.ExcelPackage.Save() in
C:\temp\XXXXXXXXXX\ExcelPackage\ExcelPackage.cs:line 348 at
Framework.Exporting.Business.ExcelExport.BuildReport(HttpContext context) at
WebUserControl.BtnXLS_Click(Object sender, EventArgs e) in
C:\TEMP\XXXXXXXXXX\XXXXXXXXXX\XXXXXXX\UserControls\ExportToExcel.ascx.cs:line 108 at
System.Web.UI.WebControls.Button.OnClick(EventArgs e) at
System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument) at
System.Web.UI.WebControls.Button.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument) at
System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument) at
System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData) at
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
---End of inner exception stack trace ---
at
System.Web.UI.Page.HandleError(Exception e) at
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) at
System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) at
System.Web.UI.Page.ProcessRequest() at
System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext context) at
System.Web.UI.Page.ProcessRequest(HttpContext context) at
ASP.XXXXXXXXXXX_aspx.ProcessRequest(HttpContext context) in
c:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\TemporaryASP.NET
Files\XXXX\cdf32a52\d1a5eabd\App_Web_enxdwlks.1.cs:line 0 at
System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() at
System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Even aside from this particular problem, in general exporting to Excel requires the instantiation of huge Excel objects on the server for each request, which I've always assumed to mean disqualifies Excel for "serious" work on a highly-loaded server. Is there any general way to export to Excel in a "light-weight" manner? Would simply streaming the data into a CSV file work for this?
If formatting is not an issue, I would say yes to CSV.
Another thing I noticed, you are using ExcelPackage. I ran into some issue using it with large files. I think It was the way it handled xml and exporting large files was really slow. I would recommend using some other library.
I would offload this to another machine. Create a Windows Service that takes the command, generates the file, zips it up, saves it to disk and then returns the path to the file so that the web app can serve it. This is one of those really memory intensive operations that you don't want bogging down your web server.
I know this post is really old but if you are using the ExcelPackage I kept getting the Out of memory exception on my "LoadFromDataTable" call for a datatable which had about 300k rows. I found for me the point was about 100k where the error would ocassionally occur.
I used a split datatable method at about 50k and iterated through the datatable collection and then called the "LoadFromDataTable" method. Seems to be working only problem is the method deletes the previous. If I could figure out how to append I would be alright. Excel exports at its finest I tell you.
I have an application where every now and then I'm getting a strange error.
This is the piece of code:
Dim XMLWriter As New System.Xml.XmlTextWriter(Me.Context.Response.OutputStream, Encoding.UTF8)
XMLWriter.WriteStartDocument()
XMLWriter.WriteStartElement("Status")
Message.SerializeToXML(XMLWriter)
XMLWriter.WriteEndElement()
XMLWriter.WriteEndDocument()
XMLWriter.Flush()
XMLWriter.Close()
The error i'm getting is:
Message: Object reference not set to an instance of an object.
on line XMLWriter.Flush();
To make things more fun, this is absolutely non-reproducible. It just happens every now and then....
Since it's happening when flushing the XML i'm guessing the Object that is now null has to be the Response.OutputStream.
This is the relevant part of the stack trace:
Description:
An unhandled exception occurred and the process was terminated.
Exception: System.NullReferenceException
Message: Object reference not set to an instance of an object.
StackTrace: at System.Web.HttpWriter.BufferData(Byte[] data, Int32 offset, Int32 size, Boolean needToCopyData)
at System.Web.HttpWriter.WriteFromStream(Byte[] data, Int32 offset, Int32 size)
at System.IO.StreamWriter.Flush(Boolean flushStream, Boolean flushEncoder)
at System.Xml.XmlTextWriter.Flush()
at RequestData.CompleteRequest(MessageBase Message) in RequestData.vb:line 142
My question is, in what cases could this be happening?
This server is a long-polling server, so the client asks for something, and I may not answer for 30 seconds...
Is it possible that this Stream will become Null if the client disconnects (ie. closes the browser window)?
Any other ideas? (any pointers appreciated)
Reflector gives this:
private void BufferData(byte[] data, int offset, int size, bool needToCopyData)
{
int num;
if (this._lastBuffer != null)
{
num = this._lastBuffer.Append(data, offset, size);
size -= num;
offset += num;
}
else if ((!needToCopyData && (offset == 0)) && !this._responseBufferingOn)
{
this._buffers.Add(new HttpResponseBufferElement(data, size));
return;
}
while (size > 0)
{
this._lastBuffer = this.CreateNewMemoryBufferElement();
this._buffers.Add(this._lastBuffer);
num = this._lastBuffer.Append(data, offset, size);
offset += num;
size -= num;
}
}
The only object that is not null checked, initialized or referenced through another method(which would show in the stack trace) is this._buffers. The only place it is set to null in that class is in RecycleBufferElements() which if you dig deeper can occur when the client disconnects.
The call to Flush is what will cause anything cached in memory to be written out to the stream and ultimately the client so yes, it may be the problem.
You mentioned that the request is expected to take a long time to execute so it may be possible that ASP.Net or IIS are timing you out too early. I would suggest having a look at the executionTimeout property in the web.config and similar settings.
No, if it happens when you call Flush, that's way later than the only time Context.Response.OutputStream is actually referenced. The value is fetched in the call to the XmlTextWriter constructor, and then not looked at again.
Do you have any more information from the stack trace?