The pprof package documentation says
The package is typically only imported for the side effect of registering its HTTP handlers. The handled paths all begin with /debug/pprof/."
The documentation says if you already have an http server running you don't need to start another one but if you are not using DefaultServeMux, you will have to register handlers with the mux you are using.
Shouldn't I always use a separate port for pprof? Is it okay to use the same port that I am using for prometheus metrics?
net/http/pprof is a convenience package. It always registers handlers on DefaultServeMux, because DefaultServeMux is a global variable that it can actually do that with.
If you want to serve pprof results on some other ServeMux there's really nothing to it; all it takes is calling runtime/pprof.StartCPUProfile(w) with an http.ResponseWriter and then sleeping, or calling p.WriteTo(w, debug) on a runtime/pprof.Profile object. You can look at the source of net/http/pprof to see how it does it.
In a slightly better universe, net/http/pprof would have a RegisterHandlers(*http.ServeMux) function that could be used anywhere, you would be able to import it without anything being registered implicitly, and there would be another package (say net/http/pprof/sugar) that did nothing except call pprof.RegisterHandlers(http.DefaultServeMux) in its init. However, we don't live in that universe.
My problematic seem to be simple, but I haven't find yet a way to solve it...
I have a legacy system which is working and a new system which will replace it. This is only rest webservices call, so I'm using simple bridge endpoint on http service.
To ensure the iso-functional run, I want to put them behind a camel route dispatching message to both system but returning only the response of the legacy one and log the response of both system to be sure there are running in same way...
I create this route :
from("servlet:proxy?matchOnUriPrefix=true")
.streamCaching()
.setHeader("CamelHttpMethod", header("CamelHttpMethod"))
.to("log:com.mylog?showAll=true&multiline=true&showStreams=true")
.multicast()
.to(urlServer1 + "?bridgeEndpoint=true")
.to(urlServer2 + "?bridgeEndpoint=true")
.to("log:com.mylog?showAll=true&multiline=true&showStreams=true")
;
It works to call each services and to log messages, but response are in a mess...
If the first server doesn't respond, the second is not call, if the second respond an error, only that error is send back to client...
Any Idea ?
You can check for some more details in multicast docs http://camel.apache.org/multicast.html
Default behaviour of multicast (your case) is:
parallelProcessing is false so routes are called one by one
To correctly implement your case you need probably:
add error handling for each external service call so exception will not stop correct processing
configure or implement some aggregator strategy and put it to the strategyRef so you can combine results from all calls to the single multicast result
I am using hiredis library in my project. I'm using async API. I schedule a read command and wait for data. That works fine. However problem occurs when I try to close the connection - I call redisAsyncDisconnect, however the callback routine isn't called until I receive data.
Is it possible to cancel the read operation? How? Or is there a way how to force close the connection?
The answer is clear now - redisAsyncFree does what I need - stops all commands and calls redisAsyncDisconnect.
Information:
The incomming message is of type HL7. I'm using in the receive pipeline the "Flafile-Disassembler" and not the "BTAHL7 2.x Disassembler" pipeline component, because the HL7-Schema has is a bit modified and the BTAHl7 disassembler split the message (multipart messages) and we don't want; And we don't want to use orchestration.
Questions:
How can I create acknowledgements in a receive pipeline in BizTalk 2010, without using "BTAHL7 Disassembler" (without spliting --> multipart messages approach)?
Or, it's possible to prevent splitting the message in the BTAHL7 Disassembler pipeline component?
a positive ACK would be enough.
Thanks.
As #boatseller says, you cannot prevent the HL7 Disassembler from creating multi-part messages.
For your other question: you can create a custom pipeline component to send back the HL7 acknowledgement, and then just use your own flat file schema (with the out-of-the-box flat file disassembler pipeline component).
1. Use a two-way receive port.
This should work using a two-way port, even with the MLLP adapter, but you need to validate and test everything and understand that Microsoft may or may not support using the MLLP adapter without the HL7 2.X disassembler pipeline or using BizTalk in the manner suggested below.
2. Correlate the request and response.
You'll need a custom pipeline component in the receive location's pipeline to make BizTalk create a subscription for the response. Use the Pipeline Component Wizard to get a head start creating the component.
In the Execute method of your custom pipeline component class (which you get from implementing the IComponent interface), write something akin to the below code.
public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
const string sysPropertyNamespace = "http://schemas.microsoft.com/BizTalk/2003/system-properties";
var epmToken = inmsg.Context.Read("EpmRRCorrelationToken", sysPropertyNamespace);
var correlationToken = epmToken != null
? (string) epmToken
: Guid.NewGuid().ToString();
inmsg.Context.Promote("EpmRRCorrelationToken", sysPropertyNamespace, correlationToken);
inmsg.Context.Promote("RouteDirectToTP", sysPropertyNamespace, true);
return inmsg;
}
The use or the EpmRRCorrelationToken and RouteDirectToTP properties for request-response message processing without an orchestration is documented in this MSDN blog post.
3. Create and Send the Acknowledgement
You can use another custom pipeline component inside your send pipeline to manually create the acknowledgement based upon the input message, perhaps with some additional pipeline component configuration that is read at runtime.
Like the first custom pipeline component, you can implement the IComponent interface's Execute method. Code like this should work:
public Microsoft.BizTalk.Message.Interop.IBaseMessage Execute(Microsoft.BizTalk.Component.Interop.IPipelineContext pc, Microsoft.BizTalk.Message.Interop.IBaseMessage inmsg)
{
string msgOut;
var bodyPart = inmsg.BodyPart;
if (bodyPart == null)
return inmsg; // Maybe throw an exception instead?
var inboundStream = new ReadOnlySeekableStream(bodyPart.GetOriginalDataStream());
inboundStream.Position = 0;
using (var reader = new StreamReader(inboundStream))
{
var builder = new StringBuilder();
// Read the input stream's first line - this should be the MSH segment:
var firstLine = reader.ReadLine();
// TODO: read search/replacement values from pipeline configuration
// TODO: make a better ACK header than this
builder.AppendLine(firstLine.Replace("ADT^A01", "ACK"));
// Construct your acknowledgement segment, preferably without hardcoding it here:
builder.AppendLine("MSA|AA|ADT^A01");
msgOut = builder.ToString();
}
// Write out the output
var outputStream = new VirtualStream();
var writer = new StreamWriter(outputStream, Encoding.Default);
writer.Write(msgOut);
writer.Flush();
outputStream.Seek(0, SeekOrigin.Begin);
inmsg.BodyPart.Data = outputStream;
pc.ResourceTracker.AddResource(inboundStream);
pc.ResourceTracker.AddResource(outputStream);
return inmsg;
}
Other than handling the inline TODOs and adding in more null checking, that should be a fairly complete solution, though your mileage may vary, especially when it comes to returning a valid HL7 acknowledgement message.
Bonus. Why not use an orchestration?
This was asked in a comment to the question, and here are a few reasons for using custom pipeline components instead of an orchestration:
Orchestrations are slow. Microsoft, in their own Optimizing Orchestration Performance article, says that you should avoid using orchestrations when possible. Orchestrations increase latency and response times because they require extra roundtrips to the BizTalk MessageBox database and additional processes and threads be spun up on the BizTalk server.
Acknowledgments return faster. HL7 integrations typically use accept/receipt acknowledgements. Many systems will not send additional messages until the last-sent message is acknowledged. So if you have to wait for an orchestration to spin up, read the input, construct the ACK output, return the ACK to the port, and then send the ACK back, you will greatly slowdown processing. Yes, this is similar to #1, above, but it is important to understand.
Orchestrations make your solution more failure-prone. An orchestration is another point of failure. If you have to take your ACK-producing orchestration or its associated host instance(s) offline, then you have lost your ability to receive messages. A great benefit to BizTalk is its ability to isolate the different parts of an integration solution, enabling high availability and increased reliability. When your decoupled receive port's pipeline components return the acknowledgement after simply publishing the inbound message to the BizTalk message box, your receipt of messages can continue, even when your internal systems/components are down.
To answer you specific questions:
The HL7 ack is a specific function of the HL7 disassembler. You would have to create you own custom Disassembler component, running ffdasm internally, and generate your own ack to simulate the HL7 Disassembler's behavior.
No, I'm not aware of a way to prevent the HL7 Disassembler from creating Multi-Part Messages. You can easily recombine the segments in a Map executed in an Orchestration.
Couple of things here.
Number 1 - In my experience, orchestrations for HL7 should only be used in extreme cases. Our implementation only has a few where we need to spin out multiple messages from a single inbound message. Messaging only approach is best fit for HL7. They are costly and slow and to echo whats been said, add an extra point of failure.
Number 2 - Bits and pieces of what have been said are to be true, at least how i do it. You can bypass certain aspects of the DASM, and I do, because it is definitely not perfect. So i'd wrap it and include your custom functionality. The DASM returns a queue to the messaging engine that includes the disassembled message and an ack, you could dispose of the ACK and enqueue your own 'static' ack.
The DASM creates and queues the ACK in xml format so that the ASM on the send pipeline can assemble it back into raw HL7 format along with some other stuff. So you could either do the same in a custom pipeline component or find a way to call those private methods in the DASM that create the ACK and let it do it all for you.
I am developing a Live DirectShow filter.
I have an H264 stream source which i can get streams via SDK API.
In my filter I have a Queue which i Enqueue (push) incoming stream from a thread.
Then I consume (Dequeue,pop) these streams inside the filter FillBuffer...
So I make a thread safe Queue...But this causes some problem....
At FillBuffer if i check is there any incoming packets and if there is , process
logic is like this:
...
bool hasElement = SynchronizedQueue.pop(element);
if(!hasElement)
{
return S_OK
}
...
...this consume to much CPU...
Howewer using boost lib to implement lock with condition variable
...
SynchronizedQueue.waitAndPop(element) ;// which wait until we have some
Which have a lover CPU ...But sometime when there is no data in Queue, this block FillBuffer function and filter may not be stopped...
So any design idea alternatives for a live source filter which takes input streams from a remote mechine and pass it to decoder?
Or how can I make my design better....lower CPU and can be stopped?
The source filter owns a pushing thread, so you need to wait there using a synchronization object (event, mutex) to yield control until a new frame is available for pushing off the output pin.
Whenever you receive a frame from SDK and you put it onto queue, you will indicate this availability using synchronization object, e.g. you will set an event. The worker thread wlil see the event and start processing the frame.
The worker thread needs to be able to respond to two events, at least: a new frame, and filter/graph stop. So you will need WaitForMultipleObjects to wait for multiple evetns and wake up on the first occurred.