Is there anyway you could call a Biztalk orchestration without placing a message in the messagebox? The point here is to use an orchestration that is stored and configured in Biztalk but avoid the performance loss of using a database to trigger it.
Message box is an integral part of BizTalk server and no transaction can occur without message box. In most cases it works great and having message box provides lots of benefits for message delivery and processing. If you are having performance issues, I would recommend you measure your solution performance and identify bottlenecks. Some key points you can look:
Orchestration Persistent points
Use BizTalk host settings and change Polling intervals for messaging
and Orchestration to reduce from 500 ms to 50 ms, it does help.
If message box is a bottleneck (which is usually not the case until your volume is very high) add slave message boxes. BizTalk allows to scale out message box by adding slave message boxes. In this case one message box serves as master and rest others as slaves to process the request. Scale out message box
The answer to the question is NO, but, you are probably laboring under a false assumption.
There is no 'performance loss' due to the MessageBox. If you can prove the MessageBox causes you to miss an SLA, then you should be considering a completely different app platform such as a Windows Service. However, many of us have implemented very low latency apps with BizTalk without issue.
So, unless you SLA approaches the definition of 'real-time', I wouldn't worry about it.
Related
I have built an orchestration with a loop to retreive paged data from REST web service. From page size and offset I am able to call the service for "next page" of data. Then I debatch it, map it to internal format and process it further. When one page is processed, I request next page from the REST web service.
As it turns out, the host running the orchestration and send ports causes the memory to constantly grow during processing of all the data, and eventually hit the throttling mode.
Why is memory not released when I am done with one page loop? Is it the "consumed" messages that are stored in the orchestration that builds up the memory? Is it possible to clear orchestration from these "consumed" messages, to release the memory used?
(No message tracking active on the orchestration, or send ports.)
Apparently, there is no way to prohibit BizTalk Orchestrations from building up a list of messages in Orchestration, including used/processed/consumed messages. Putting things in Scope does not prohibit this behaviour.
Hence, for long-running Orchestrations there can be a lot of messages building up. Especially for singleton Orchestrations, where the general solution proposed to deal with this problem is to make sure Orchestration shuts down once in a while (when idle, e.g.).
My solution was to split the Orchestration into two, and have the initial Orchestration start the second Orchestration with the Start Orchestration, which in turn calls the second Orchestration recursively, and so on, until last page is received and the last Orchestration ends.
Yes, what you need to do is to have scopes, and that the messages are initialised in the scope (Green highlight below) rather than an the top level (yellow), and that means they will also be disposed of at the end of the scope. Note: That means those message can't be used outside of the scope.
However if you are just re-using the same messages in the loop, then I wouldn't expect it to increase memory usage. So there is possibly something else going on. I suspect that you must be adding each page to a message, and that is what is growing
Had an orchestration following the parallel convoy pattern to receive and wait a bunch of messages before kicking off the flow. Question is we are to move to an high variability environment with multiple active-active biztalk servers, would this be a problem? will the messagebox smart enough to figure out which host instance it should push the message to?
It's no problem at all. Don't even think about it.
An Orchestration can run an any available Host Computer but a specific instance of an Orchestration would only run on one at a time.
It doesn't matter which one.
Because a required pipeline components seems to have trouble hitting a database to details of messages, I am planning to use Host Throttling to limit the amount of files BizTalk is processing at the receive location. I want to be able to indicate that X number of messages should be processed within Y seconds (or any other feasible timespan). Does anyone know which throttling settings can be used to force this behavior?
I know how to set values, however I cannot find the best configuration.
(note: one of the solutions might also adjust the pipeline, but it contains third party components which cannot be adjusted.)
From How BizTalk Server Implements Host Throttling BizTalk looks at
Amount of memory in use (both systemwide and host process memory).
Number of in-process messages being delivered or processed (threshold for outbound throttling).
Number of threads in use. Database size, measured by the number of items in the queue tables for all hosts and the number of items in the spool and tracking tables.
Number of concurrent database connections.
Rate of message publishing (inbound)
and delivery or processing (outbound).
The only one that throttles inbound is the Rate of message publishing, but that is possible after the pipeline/port has processed the message so may not be of any use in this scenario, but you would have to test that.
You will probably want to set up that process under it's own host so if it hits throttling thresholds it does not throttle everything else as well.
If possible you should move the component to a send port pipeline as throttling send ports is much more controllable. One way is to set the send port to ordered delivery, although that can cause a backlog especially if you get a suspended message.
I think your most straightforward approach here would be to write a custom adapter. Unfortunately, the out of the box File Adapter does not directly support throttling/polling intervals, and I don't think the suggestions given already would not directly impact the custom pipeline processing if it's directly hitting the DB through ADO.NET (but it couldn't hurt to try). You can set the BatchSize property on the file adapter settings, but even then there's nothing stopping it from submitting that batchsize as fast as it possibly can over and over again.
A custom adapter could be created to wait for a period before submitting additional files for processing. You could base it on the SDK File Adapter sample.
Is using Sticky Sessions a supported scale out scenario? Has anyone deployed SignalR with Sticky Sessions and were there any unexpected issues?
We're investigating SignalR for a load-balanced, broadcast based project (similar to a stock ticker) where messages latency is an important factor. After reading though the Scale Out documentation it seems that the Backplane model could introduce significant latency in the messages, especially when message rates are high.
I've found some references implying that it would work with some side effects but not what the reliability and performance implications are.
Thanks!
If you use SignalR without a backplane, any client method invocation will only be able to reach clients connected directly to the server making the invocation.
This might be fine if you are only using Clients.Caller since the caller should always come back to the same server given sticky sessions. This could become an issue if you are using Clients.All, Clients.Others Clients.Client(connectionId), Clients.User(userName), Clients.Group(groupName), etc... In these cases, any client that is connected to a server different from the one executing the Clients... code will not receive the invocation regardless of whether the client is connected to the same Hub, has the right connectionId, etc...
I am currently in the process of porting an existing application (BizTalk 2004) to a newer version of BizTalk. The current solution takes multiple types of EDI documents, modifies it if its necessary and sends it to our legacy system to be loaded and processed.
This process is developed using a combination of Receive Ports, Pipeline component, Send Ports and Maps, Schema and Message Queue Components. This solution uses 10 send & receive ports to handle various aspects of the process such as Bursting EDI into individual messages, Transforming Messages, Error handling, EDI Validation and Batching of EDI Messages. All the modification of EDI is done using Message Queue Components.
This solution does NOT use orchestration at all. I am considering implementing the current solution as a BizTalk orchestration. I have read up a bit on orchestrations and worked through few sample applications. But I am still very confused over what benefit of using orchestration, if a solution can be developed without it. I am sure I am missing something here. What additional benefit orchestration gives that the current solution does not?
Edit:...I should clarify the question...I can do this app without using Orchestration using content based routing & maps. My question is, if I am missing something by not using Orchestration?
If you can perform your task at hand with message based routing, an orchestration is overkill.
Orchestrations will help you with calling rules, or handling transactions. The following points can help you decide whether to use orchestration or not:
Is the handling Transactional
Is ordering of messages important
Are you going to process the message using business rules
Do you have to call external assemblies
A quote from "Microsoft BizTalk Server Pattern"
Orchestrations come at a considerable cost. Many of these costs manifest themselves as roundtrips to the messagebox, which means crossing a process boundary and writing to and reading from a database -the messagebox
An orchestration can potentially take twice as long for the same process. For example: A simple process of receiving a message and sending it will make 2 message hops with the messaging approach vs 4 with the orchestration.
Here are the steps for a messaging only solution
Receive the message via the adapter save it to the message box
Retrieve the message for the send port
vs:
Steps for Orchestration approach
Receive the message via the adapter and save it to the message box
Retrieve the message to start the orchestration
Do your mapping if you need to
Retrieve the item again for the send port.
Choose wisely
It sounds like you could re-implement the solution in a messaging only solution and don't need an Orchestration. If you can that's great, we prefer messaging only as they are simpler to maintain and generally more efficient. Orchestration are useful if you need to have a workflow of multiple actions, or special error handling that you can't easily do with a messaging only solution.