I have developed a BizTalk Application. It receives a xml file and, after applying the business logic, it sends the file to another location using FILE adapter. I need to track the start and end time for both Receive Port and Send Port. I have created BAM activities and view and have created a tracking profile using Tracking Profile Editor. I have used Interchange ID as continuation ID token.
The problem is that in the BAM tracking, I am getting two rows, one for the receive port and second for the send port. The continuation between the receive and send port is not working.
The Continuation is not working most likely because InterchangeID is not naturally Promoted.
The small issue you have is that there is no naturally Promoted Property that can be used out of the box for this.
The simplest solution would be to create a custom Pipeline Component that Promotes InterchangeID (same property, just Promoted). Then your Tracking Profile should start working.
FYI, it this point, you don't really need BAM as it's pretty easy to query tracking directly using the same Promoted Property (which is what BAM is essentially doing using a slightly different path).
The interchange id will be present in the message context. Can you please confirm you mapped receive and send ports to the continuations in the tracking profile editor. Refer the article https://www.biztalk-server-tutorial.com/2013/02/08/how-to-enable-bam-continuation-between-receive-send-ports-using-tracking-profile-editor/ which shows the steps to add continuation correctly.
Related
I'm trying to figure out what one could do to see where a flowfile is at in a nifi flow over http. For example, say I have a webpage where a user could upload files. I want to indicate to the user that that file is currently being ingested/processed, and possibly what stage its at. What does nifi offer that I could leverage to get this information? Like is there a way to see which processors a flowfile has gone through, or the processor/queue it's currently in?
Thanks
One option would be to use NiFi's provenance events and issue a provenance query for the flow file UUID to see all the current events which should give you a graph of all the processors it has passed through:
https://nifi.apache.org/docs/nifi-docs/html/user-guide.html#data_provenance
You can open Chrome Dev Tools while using provenance features in the UI and see what calls are being made to the REST API.
Another option is to build in some kind of status updates into your flow. You could stand up your own HTTP services that receives simple events like an id, timestamp, and processor name, then in your flow you could put InvokeHttp processors wherever you want to report status to your service. Then your UI would use the status events in it's own DB or wherever you store them.
By reading the Handing off work section in wiki, it seems that you can do one way publishing in rebus with one way mode. But I thought one way mode only allows bus.send?
If in a scenario, where I have two applications both can publish SomethingHappened message, but only one of the application should handle the reply from ProcessManager, e.g. DoSomethingElse, how should I configure rebus for both applications?
Regards
Yin
But I thought one way mode only allows bus.send?
Nothing will prevent you from doing a bus.Publish from a one-way client as long as it has a properly set up subscription storage. And if something else populates that subscription storage with some subscriptions (e.g. you, by inserting them manually in the db, or by having another publisher handle subscribe/unsubscribe) then something might actually happen when you publish ;)
how should I configure rebus for both applications?
Without having a totally clear image of your scenario, I'm thinking that
one publisher, A, has an input queue: publisherA
the other publisher, B, has no input queue (i.e. it's a one-way client)
the two publishers share their subscription storage (e.g. a table in SQL Server)
the two publishers publish messages from assembly StuffToPublish
other endpoints have an endpoint mapping that maps StuffToPublish to publisherA - therefore, a bus.Subscribe with an event type from StuffToPublish will make publisher A establish the subscription
published messages from A and B will go to the same subscribers because they share their subscription storage
If you want publisher A to handle replies even though the event was published by publisher B, you can make B supply the rebus-return-address header (which would otherwise automatically be set, had B had an input queue) like so:
bus.AttacheHeader(someEvent, Headers.ReturnAddress, "publisherA");
This way, replies will be sent to the publisherA input queue.
I have a receive port. It calls a stored procedure for transport, then the receive pipeline is just passthrureceive (I tried XMLReceive, but that caused many more issues). I have a send port that has the filter set so it picks up the type of the receive port. The send port has a map and send pipeline. The map maps the xml from the receive port to a flat file schema, and then sends it to it's location. THe send pipeline has a flat file assembler.
So the map never runs. I just get errors from the send pipeline saying it can't match the document. No it can't, because it wasn't mapped. I read that you need an XML Disassembler on the recieve pipeline of the receive location. I added that, and that just started destroying my messages. They just get turned into a blank message or just a " in the message. So the XML disassembler is clearly not working right. I'm not sure what to do at this point.
OK, I figured it out. I have to use the XML disassembler to make the map run. The reason it was erasing my messages is, well I don't know the reason, but when I set the schema to "elementFormDefault = Qualified", it worked. I'm not really sure what that did, as I really don't have a good understanding of the whole qualified message thing, but that was the problem for me.
There is a wizard in Visual Studio to help you create the schemas and bindings that you use to communicate with stored procedures. I guess you didn't use that? (Right click on the project, hit "Add", then "Add Generated Items", then "Consume Adapter Service")
I'm calling my stored procedure from an orchestration hooked to two way send/receive port. I'm using a custom WCF type port with XMLTransmit and XMLReceive for the pipelines. This seems to work fine. The caveat being it's always a bit fiddly getting WCF to work since there are so many options.
In order to map from one format to another on ports, you need to have MessageType promoted. In your case it can be accomplished by using XMLReceive on the receive pipeline.
I’m a complete newbie at BizTalk and I need to create a BizTalk 2006 application which broadcasts messages in a specific way. I’m not asking for a complete solution, but for advise and guidelines, which capabilities of BizTalk I should use.
There’s a message source, for simplicity, say, a directory where the user adds files to publish them. There are several subscribers, each having a directory to receive published files. The number of subscribers can vary in the course of exploitation of the program. There are also some rules which determine if a particular subscriber needs to receive a particular file, based on the filename. For example, each subscriber has a pattern or mask of filename which files they receives must match. Those rules (for example, patterns) can change in time as well.
I don’t know how to do this. Create a set of send ports at runtime, each for each destination? Is it possible? Use one port changing its binding? Would it work correctly with concurrent sendings? Are there other ways?
EDIT
I realized my question may be to obscure and general to prefer one answer over another to accept. So I just upvoted them.
You could look at using dynamic send ports to achieve this - if your subscribers are truly dynamic. This introduces a bit of complexity since you'll need to use an orchestration to configure the send port's properties based on your rules.
If you can, try and remove the complexity. If you know that you don't need to be truly dynamic when adding subscribers (i.e. a subscriber and it's rules can be configured one time only) and you have a manageable number of subscribers then I would suggest configuring each subscriber using it's own send port and use a filter to create subscriptions based on message context properties. The beauty of this approach is that you don't need to create and deploy an orchestration and this becomes a highly performant and scalable solution.
If the changes to the destination are going to be frequent, you are right in seeking a more dynamic solution. One nice solution is using dynamic send ports and the Business Rules Engine. You create rule set for the messages you are receving. This could be based on a destination property or customer ID in the message. Using these facts, the rules engine can return a bunch of information like file mask, server name, ip address of deleiver server, etc. You can thenuse this information to configure the dynamic send in the orchestration. The real nice thing here is that you can update the rule set in the rules engine without redeploying the whole solution. As a newb, these are some advanced concepts, but not as diificult as you may think.
For a simpler solution, you might want to look at setting the FILE Send adapters properties via it's Propery Schema (ie. File name, Directory, etc.). You could pull these values from a database with a helper class inside an expresison shape. On each message ogig out, use the property shcema to set where the message will be sent and named. This way, you just update the database as things change.
Good Luck!
I am not sure if I ask the right question, but this is the scenario I am trying to run:
Multiple files (XML and a few related files, "attachments") have to get into BizTalk as a single message. I have looked into existing adapters, and don't see that done with existing once. To be more accurate, files are taken from file system. Files are not found at the same time, but arrive one at a time, when order is not ensured. XML (content) file is the one that knows what attachments it has to have (what other files).
We are looking into BizTalk 2009 and I was wondering would be that responsibility of a custom Adaptor, or something else. And were I could look for samples.
Thanks.
It is probably possible to do what you want using a custom adapter, though I'd recommend against it. You can achieve what you require using orchestration.
What you are looking for is likey a convoy, or at the least some use of correlation.
In BizTalk a convoy is a messaging pattern (as opposed a BizTalk feature) that allows groups of messages to be processed by a single orchestration.
You essentially use correlation on a receive port to group messages together in either a parallel (what you probably want) or sequential fashion.
There is an article [here](http://msdn.microsoft.com/en-us/library/ms942189(BTS.10\).aspx) by Stephen W. Thomas about convoys (it is for BT 2004 but the concepts still hold) and there is a lot of additional information on the web and in books (Professional BizTalk server 2006 has a subsection on them)
Without more details on your scenario it is hard to know exactly how the convoy would be built but below are two approaches to look at (also, I've not had a chance to properly use BT2009 so there may be extended support for correlation scenarios that help you out).
Flexible Correlation
If you don't know anything about the files listed in the context XML you will probably need a pattern like the one described by Charles Young in this post.
Non-uniform sequential convoy
If you do have a little bit of info before hand one way might be as follows (basically a Non-uniform sequential convoy):
This makes the assumption that there is some way of linking all the files together so you can correlate them.
Create a single orchestration that subscribes to you inbound receive port (which contains the file receive location).
This orchestration will have a single activation receive shape that is set up for your content file.
Once the orchestration is started by a content file a second correlated receive shape starts picking up the messages that match that content file. (this second receive could possible be in a loop to allow for variable numbers of files)
You then pack them all together into a single outbound file of your design and send them out once the full number of files has been received.
Seems to me a better approach would be to implement the above requirements with a combination of a custom pipeline component and/or a custom adapter. I assume you do not really need to manipulate the incoming files - except for the content XML file - or that you couldn't since they are in binary format. This calls for a custom pipeline component.
What you can do is develop a custom BizTalk adapter to interact with the file system and to implement the listening and looping logic. Next you can develop a custom pipeline component to create a single BizTalk message perhaps with base64 data type in it for binary data. Additionally you could also promote messages right in this component to enable orchestration subscriptions.
Orchestrations are more suited for implementing business work-flow scenarios where the messages are already in XML format. This do not appear to be the case. In any case I think at the very least a custom pipeline component would be needed.
David's answer is the correct answer.
Even in cases where you don't know absolutely nothing about the contents of the expected attachments, surely you know their names and locations. Therefore you can use the Flexible Correlation linked to in david's answer like this:
The key to the solution is to correlate on the builtin BTS.ReceivedFileName property.
First, create a custom receive pipeline, with a custom pipeline component that promotes the BTS.ReceivedFileName context property of the received messages. This simple custom component is fairly easy to write but you can make it straightforward by using third-party frameworks such as, (shameless plug, here) my PipelineComponentBase class or the excellent BizTalk Server Pipeline Component Wizard.
Now for the easy part:
Attachments are received in a specific location, designated by its path on the filesystem.
Create a receive location that listens to an alternate location, used only to control when files are actually swallowed by BizTalk.
In your orchestration, create a correlation type with the BTS.ReceivedFileName property and a correlation set base on this correlation type.
When you want to receive binary attachments, send a dummy message with the BTS.ReceivedFileName context property set to the filename of the binary attachment but with the path matching the alternate location ; the one used by the receive location. Initialize the correlation on the send shape.
Use an expression shape to copy the binary file from its original location to the one used by the receive location.
Finally, use a receive shape bound to the receive port that contains the receive location whose custom receive pipeline will promote the BTS.ReceivedFileName property.
Notice that you actually need to send a message in order to initialize the correlation. It does not matter what message you send actually. What I'd do is send the message through a send pipeline that contains an empty pipeline component. That is a pipeline component that reads the message but return null (so that the message vanishes into thin air before it reaches the adapter). A more elaborate solution would be to use a null adapter. That is an adapter that reads the message but does not do anything about it.
These two solutions avoid having many files accumulate in a temporary location somewhere, just for the sake of initializing a correlation!