BizTalk Flat File, multi-line record debatching - biztalk

I've tried to find the answer in other questions, and none of the "standard" answers are working, so I'm hoping someone can either point me to where this has already been answered, or can tell me how to do this.
I have a large file with multiple documents within it. For a sample, assume something like this
DOCUMENT_IDENTIFIER 123400000000000000000123457 OTHER STUFF HERE
LINE WITH STUFF HERE
LINE WITH STUFF HERE
DOCUMENT_IDENTIFIER 123500000000000000000127456 OTHER STUFF HERE
LINE WITH STUFF HERE
LINE WITH STUFF HERE
Now, I'll need to preserve everything in the DOCUMENT_IDENTIFIER Line starting with the first 0 through the 123 (or 127 in the second Document) That header line, plus the all the LINE WITH STUFF HERE lines below it should make up one Document, and a new document should start at the second DOCUMENT_IDENTIFIER line.
When I attempt to use the standard Debatching techniques, the pipeline fails: either it just fails completely (when, for instance, I try to define a header and body schemas for the pipeline) or it never starts the second document (if I try just a body schema).
I'm certain this is something fairly simple, but I'm completely missing how to get it done. Any suggestions/direction would be welcome.
If it matters, I'm stuck on BT2006 R2, at current.

What does your Body Schema look like? I would start getting that right and make sure that you have something that will create xml with separate records of all the "DOCUMENT_IDENTIFIER 1234" records.
I would use the "DOCUMENT_IDENTIFIER "1234 bit as the Tag Identifier, and then I would set the Tag Offset to 4, to avoid the first 4 characters.
You should have a
RecordForDocumentIdentifiers (Root of your Schema) Group Maxoccurs=*
RecordForDocumentIdentifier (Set the Tag Identifier here)
Fields for the columns you want to parse
RecordForOtherLines Group Maxoccurs=*
RecordForOtherLine Maxoccurs=* or whatever your rules are.
Fields for the columns of other lines
When that seems to parse your example okay, and generate the XML you want, I would start creating my header and body schemas from that. I know it is 2 steps, but it takes some of the guesswork out of it.
I guess the Header schema would be picked from the RecordDocumentIdentifier and the body would be RecordForOtherLines (The outer record for that).
I hope that helps. If not, post your actual file and schema and let us take a look at it.

Related

Make basic Math in Draw.io (diagrams.net)

I want to make some basic math stuff like Sum in Diagrams.net (old Draw.io).Is it possible ?
Exemple : I create a new parameter on a shape, like "Elec : T16" and make several copy on this shape. Is it possible to have a Text which can give me the total of the shape with this parameter ?
Best Regards.
I search a lot in the Diagrams.net blog but anything relevent.
This is not supported.
Regards,
I also wanted to do something similar and while it doesn't seem possible to do it completely in the software (as of v20.3.0), I did find a bit of a workaround: If you add properties to the shape data, then do File > Export As > XML, the properties will be there in the XML data. You can then count them one of two ways:
Open the XML file with a text editor like Notepad++, do a find on the value you want to count. If you choose "Find All" it will tell you how many times it appears.
Use a programming language like Python to read through the file and count the instances of that value.
Example:
I created a red circle in a new diagram, edited the text to say "RedCircle" and used Edit Data to add a property called TestValue, to which I assigned a value of 1. When I exported to XML it contained this element:
<object label="RedCircle" TestValue="1" id="6byQ5fOap-RXn7mFit_J-1">
Notes
When you export, make sure you turn off the Compressed option, this will create an unusable file.
Don't use Save As > XML, this will also use compression.
Diagrams.net natively saves in a compressed XML format, with only slight differences between that and the other compressed XML options, but it seems happy to also read in the exported uncompressed XML. I didn't test but if you go the programming route and want to take it a step further, it seems you could have the program update the value of a given "counter" element with the count, then open the XML file in diagrams.net to see the updated value and save it as a native .drawio file or publish in whatever format you like.
Edit: I discovered that under File > Properties you can turn off the compression on the actual .drawio file. If you do that you can just work from this file instead of exporting, but you might want to check the size of your file with and without it.
I'm sure a plugin could be created to do all of that within the app itself, but the other methods are enough for me at this point.
Hope this helps you!

Missing Root node in xml file in biztalk

I have researched a lot on internet, but still not able to get exact solution for my problem. The problem mentioned in below link is much similar to mine, but this will not work.
If record exists then map, if not map another - Biztalk mapping
My Problem:
I have xml source file which has invoice information, something like this.
-<invoice>
<invoiceNumber>1124</invoiceNumber>
+<invoiceHeader>
+<invoiceDetails>
</invoice>
<invoiceNumber>1598</invoiceNumber>
+<invoiceHeader>
+<invoiceDetails>
</invoice>
It is possible that, sometimes the subnode ie. invoiceDetails information can be missing for few invoice Numbers,but header information will be present.
Output required:
The output file (.csv) should have all invoice numbers irrespective of detail tag present or not?
Add a Looping Functoid to the Map with the source link from and the target from the Looping Functoid linked to the row element in the Flat File Schema.
It will work, trust us. :)
It depends on how is the output schema, but solution must be easy only link tag to the output.
Behind the scenes, the XSLT produced is a loop for each tag and copy that value to the output, irrespective of detail tag present or not.

What's the correct format for TCDL linkAttributes?

I can see the technology-independent Tridion Content Delivery Language (TCDL) link has the following parameters, which are pretty well described on SDL Live Content.
type
origin
destination
templateURI
linkAttributes
textOnFail
addAnchor
VariantId
How do we add multiple attribute-value pairs for the linkAttributes? Specifically, what do we use to escape the double quotes as well as separate pairs (e.g. if we need class="someclass" and onclick="someevent").
The separate pairs are just space delimited, like a normal series of attributes. Try XML encoding the value of linkAttributes however. So, " become &quote;, etc...
If you are using some Javascript, you might take care of the Javascript quotes too, as in \".
Edit: after I figured out your real question, the answer is a lot simpler:
You should wrap the values inside your linkAttributes in single quotes. Spaces inside linkAttributes are typically handled fine; but if not, escape then with %20.
If you need something more or want something that isn't handled by the standard tcdl:ComponentLink, remember that you can always create your own TCDL tag and and use a TagHandler or TagRenderer (look them up in the docs for examples or search for Jaime's article on TagRenderer) to do precisely what you want.
My original answer was to a question you didn't ask: what is the format for TCDL tags (in general). But the explanation might still be useful to some, so remains below.
I'd suggest having a look at what format the default building blocks (e.g. the Link Resolver TBB in the Default Finish Actions) output and use that as a guide line.
This is what I could quickly get from the transport package of a published page:
<tcdl:Link type="Page" origin="tcm:5-199-64" destination="tcm:5-206-64"
templateURI="tcm:0-0-0" linkAttributes="" textOnFail="true"
addAnchor="" variantId="">Home</tcdl:Link>
<tcdl:ComponentPresentation type="Embedded" componentURI="tcm:5-69"
templateURI="tcm:5-133-32">
<span>
...
One of the things that I know from experience: your entire TCDL tag will have to be on a single line (I wrapped the lines above for readability only). Or at least that is the case if it is used to invoke a REL TagRenderer. Clearly the tcdl:ComponentPresentation tag above will span multiple lines, so that "single line rule" doesn't apply everywhere.
And that is probably the best advice: given the fact that TCDL tags are processed at multiple points in Tridion Publishing, Deployment and Delivery pipeline, I'd stick to the format that the default TBBs output. And from my sample that seems to be: put everything on a single line and wrap the values in (double) quotes.

Removing header from a flat file in BizTalk

What is the simplest way to remove the header row from a flat file in BizTalk? I have implemented this by creating a separate schema for the header row (and one for the body) and then set the HeaderSpecName property in the 'Configure Pipeline' dialog to the header schema I've just created and then the PreserveHeader property to false. My problem, however, is that with this solution I need to create a schema that does nothing other than it needs to exist so I can remove the header row.
I believe what you stated is the recommended way to do what you are asking. I just had to do this on a project i am working on and chose to have the extra schema.
Another option would be to write your own custom pipeline component for the Disassemble stage that executes before the flat file dissassembler. The Decode stage would work too, but it seems that the disassemble stage is made for this type of work.
This second option would make the removing of the header row more generic and could be used across many different schemas so you wouldn't have to create a separate header schema for each flat file schema.
i recommend doing this through the flat file schema wizard; define your first record as a singular record and then your other records as repeating records. then in whatever map or transformations you're working with you can just ignore the header record. you could also ignore the header record when going through the FF wizard by ignoring it altogether, but i prefer to have everything defined in my schemas...
check out the help and microsoft tutorials, also there's a decent article on the code project that incorporates identifying header records. note that if you know the absolute position of the header record (like: "always the first line") you don't need to mess with record tagging.
http://www.codeproject.com/Articles/13706/Creating-Flat-File-schemas-using-the-BizTalk-Serve
You could always just parse the header row and not map it.
The schema exists for many reasons. One such reason is to capture the structure of the message generated by a party which effectively means the agreement. So it is good to capture the schema as it is even though you may not use the fields.
I don't think PreserveHeader acts as you may expect. This property simply determines whether or not values from the header are promoted into the context of the XML message that comes out of the disassembler.

Deleting / Replacing A Node in E4X (AS3 - Flex)

I'm building a listing/grid control in a Flex application and using it in a .NET web application. To make a really long story short I am getting XML from a webservice of serialized objects. I have a page limit of how many things can be on a page. I've taken a data grid and made it page, sort across pages, and handle some basic filtering.
In regards to paging I'm using a Dictionary keyed on the page and storing the XML for that page. This way whenever a user comes back to a page that I've saved into this dictionary I can grab the XML from local memory instead of hitting the webservice. Basically, I'm caching the data retrieved from each call to the webservice for a page of data.
There are several things that can expire my cache. Filtering and sorting are the main reason. However, a user may edit a row of data in the grid by opening an editor. The data they edit could cause the data displayed in the row to be stale. I could easily go to the webservice and get the whole page of data, but since the page size is set at runtime I could be looking at a large amount of records to retrieve.
So let me now get to the heart of the issue that I am experiencing. In order to prevent getting the whole page of data back I make a call to the webservice asking for the completely updated record (the editor handles saving its data).
Since I'm using custom objects I need to serialize them on the server to XML (this is handled already for other portions of our software). All data is handled through XML in e4x. The cache in the Dictionary is stored as an XMLList.
Now let me show you my code...
var idOfReplacee:String = this._WebService.GetSingleModelXml.lastResult.*[0].*[0].#Id;
var xmlToReplace:XMLList = this._DataPages[this._Options.PageIndex].Data.(#Id == idOfReplacee);
if(xmlToReplace.length() > 0)
{
delete (this._DataPages[this._Options.PageIndex].Data.(#Id == idOfReplacee)[0]);
this._DataPages[this._Options.PageIndex].Data += this._WebService.GetSingleModelXml.lastResult.*[0].*[0];
}
Basically, I get the id of the node I want to replace. Then I find it in the cache's Data property (XMLList). I make sure it exists since the filter on the second line returns the XMLList.
The problem I have is with the delete line. I cannot make that line delete that node from the list. The line following the delete line works. I've added the node to the list.
How do I replace or delete that node (meaning the node that I find from the filter statement out of the .Data property of the cache)???
Hopefully the underscores for all of my variables do not stay escaped when this is posted! otherwise this.&#95 == this._
Thanks for the answers guys.
#Theo:
I tried the replace several different ways. For some reason it would never error, but never update the list.
#Matt:
I figured out a solution. The issue wasn't coming from what you suggested, but from how the delete works with Lists (at least how I have it in this instance).
The Data property of the _DataPages dictionary object is list of the definition nodes (was arrived at by a previous filtering of another XML document).
<Models>
<Definition Id='1' />
<Definition Id='2' />
</Models>
I ended up doing this little deal:
//gets the index of the node to replace from the same filter
var childIndex:int = (this._DataPages[this._Options.PageIndex].Data.(#Id == idOfReplacee)[0]).childIndex();
//deletes the node from the list
delete this._DataPages[this._Options.PageIndex].Data[childIndex];
//appends the new node from the webservice to the list
this._DataPages[this._Options.PageIndex].Data += this._WebService.GetSingleModelXml.lastResult.*[0].*[0];
So basically I had to get the index of the node in the XMLList that is the Data property. From there I could use the delete keyword to remove it from the list. The += adds my new node to the list.
I'm so used to using the ActiveX or Mozilla XmlDocument stuff where you call "SelectSingleNode" and then use "replaceChild" to do this kind of stuff. Oh well, at least this is in some forum where someone else can find it. I do not know the procedure for what happens when I answer my own question. Perhaps this insight will help someone else come along and help answer the question better!
Perhaps you could use replace instead?
var oldNode : XML = this._DataPages[this._Options.PageIndex].Data.(#Id == idOfReplacee)[0];
var newNode : XML = this._WebService.GetSingleModelXml.lastResult.*[0].*[0];
oldNode.parent.replace(oldNode, newNode);
I know this is an incredibly old question, but I don't see (what I think is) the simplest solution to this problem.
Theo had the right direction here, but there's a number of errors with the way replace was being used (and the fact that pretty much everything in E4X is a function).
I believe this will do the trick:
oldNode.parent().replace(oldNode.childIndex(), newNode);
replace() can take a number of different types in the first parameter, but AFAIK, XML objects are not one of them.
I don't immediately see the problem, so I can only venture a guess. The delete line that you've got is looking for the first item at the top level of the list which has an attribute "Id" with a value equal to idOfReplacee. Ensure that you don't need to dig deeper into the XML structure to find that matching id.
Try this instead:
delete (this._DataPages[this._Options.PageIndex].Data..(#Id == idOfReplacee)[0]);
(Notice the extra '.' after Data). You could more easily debug this by setting a breakpoint on the second line of the code you posted, and ensure that the XMLList looks like you expect.

Resources