Mapping multiple source structures to a single looping destination structure in BizTalk - biztalk

I am trying to map parts of the following source structure that has two sets of properties - one flat and one looped:
Source Document
<root>
<flat>
<prop1>foo</prop1>
<prop2>bar</prop2>
...
</flat>
<loop>
<prop>
<qual>propA</qual>
<data>baz</data>
<more>blah</more>
</prop>
<prop>
<qual>propB</qual>
<data>qux</data>
<more>bhal</more>
</prop>
...
</loop>
</root>
Specifically, the flat part is the PO1 segment of an X12 850 EDI document, and the looping properties are the subsequent REF segments.
These should be mapped to a looping destination structure of key-value pairs that looks like this:
Destination Document
<root>
<props>
<prop>
<name>prop1</name>
<value>foo</value>
</prop>
<prop>
<name>propA</name>
<value>baz</value>
</prop>
</props>
</root>
I would like to map only some of the values, depending on the property name.
What I've Tried
I have successfully mapped the flat portion to the destination using a table looping functoid and two table extractor functoids:
I have also successfully mapped the looping portion to the destination using a looping functoid and some equality checks to select only certain qual values:
When I attempt to include both of these mappings at the same time, the map succeeds, but doesn't generate the combined output.
The Question
How I can I map both sections of the source document to the same looping section in the destination document?
Update 1
Turns out I had oversimplified the problem; the flat group of properties actually contains the property name in one node and the value in another node. This is what they actually look like:
<flat>
<name1>prop1</name1>
<value1>foo</value1>
<name2>prop2</name2>
<value2>bar</value2>
...
</flat>
The concept of #Dijkgraaf's answer still works with this change if you use a Value Mapping (Flattening) functoid to get the property name from the correct location.

Usually the only way to solve this is with either
Inline Custom XSLT via the Scripting Functoid
Custom XSLT setting Custom XSLT Path for the whole map
Having an intermediate schema that contains two Option nodes and having two maps. The first that maps the flat structure to one node and the looping to the second. Then a second map that loops across both and maps to to the same node.
In your case however, you need to have both (prop1,prop2,..) and the looping prop linked to the same looping functoid, and linking to the name and value and setting the link properties on the links from prop1,prop2 etc. to Copy name instead of value.
With your sample input that gives
<root>
<props>
<prop>
<name>prop1</name>
<value>foo</value>
</prop>
<prop>
<name>prop2</name>
<value>bar</value>
</prop>
<prop>
<name>propA</name>
<value>baz</value>
</prop>
<prop>
<name>propB</name>
<value>qux</value>
</prop>
</props>
</root>

Related

xQuery - How to query based on number of elements in XML document?

I'm still new to xQuery / MarkLogic and I'm having trouble understanding how to query based on the number of elements in the XML document. For example, imagine I have a database of XML documents roughly similar to the following:
<?xml version="1.0" encoding="UTF-8"?>
<bookstore>
<book category="cooking">
<title lang="en">Everyday Italian</title>
<author>Giada De Laurentiis</author>
<year>2005</year>
<price>30.00</price>
</book>
<book category="children">
<title lang="en">Harry Potter</title>
<author>J K. Rowling</author>
<year>2005</year>
</book>
<book category="web">
<title lang="en">Learning XML</title>
<author>Erik T. Ray</author>
<year>2003</year>
<price>39.95</price>
</book>
</bookstore>
As you can see in book[2], price is missing. Most documents in the database I'm working with would either have the child element price for each book or no price element attached to any of the book elements. My goal is to find only the documents where some of the child elements are missing (like the above XML); and ignore the documents where either all the child elements exist or where none of the child elements exist. So in my head the logic is something along the lines of "return results where the number of price elements is < the number of book elements AND > 0."
The best I can do so far is the following query:
let $some-docs := cts:search(fn:collection('/my/collection'),
cts:and-query((
cts:element-query(xs:QName("book"), cts:true-query()),
cts:not-query(cts:element-query(xs:QName("price"), cts:true-query()))
)))
return (xdmp:node-uri($some-docs))
But this obviously only returns documents where book elements exist and no price elements exist. I need a way of indicating I want the documents where the price element exists, but is missing for some books.
I prefer a solution that is using the cts:search function, but any help is appreciated
I need a way of indicating I want the documents where the price element exists, but is missing for some books.
So basically you need to find documents that have both <bookstore><book><price/></book></bookstore> and ones missing the child <price/> element?
The simplest thing to do is modify the existing documents using a tool like CORB to include an element indicating that document matches your criteria or perhaps place them in a distinct collection. Then just use CTS to return documents with that added indicator.
If you don't want to touch the dataset you could create a field range index on /bookstore/book/price and /bookstore/book[not(./price)]/title. Then you just need to query for documents where both indexes are present with something like:
cts:and-query((
cts:field-word-query("field1", "*", ("wildcarded")),
cts:field-word-query("field2", "*", ("wildcarded"))
))
Getting the count of elements within a document isn't something that is exposed and available for a query. You could apply a predicate filter and test if there are any book that do not have a price for the docs returned from the search for those bookstore docs:
cts:search(fn:collection('/my/collection'),
cts:element-query(xs:QName("book"), cts:true-query())
)[bookstore/book[not(price)]]
return results where the number of price elements is < the number of book elements AND > 0
You could write not(count(//price) = (count(//book), 0))
or perhaps
empty(//price) or empty(//book[not(price)]
It seems a very strange query though. Perhaps you should be using a schema for validation?

Using Java API for Container Constraint (nested)

I'm using MarkLogic v8.
I am trying to apply a container constraint on a structured query to return only documents with value x in element c (nested within elements a and b).
queryBuilder.containerConstraint() takes a parameter for an option name and a StructuredQueryDefinition. My option looks like this:
<options xmlns='http://marklogic.com/appservices/search'>
<constraint name='language'>
<element name=\"name\" ns=\"\"/>
</constraint>
</options>
"name" is the name of the innermost element (c) containing the value I want to reference against. Is this how the option should be constructed, or should 'name' instead be the name of the outermost element?
How should the StructuredQueryDefinition (that is accepted as a parameter by containerConstraint()) be constructed? Should I be writing raw XML, or are there contruction methods to be passed in?
Is there a better way to do this? I already have a working Term search, I just need to be able to filter by a property set inside the document.
I think I found an answer:
Option was as follows:
<search:options
xmlns:search='http://marklogic.com/appservices/search'>
<search:constraint name='language'>
<search:word>
<search:element name='name' ns=''/>
</search:word>
</search:constraint>
</search:options>
Then called the option in a Word Constraint:
queryBuilder.wordConstraint("language", MY_LANGUAGE)
This appears to do what I wanted it to.

XQUERY to delete a node with an ID property in BaseX

I am using BaseX to store XML data with multiple nodes in the following format:
<root>
<item id="65816" parent_id="45761" type="test">
<content>
<name>Name of my node on the tree</name>
</content>
</item>
</root>
The code above is essentially one typical node under 'root'.
Now, I am trying to delete a node based on the 'id' property of the 'Item' object.
I looked at the documentation on BaseX.org but that does not explicitly tell me how to deal with nodes which have IDs linked to it. I am trying to something like this:
XQUERY delete node //root/item.id="65816"
Note: The above line doesn't work. That is just to give an idea of what I am trying to achieve.

How can I attach a datarule to a data broker's key-generation operation?

I have a Workflow SQL databroker in which the ID field is a sequential number with additional character padding, i.e. P001 or TM234.
I have defined the query for ID generation as:
<key-generation field="id" query="queries/asset-patent-get-id" />
which simply retrieves next sequence value based on provided parameters. So for every created record I need to make sure that by the time ID gets stored in database it was already padded as per required rules and stored as a string rather than just a number.
First thing to note: a data rule isn't bound to an operation, but instead to a field. So in this case, just think of the key-generation operation as retrieving the next number in a sequence, nothing more. Once the sequence value is retrieved, an appropriate bound data rule should be able to format the sequence value before it is persisted.
You should be able to define a data rule something like this:
<!-- datarules/format-asset-patent-id/definition.xml -->
<data-rule name="format-asset-patent-id"
factory-class="com.aviarc.framework.datarule.xml.DefaultXMLDataRuleProviderFactoryImpl"
datarule-class="com.aviarc.framework.datarule.workflow.WorkflowDataRule">
<parameters>
<parameter name="field" mandatory="y"/>
</parameters>
<event name="onBeforeDatasetPersisted" workflow="workflows/format-asset-patent-id" field="{#field}"/>
</data-rule>
Then in the format-asset-patent-id workflow you can take care of formatting the id. (Note: I haven't had the chance to test whether the onBeforeDatasetPersisted event fires before or after the key-generation operation is performed.)
Then you could bind it with a data binding, for example:
<!-- databindings/asset-patent.xml -->
<databinding databroker="asset-patent"
factory-class="com.aviarc.framework.databinding.basic.BasicDataBindingFactoryImpl">
<on-dataset-bound>
<data-rules>
<format-asset-patent-id field="id"/>
</data-rules>
</on-dataset-bound>
</databinding>

Can't suppress nodes in BizTalk mapping

Afternoon all,
I'm learning BizTalk and am stuck on this problem.
There's a choice group with either telephone or address.
I can get the some of the address fields to be suppressed when outputting the telephone.
To make things simpler I'll just discuss the "CityName" field .
I've tried the following:
Mapped the "City" (text) node in the input file to "CityName" in the output. Connected an Exists logical functoid to the "Address1" node in the input node and the "CityName" in the output. I always get a city name with text filled in.
Mapped the "City" node in the input file and the Exists logical functoid to a Value Mapper functoid. Connected the Value Mapper functoid to the CityName in the output. I get a city name in the address node (Correct) and an empty node in the telephone node (Incorrect).
Mapped the "City" node in the input file and the Exists logical functoid to a Value Mapper (Flattening) functoid. Connected the Value Mapper (Flattening) functoid to the CityName in the output. I get a city name in the address node (Correct) and an empty node in the telephone node (Incorrect).
I thought one of the last two should have worked but both give an empty node instead of a suppressed node. The empty nodes have no attributes.
The CityName node is described by this schema type:
<xsd:complexType name="NameType">
<xsd:simpleContent>
<xsd:extension base="xsd:string">
<xsd:attribute name="languageID" type="xsd:language" use="optional">
</xsd:attribute>
<xsd:attribute name="sequence" type="IntegerNumericType">
</xsd:attribute>
<xsd:attribute name="sequenceName" type="StringType" />
</xsd:extension>
</xsd:simpleContent>
</xsd:complexType>
<xsd:simpleType name="IntegerNumericType">
<xsd:restriction base="xsd:integer" />
</xsd:simpleType>
<xsd:simpleType name="StringType">
<xsd:restriction base="xsd:string" />
</xsd:simpleType>
Is there something in the type definition that's forcing an empty node to be created?
Thanks
EDIT: Below is the incorrect output. The second and third Communication nodes have
an Address Child node and should not.
I have a looping functoid on the phone numbers and the first address line in a flat input schema. That produces the three Communications nodes. I'm using a Logical Existance functoid on the address line to suppress the phone info in the first node. I have a Logical Not functoid on the output of that which is tied directly to the Address node in the output schema. I thought that should turn off the entire address node but it doesn't.
I tried adding a Value Mapping functoid connected to the Logical Existance functoid to the City Name and Postal code elements and that makes them empty, but they just will not go away. ARG!
I checked and the fields are all set to zero MinOccurs and I don't believe they're assigned a default value.
<ns0:Communication>
<ns0:Address>
<ns0:AddressLine sequence="1">1234 My St</ns0:AddressLine>
<ns0:AddressLine sequence="2">Apt. 2</ns0:AddressLine>
<ns0:CityName>Kansas City</ns0:CityName>
<ns0:CountrySubDivisionCode name="State">MO</ns0:CountrySubDivisionCode>
<ns0:CountrySubDivisionCode name="County">Jackson</ns0:CountrySubDivisionCode>
<ns0:CountryCode>US</ns0:CountryCode>
<ns0:PostalCode>64099</ns0:PostalCode>
</ns0:Address>
</ns0:Communication>
<ns0:Communication sequence="1">
<ns0:ChannelCode>Telephone</ns0:ChannelCode>
<ns0:UseCode>Personal</ns0:UseCode>
<ns0:DialNumber>1234567890</ns0:DialNumber>
<ns0:Address>
<ns0:CityName />
<ns0:CountryCode />
<ns0:PostalCode>64099</ns0:PostalCode>
</ns0:Address>
</ns0:Communication>
<ns0:Communication sequence="2">
<ns0:ChannelCode>Telephone</ns0:ChannelCode>
<ns0:UseCode>Business</ns0:UseCode>
<ns0:DialNumber>0987654321</ns0:DialNumber>
<ns0:Address>
<ns0:CityName />
<ns0:CountryCode />
<ns0:PostalCode>64099</ns0:PostalCode>
</ns0:Address>
</ns0:Communication>
It would help to have sample input and output xml snippets. Without that, I can guess at the xml structures. If they look something like this, then the below mapping should work fine:
Do your schemas look like that? The output produced by such a map is valid IF the <Choice> in the destination schema is set to Max Occurs = unbounded. If that's not the case, and if you can only have one Contact output, then you would have to only output the Phone if the Address is not there, like this:
Of course, that seems a bit silly, since one would expect to have both an Address and a Phone in the source xml, and the destination xml prevents you from having both.
If none of these scenarios matches up with yours, then please edit your question to provide more details.
The min & max occurrences on the schema defaults to 1. The mapper looks at that when generating the XSLT. Try setting Min Occur to 0.
The way to solve this issue is to use xsl directly instead of the built in mapper.
With xsl you can control when and how values are selected. That's very difficult
when using multiple looping functoids.

Resources