Have anyone used the Cumulative Maximum functoid and noticed performance problems?
Abstract
If one wants to map the maximum value of a field you can use the functoid Cumulative Maximum.
Problem
After we had used it for a while we noticed degraded performance on larger files.
Examining the xslt one notices that the max calculation is made for each looping record...
One could move the calculation to the grand parent, and point out the new xslt in the Custom XSL Path, but I really like to keep the possibility to map in the mapping tool.
Any suggestions?
Kind Regards
Martin Bring
http://martinbring.blogspot.com
By removing the Cumulative Maximum and adding 3 scripting functoids, doing the calculation in another way, the problem is solved. Mapping time decreased by a factor of 40.
11 Mb, 10 000 rows, was previously mapped in 200 minutes is now mapped in 5 minutes.
Solution
One scripting functoid, "Inline XSLT Call Template" with no input or output, containing the max() portion of the library from EXSLT Math library found here. Instead of using the whole library I unzipped the file and "extracted" the max() template.
<xsl:template name="GetMax">
<xsl:param name="nodes" />
<xsl:choose>
<xsl:when test="not($nodes)">NaN</xsl:when>
<xsl:otherwise>
<xsl:for-each select="$nodes">
<xsl:sort data-type="number" order="descending" />
<xsl:if test="position() = 1">
<xsl:value-of select="number(.)" />
</xsl:if>
</xsl:for-each>
</xsl:otherwise>
</xsl:choose>
</xsl:template>
One scripting functoid, "Inline XSLT Call Template" with no input or output, containing a variable which select attribute points at the template with the node set to calculate
<xsl:variable name="var:MaxValueDate">
<xsl:call-template name ="GetMax">
<xsl:with-param name ="nodes" select="Root//Parent/ValueToCalculate" />
</xsl:call-template>
</xsl:variable>
One scripting functoid, "Inline XSLT" with one output, using the variable to populate an output element with its value.
<OutputElement>
<xsl:value-of select="$var:MaxValueDate" />
</OutputElement>
Voila!
Related
I need to delete several occurrences of a symbol (single quote) within comments in an XML file in R, and then save it back to XML. I have to do this in 1000s of XML file.
In each XML file the single quote appears more than 50 times and present in different child hierrachial structure (some in child and some in sub-child). But always present within the comments.
I Tried using XML package in R. I first tried to enable this one XML file, but didn't know how to proceed further.
library(XML);library(xml2);library(methods);library(tidyverse)
#Read one XML file
filepath <- "C:/Users/PeriaPr/Desktop/repex1.xml"
onefile <- xmlTreeParse(gsub("'","",readLines(filepath)),asText = TRUE)
xmlroot <- xmlRoot(onefile)
var <- xmlSApply(xmlroot, function(x) xmlSApply(x, xmlValue))
Here is a reproducible example of my XML file. The single quotes (around Orange, Apple and Banana) need to be removed in this multiple tree hierarchical structure. The quotes occur almost 50 times within one XML file, and I need to process (delete single quotes) 1000s of XML files
<?xml version = "1.0" encoding = "windows-1252"?><document id="myrepex.xml">
<action_step step_no="1.3.1.1">
<step>1</step>
<title><![CDATA[Part1 - 'Apple']]></title>
<start><![CDATA[2019/08/09 7:57:17]]></start>
<duration><![CDATA[0 Hr. 12 Min. 22 Sec.]]></duration>
<status><![CDATA[Passed]]></status>
</action_step>
<action_step step_no="1.4.1.1">
<step>2</step>
<title><![CDATA[Part2 - 'Orange']]></title>
<start><![CDATA[2019/08/09 8:09:39]]></start>
<duration><![CDATA[0 Hr. 32 Min. 55 Sec.]]></duration>
<status><![CDATA[Passed]]></status>
</action_step>
<action_step step_no="1.5.1.1">
<step>68</step>
<title><![CDATA[Part3 - 'Banana']]></title>
<start><![CDATA[2019/08/09 8:42:35]]></start>
<duration><![CDATA[0 Hr. 36 Min. 28 Sec.]]></duration>
<status><![CDATA[Passed]]></status>
</action_step>
<action_step2 secondchild="secondchild">
<action_step2subchild subchild="subchild">
<title><![CDATA[Part3 - 'Banana']]></title>
</action_step2subchild>
</action_step2>
To delete all single quotes in the content of title elements, you can use the following XSLT 3.0 stylesheet:
<xsl:transform version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:mode on-no-match="shallow-copy"/>
<xsl:template match="title/text()">{translate(.,"'","")}</xsl:template>
</xsl:transform>
If you want to retain the CDATA tagging (which is considered by XSLT to be pure noise), add an xsl:output declaration with cdata-section-elements attribute.
To apply this transformation to many source XML files, you could either (a) use Saxon's ability to process all files in a directory from the command line, (b) put the control logic within the XSLT stylesheet (using the collection() function), or (c) use some external control logic, e.g. XProc, Ant, or a shell script.
Consider this XSLT 1.0 script and then iterate with R's xslt package (extension of xml2):
XSLT (save as .xsl file to be read in R)
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" omit-xml-declaration="no" indent="yes"
cdata-section-elements="title start duration status"/>
<xsl:strip-space elements="*"/>
<xsl:template match="node()|#*">
<xsl:copy>
<xsl:apply-templates select="node()|#*"/>
</xsl:copy>
</xsl:template>
<xsl:variable name="single_quote">'</xsl:variable>
<xsl:template match="title">
<xsl:copy>
<xsl:value-of select="translate(., $single_quote, '')"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>
Online Demo
R
library(xml2)
library(xslt)
# RETRIEVE ALL XML FILES
setwd("C:/path/to/xml/files")
xmls <- list.files(pattern="xml")
# LOAD XSLT
style <- read_xml('C:/path/to/Script.xsl', package = "xslt")
# USER-DEFINED METHOD
proc_xml <- function(xml_file) {
# LOAD XML AND XSLT
doc <- read_xml(xml_file)
# TRANSFORM XML
new_xml <- xslt::xml_xslt(doc, style)
# SAVE NEW XML (REMOVE paste0 TO OVERWRITE ORIGINAL XML)
write_xml(new_xml, paste0("new_", xml_file))
return(new_xml)
}
# ITERATE THROUGH XML FILES
new_xmls <- lapply(xmls, proc_xml)
I have a small problem - I have to list all the labels of several large shared projects, so labels that are missing a translation are identified updated.
Now what I'm looking is something like Palle Agermark's label export job (http://www.agermark.com/2011/10/export-and-import-labels-for.html) but one that goes through one or more projects and pulls all the label IDs (regardless of label series) and values for three languages to Excel.
Can this be done, any pointers? :)
This expands the idea of Jan B. Kjeldsen's last comment to get the labels from the project xpo file. The labels are saved in the xpo file in the following XML structure:
<Table:Record name="TmpSysLabel" xmlns:Table='urn:www.microsoft.com/Formats/Table'>
<Table:Field name="Language">de</Table:Field>
<Table:Field name="Label">Geplante Produktionsaufträge</Table:Field>
<Table:Field name="Description"></Table:Field>
<Table:Field name="LabelId">#SYS119128</Table:Field>
<Table:Field name="SysLabelApplModule">0</Table:Field>
<Table:Field name="recVersion">0</Table:Field>
<Table:Field name="Partition">5637144576</Table:Field>
</Table:Record>
My first idea was to import this XML to Excel, but Excel does not play nice with column names in attributes. So instead I wrote an xsl that transforms the XML into something like the following:
<?xml version="1.0" encoding="UTF-8"?>
<labels>
<record>
<Language>de</Language>
<Label>Geplante Produktionsaufträge</Label>
<Description />
<LabelId>#SYS119128</LabelId>
<SysLabelApplModule>0</SysLabelApplModule>
<recVersion>0</recVersion>
<Partition>5637144576</Partition>
</record>
</labels>
To do this transformation, the original XML has to be cleaned up a bit, replace all occurences of Table:Record with record and all occurences of Table:Field with field. Also add a root XML node labels. It should look like
<labels>
<record>
<field name="Language">de</field>
<field name="Label">Geplante Produktionsaufträge</field>
<field name="Description"></field>
<field name="LabelId">#SYS119128</field>
<field name="SysLabelApplModule">0</field>
<field name="recVersion">0</field>
<field name="Partition">5637144576</field>
</record>
</labels>
Now for the fun part (at least for me because I had not much prior knowledge of xsl syntax), the xsl transformation file. I ended up with the following, which will produce the wanted XML using http://www.freeformatter.com/xsl-transformer.html:
<?xml version="1.0" encoding="utf-16"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="xml" omit-xml-declaration="yes" indent="no" />
<xsl:template match="/">
<labels>
<xsl:for-each select="Labels/*">
<record>
<xsl:for-each select="*">
<xsl:element name="{#name}">
<xsl:value-of select="." />
</xsl:element>
</xsl:for-each>
</record>
</xsl:for-each>
</labels>
</xsl:template>
</xsl:stylesheet>
The resulting XML can then be imported into Excel using the Excel developer Tools for XML. The result will look like
The simplest solution would be to go to the label source directly.
In AX 2009 the labels are stored in the application folder and named like like axSYSen-us.ald with the following parts:
ax - the file prefix
SYS - the label name, #SYS123 labels are stored there
en-us - the language specifier
ald - the label file extension
The files are in text format and can easily be imported to Excel or edited with an text editor.
You could import to Excel, one sheet for each label file, then combine the output in a separate sheet (in the same workbook) using Excel formulas.
In AX 2012 the labels are stored in the model database and the .ald files are created on startup.
My XSLT is shown below.
aic is a namespace.
What is cstyle?
is it a built-in XSLT element/function?
Or an element within the expected input xml?
<xsl:stylesheet exclude-result-prefixes="aic"
xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:aic="http://ns.adobe.com/AdobeInCopy/2.0/" >
<xsl:template match="/">
</xsl:template>
<xsl:template match="aic:cstyle[contains(#name,'bold')]">
</xsl:template>
</xsl:stylesheet>
It is an element within the expected input XML. The XPaths in an XSLT's match attributes are generally applied to contents from the input XML.
Exactly as in my answer to your previous question, aic:cstyle is a selector that matches elements whose local name is cstyle and whose namespace URI is http://ns.adobe.com/AdobeInCopy/2.0/ (the URI bound to the aic prefix in the xsl:stylesheet element). Thus
<xsl:template match="aic:cstyle[contains(#name,'bold')]">
is a template that will apply to any {http://ns.adobe.com/AdobeInCopy/2.0/}cstyle element that has a name attribute that contains the substring bold. (So, to answer your question directly: the expression in question will match elements in the input streams for which the stylesheet was written.)
As with any new programming language, I would strongly recommend that you find a decent tutorial and work through that to get comfortable with the syntax and idioms of the language through simple examples before you start trying to decode a large and complex XSLT that you've inherited from elsewhere.
Relating to this question, is it possible to use variables inside xsl:key? I want to do smth like this:
<xsl:key name="ChargesKey" match="$ChargesForDisplay/charge" use="Name"/>
I'm using XSLT 1.0 with ASP.Net
I believe I can safely assume that you are referring to dynamically-generated node-set variables (as opposed to those selected from the source DOM, which are trivial), and yes it is possible to perform a key-match on the contents of a dynamically generated node-set variable (as I demonstrate for this question).
Assuming you have a variable like this:
<xsl:variable name="ChargesForDisplay">
<charge>
<Name>Name1</Name>
</charge>
<charge>
<Name>Name2</Name>
</charge>
<charge>
<Name>Name1</Name>
</charge>
<charge>
<Name>Name3</Name>
</charge>
</xsl:variable>
You would define the key like this:
<xsl:key name="ChargesKey" match="charge" use="Name"/>
And then you can apply it like this:
<xsl:template match="/">
<xsl:apply-templates select="msxsl:node-set($ChargesForDisplay)" />
</xsl:template>
<xsl:template
match="charge[generate-id(.)=generate-id(key('ChargesKey',Name)[1])]">
<xsl:variable name="matchingItems" select="key('ChargesKey', Name)" />
...
</xsl:template>
Of course, if the variable contains a selection of nodes from the source XML DOM, then it's just the same approach, except you don't need to use msxsl:node-set().
I suspect that having a key on a node-name that's also present in the source XML document or multiple dynamically-generated node-set variables may cause grouping to produce unexpected results(because the key() function would locate nodes from both the variable and the source document). For this reason, I'd suggest defining keys on nodes that would only be present in one particular variable and nowhere else.
The match attribute of xsl:key must be a valid pattern and $x/y is not a valid pattern. So, the answer is no. Now tell us what you are trying to achieve and we can help you achieve it. (JLRishe makes some wild guesses, which s/he calls "wild assumptions", and which may well be right; but I don't know where the guesses come from).
I am using some code to subtract one date from another using XSLT 2.0:
<xsl:template match="moveInDate">
<xsl:value-of select="current-date() - xs:date(.)"/>
</xsl:template>
This works, however it leaves me with an answer of P2243D, which I assume corresponds to a "Period of 2243 Days" (which is correct in terms of the math).
Since I only need the number of days, not the P and the D, I know I could use substring or something similar, but as a newbie to XSLT, I'm curious if there is a better, more elegant way to do this than simple string manipulation.
You could simply use fn:days-from-duration() to get the duration as a xs:integer:
days-from-duration($arg as xs:duration?) as xs:integer?
Returns an xs:integer representing the days component in the canonical lexical representation of the value of $arg. The result may be negative.
See the XQuery 1.0 and XPath 2.0 Functions and Operators specification for more information.
In your case:
<xsl:template match="moveInDate">
<xsl:value-of select="days-from-duration(current-date() - xs:date(.))"/>
</xsl:template>
Hope this helps!
EDIT: You could also do it the way you say, with substring processing. But as you point out, it's not prefered. If you for some reason would like to do something similar you need to think of the data types. The result of current-date() - xs:date(.) is returned as xs:duration which cannot be processed by the substring functions without being casted:
<xsl:template match="moveInDate">
<xsl:variable name="dur" select="(current-date() - xs:date(.)) cast as xs:string"/>
<xsl:value-of select="substring-before(substring-after($dur, 'P'), 'D')"/>
</xsl:template>