Oozie fs:exists with variables - oozie

I'm struggeling on the following problem using variables in an Oozie workflow definition checking if a specific file was created.
It is working with absolute path like the following, but I cannot use an absolute path:
${fs:exists('/mypath/file.hql')}
In my case, the nameNode and the workflow id has to be replaced but in the decision node this is not working.
The variables are not replaced, what is the correct syntax to do this?
<decision name="check-hql-file-created">
<switch>
<case to="hive-exec-il2rl-hql4baseentity">
${fs:exists(${nameNode}'/tmp/oozie_tmp/'${wf:id()}'.hql')}
</case>
<default to="il2rl-loop"/>
</switch>
</decision>

it is working with concatenation like the following:
<switch>
<case to="hive-exec-il2rl-hql4baseentity">
${fs:exists(concat(concat(concat(concat(concat(nameNode, '/tmp/oozie_tmp/'), wf:id()), '_'), replaceAll(asJson, "\\{\"|,.+$", "")), '.hql')) == "true"}
</case>
<default to="il2rl-loop"/>
</switch>

Say you plan to use variables 'X' and 'Y', in some format such as /tmp/X/Y (let's call this 'PATH'), then:
Define X, Y and 'PATH' as a variable like so
PATH = /tmp/${X}/${Y}/
Then use PATH as:
${fs:exists(PATH)}
this works.
However, if you want to run this workflow via coordinators, it is better to first assign each of those variables separately too. Like so:
Define X, Y and 'PATH' as a variable like so
X = value1
Y = value2
PATH = /tmp/${X}/${Y}/
Then use PATH as:
${fs:exists(PATH)}
and do exactly the same in your coordinator.
Note: value1 and value2 can be Oozie expressions.

Related

How to Concatenate multiple repetitive nodes into a single node - BizTalk

I have something like this in an input XML
<OrderText>
<text_type>0012</text_type>
<text_content>Text1</text_content>
</OrderText>
<OrderText>
<text_type>ZT03</text_type>
<text_content>Text2</text_content>
</OrderText>
The above data I need to map after concatenating as the below schema
<Order>
<Note>0012:Text1#ZT03:Text2</Note>
</Order>
Can anyone please help?
I'm going to assume that your input actually has a Root node, as otherwise it is not valid XML.
<Root>
<OrderText>
<text_type>0012</text_type>
<text_content>Text1</text_content>
</OrderText>
<OrderText>
<text_type>ZT03</text_type>
<text_content>Text2</text_content>
</OrderText>
</Root>
Then all you need is a map like this
With a String Concatenate functoid with
Input[0] = text_type
Input[1] = :
Input[2] = text_content
Input[3] = #
That goes into a Cumulative Concatenate
This will give you an output of
<Order>
<Note>0012:Text1#ZT03:Text2#</Note>
</Order>
Note: There is a extra # at the end, but you could use some more functoids to trim that off if needed.
You can use the Value-Mapping Flattening functoid in a map, then feed the result of each into a Concatenate functoid to generate the result string. The map can be executed on a port or in an orchestration.

How to add a value to the existing element value and return it as a new value

This is the xml file.
<?xml version="1.0" encoding="UTF-8"?>
<root>
<AtcoCode> System-Start-Date= 2018-05-16T12:35:48.6929328-04:00, " ", System-End-Date = 9999-12-31, " ", 150042010003</AtcoCode>
<NaptanCode>esxatgjd</NaptanCode>
<PlateCode>
</PlateCode>
<CleardownCode>
</CleardownCode>
<CommonName>Upper Park</CommonName>
<CommonNameLang>
</CommonNameLang>
<ShortCommonName>
</ShortCommonName>
<ShortCommonNameLang>
</ShortCommonNameLang>
<Landmark>Upper Park</Landmark>
<LandmarkLang>
</LandmarkLang>
<Street>High Road</Street>
<StreetLang>
</StreetLang>
<Crossing>
</Crossing>
<CrossingLang>
</CrossingLang>
<Indicator>adj</Indicator>
<IndicatorLang>
</IndicatorLang>
<Bearing>NE</Bearing>
<NptgLocalityCode>E0046286</NptgLocalityCode>
<LocalityName>Loughton</LocalityName>
<ParentLocalityName>
</ParentLocalityName>
<GrandParentLocalityName>
</GrandParentLocalityName>
<Town>Loughton</Town>
<TownLang>
</TownLang>
<Suburb>
</Suburb>
<SuburbLang>
</SuburbLang>
<LocalityCentre>1</LocalityCentre>
<GridType>U</GridType>
<Easting>541906</Easting>
<Northing>195737</Northing>
<Co-ordinates>51.64255,0.04944</Co-ordinates>
<StopType>BCT</StopType>
<BusStopType>MKD</BusStopType>
<TimingStatus>OTH</TimingStatus>
<DefaultWaitTime>
</DefaultWaitTime>
<Notes>
</Notes>
<NotesLang>
</NotesLang>
<AdministrativeAreaCode>080</AdministrativeAreaCode>
<CreationDateTime>2006-11-06T00:00:00</CreationDateTime>
<ModificationDateTime>2010-01-16T07:58:02</ModificationDateTime>
<RevisionNumber>5</RevisionNumber>
<Modification>rev</Modification>
<Status>act</Status>
</root>
How to achieve this?
Question: Create the path range index for the status element and fetch all the documents that has status del
after fetching all the documents, you need to create the new element called currentreservationnumber under RevisionNumber element.
The value of the currentrevisionnumber will be +1 to the RevisionNumber.
I think the warning about sequential numbers is related to system-wide unique numbers/ids (like Oracle sequence), so not a worry in this case?
If you only ever have one RevisionNumber, and you can find it without a path index, you can maybe get by with element-value query on the RevisionNumber since it's already indexed.
Given that you get the document somehow, it could be as simple as:
let $doc := fn:doc ('/foo.xml')
let $rev-node := $doc/root/RevisionNumber
return xdmp:node-insert-after ($rev-node, <currentreservationnumber>{$rev-node + 1}</currentreservationnumber>)
though remember to consider locking if you are doing a big query/update. And you might need to switch to node-replace if there is already a currentreservationnumber.

How to use a variable with setwd()?

Example:
var<-"name"
setwd("/media/data/var")
gives an error 'cause directory "/media/data/var" does not exist, but "/media/data/name".
So, how to declare var as variable within a quoted string?
You have to use paste:
setwd(paste0("/media/data/",var))
Or you can use file.path:
setwd(file.path("/media/data",var))

XQuery To Select Related Node

Given the below XML, what would be the proper SQL XQuery to retrieve the SubscriberStatus where the SubscriberID is empty? Given the XML is stored in a column with the XML datatype.
<ObjectEntry>
<Key>Key1</Key>
<DicValue>
<ObjectEntry>
<Key>SubscriberStatus</Key>
<Value xsi:type="xsd:string">Active</Value>
<DicValue />
</ObjectEntry>
<ObjectEntry>
<Key>SubscriberID</Key>
<Value xsi:type="xsd:string" />
<DicValue />
</ObjectEntry>
</DicValue>
</ObjectEntry>
Try this:
If $node holds your xml fragment then
$node//ObjectEntry[DicValue/ObjectEntry[Key eq "SubscriberStatus"] and DicValue/ObjectEntry[Key eq "SubscriberID"][Value ne ""]]
will give you back the ObjectEntry parent for the non empty SubscriberIDs
This is a simply XPath expression, there is no need for true XQuery. XPath is a subset of XQuery. Given that you want the <Value/> element of the SubscriberStatus you can get it like the following:
//ObjectEntry/DicValue[ObjectEntry[Key = "SubscriberID"]/Value = ""]/ObjectEntry[Key = "SubscriberStatus"]/Value
This fetches all ObjectEntries which do have an empty SubscriberID and then navigates to the SubscriberStatus. If you just want the actual string, you cann append /string()
Thanks for the suggestions! Unfortunately they didn't do what I was asking, but did help me get the right syntax. Here's a solution that works.
select Request.query('//ObjectEntry/DicValue/ObjectEntry[Key = "SubscriberStatus"]/Value') as SubscriberStatus
from RequestLog
where
Request.exist('//ObjectEntry/DicValue[ObjectEntry[Key = "SubscriberID" and Value = ""]]') = 1

NLog Colored Console

I am using NLog to log errors. Here is the Config code
<target name="console" xsi:type="AsyncWrapper" >
<target xsi:type="ColoredConsole" layout="${longdate:padding=-10}${callsite:className=false:includeSourcePath=false:methodName=false} | ${message}" >
<highlight-row condition="level >= LogLevel.Info" foregroundColor="Green" backgroundColor="NoChange"/>
</target>
</target>
I have a custom property set on the log event like
private LogEventInfo GetLogEvent(string loggerName, LogLevel level, string message, ConsoleColor color)
{
var logEvent = new LogEventInfo(level, loggerName, message);
logEvent.Properties["color"] = color;// color= any console color
}
and this sets the "color" property.(lets say "Red" here)
and I am trying to use this "color" property in the target like
<highlight-row condition="equals('${color}','Red')" foregroundColor="Red" backgroundColor="NoChange"/>
this dosent work and I tried
<highlight-row condition="equals('${event-context:item=color}','Red')" foregroundColor="Red" backgroundColor="NoChange"/>
but no luck.
Am I missing something or is there a better way of doing this? Can we use Layout renderers in this case? If yes how do we implement this?
First, since you are storing values in LogEventInfo.Properties, you should be using your second configuration example, which gets the value from event-context.
I have not used the ColoredConsoleTarget, so take this as a suggestion, not as something that I know for a fact will work.
I suspect that the NLog Condition object does not know about the ConsoleOutputColor enum. So, when you store a ConsoleOutputColor enum value in LogEventInfo.Properties, the Condition does not know that 'Red' (in the condition) refers to ConsoleOutputColor.Red. I have two suggestions:
First option: store the string value of the ConsoleOutputColor in LogEventInfo.Properties. Using ToColor might be sufficient. Something like this:
var logEvent = new LogEventInfo(level, loggerName, message);
logEvent.Properties["color"] = color.ToString();
Then, in your Condition, you should be able to compare against the ConsoleOutputColor string values (what you have in your configuration might be right, if you store the color name string as I suggested).
If that doesn't work, you could try...
Second option: store the ConsoleOutputColor value in LogEventInfo.Properties, as you are doing now, but change your condition in the config file to compare the "color" from event-context to the numeric value of the ConsoleOutputColor values. Something like this (I have not tried this so I don't know for sure that it is correct):
<highlight-row condition="equals('${event-context:item=color}','12')" foregroundColor="Red" backgroundColor="NoChange"/>
(In the ConsoleOutputColor enum, Red is 12).

Resources