I'm running a simple Workflow with Oozie 4.0.0. I need to run bash scripts through SSH actions and then capture the stdout.
This is the xml:
<workflow-app name="Test" xmlns="uri:oozie:workflow:0.4">
<start to="LS"/>
<action name="LS">
<ssh xmlns="uri:oozie:ssh-action:0.1">
<host>user#host</host>
<command>ls</command>
<capture-output/>
</ssh>
<ok to="Mail"/>
<error to="kill"/>
</action>
<action name="Mail">
<email xmlns="uri:oozie:email-action:0.1">
<to>myaddress#mail.com</to>
<subject>Test mail</subject>
<body>${wf:actionData('LS')}</body>
</email>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>
</workflow-app>
The body of the mail contains only this:
{}
The same script with shell action works without problems, but I need the SSH action.
I'm going crazy, anybody could help me?
I've found the problem.
Seeing the logs I've noticed that the SShActionExecutor look at the wrong stdout file. This is the the command that I found in the log
ssh -o PasswordAuthentication=no -o KbdInteractiveDevices=no -o StrictHostKeyChecking=no -o ConnectTimeout=20 workflow#redhat5 cat oozie-oozi/0000008-140708170302737-oozie-oozi-W/fileProcessing--ssh/22350.0000008-140708170302737-oozie-oozi-W#fileProcessing#1.stdout
But the generated stdout file is /home/instadoc/oozie-oozi/0000008-140708170302737-oozie-oozi-W/fileProcessing--ssh/22350.0000008-140708170302737-oozie-oozi-W#fileProcessing#0.stdout
If you relaunch the action the ID that prepend the .stdout extension could vary (ie: #9.stdout, #4.stdout) but the SShActionExecutor looks always at #0.stdout.
Related
I have a yaml file where I want to write a XML:
script:
- >
cat > settings.xml << EOF
<?xml version="1.0" encoding="UTF-8"?>
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<servers>
<server>
<username>$USERNAME</username>
<password>$PASSWORD</password>
<id>central</id>
</server>
</servers>
</settings>
EOF
If I run the cat command in my terminal, it works but when it's executed by my pipeline, this command become a single line failing:
cat > settings.xml << EOF <?xml version="1.0" encoding="UTF-8"?> <settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <servers> <server> <username>$USER</username> <password>$PASSWORD</password> <id>central</id> </server> </servers> </settings> EOF
With the error:
parse error near `<'
Any idea how to fix this error or generate a well formated XML?
There are two problems:
Incorrect syntax (incorrect cat > settings.xml << EOF instead of correct cat << EOF > settings.xml)
The end-string EOF must be at the beginning of the line but it isn't.
To run this from the command line:
Type cat << EOF > settings.xml and press ENTER.
Paste (or type by hand) your XML document (without the end-string EOF) and press ENTER.
Type EOF (make sure it is at the beginning of the line!) and press ENTER.
You will be presented with the command prompt. Type ls -l settings.xml to see the newly created file.
To run this from a script:
Open your text editor and follow the steps 1, 2 and 3.
Save it under the name, lets say script.sh and make it executable:
chmod 755 script.sh
Run it:
./script.sh
When the command promt is back, type ls -l settings.xml, to see the newly created file.
I want to create Salesforce Dynamic package.xml file. I followed the process mentioned in this (apexandbeyond.wordpress.com/2017/03/15/dynamic-package-xml-generation/) blog, it is neither updating not inserting dat in xml file. I am trying following way but i doesn't work for me. please help me on this.
xmlstarlet edit -L --subnode "/Package/types[name='ApexClass']" --type elem -n members -v "LoginBroker" testpackage.xml > edipackage.xml
xmlstarlet edit --insert /Package -t attr -n xmlns -v "http://soap.sforce.com/2006/04/metadata" testpackage.xml > edipackage.xml
Original file: testpackage.xml
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
<types>
<members>*</members>
<name>ApexClass</name>
</types>
<types>
<members>*</members>
<name>ApexComponent</name>
</types>
<types>
<members>*</members>
<name>ApexPage</name>
</types>
<version>48.0</version>
</Package>
Expected result:
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
<types>
<members>LoginBroker</members>
<name>ApexClass</name>
</types>
<types>
<version>48.0</version>
</Package>
If I understand you correctly, you are looking for something like this:
xmlstarlet edit -N x="http://soap.sforce.com/2006/04/metadata"\
-u "//x:Package//x:types[x:name[1][text()='ApexClass']]/x:members" \
-v 'LoginBroker' \
testpackage.xml > edipackage.xml
Executed the command "sudo -u oozie ./bin/oozie-setup.sh sharelib create -fs hdfs://localhost:54310 -locallib oozie-sharelib-4.1.0.tar.gz"
Iam getting the error "Error: E0902: Exception occured: [User: oozie is not allowed to impersonate oozie]" when trying to execute sharelib command.
In core-site.xml in hadoop conf, change value as
<property>
<name>hadoop.proxyuser.oozie.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.oozie.groups</name>
<value>*</value>
</property>
And restart hadoop services and try again
Just wondering if anyone can help me.
I'm currently building a pfsense firewall, which uses VPN connections to secure the traffic.
The VPN provider does provide a port forwarding mechanism, but the incoming port number changes every hour. I have a script which allows me to discover the new port, but I need a scripted way to modify the port forward settings in the firewall to match.
A snippet of the firewall config file that controls this is as follows:
<?xml version="1.0"?>
<pfsense>
<nat>
<rule>
<source>
<any/>
</source>
<destination>
<any/>
<port>53400</port>
</destination>
<protocol>tcp/udp</protocol>
<target>192.168.0.15</target>
<local-port>53400</local-port>
<interface>opt2</interface>
<descr><![CDATA[Torrent]]></descr>
<associated-rule-id>nat_52d81d2dc904f5.77023355</associated-rule-id>
<created>
<time>1389894957</time>
<username>admin#192.168.0.20</username>
</created>
<updated>
<time>1389980696</time>
<username>admin#192.168.0.20</username>
</updated>
</rule>
</nat>
<filter>
<rule>
<id/>
<type>pass</type>
<interface>opt2</interface>
<ipprotocol>inet</ipprotocol>
<tag/>
<tagged/>
<max/>
<max-src-nodes/>
<max-src-conn/>
<max-src-states/>
<statetimeout/>
<statetype>keep state</statetype>
<os/>
<protocol>tcp/udp</protocol>
<source>
<any/>
</source>
<destination>
<address>192.168.0.15</address>
<port>53400</port>
</destination>
<log/>
<descr><![CDATA[NAT Torrent]]></descr>
<associated-rule-id>nat_52d81d2dc904f5.77023355</associated-rule-id>
<created>
<time>1389894957</time>
<username>NAT Port Forward</username>
</created>
<updated>
<time>1389899075</time>
<username>admin#192.168.0.20</username>
</updated>
</rule>
</filter>
</pfsense>
In the XML above, we have the two parts which comprise a port forward rule for pfsense. The part enclosed in the <nat> section is the port forward. The section in the <rule> is an interface specific incoming firewall rule. Both have to be modified for the new port forward setting to be effective.
I was thinking to use xmlstarlet to modify the config file, using the <descr> as my key for identifying which sections to change.
I'm aware that you can have data like:
<username><![CDATA[name]]></username>
<password><![CDATA[password]]></password>
<dbname><![CDATA[name]]></dbname>
and modify it with:
xml ed -P -O -L \
-u '//username/text()' -v 'something' \
-u '//password/text()' -v 'somethingelse' \
-u '//dbname/text()' -v 'somethingdifferent' file.xml
and also that you can have something like:
<objects>
<object>
<name>Foo</name>
<constant1>10</constant1>
<constant2>20</constant2>
</object>
<object>
<name>Bar</name>
<constant1>15</constant1>
<constant2>40</constant2>
</object>
<objects>
and update attributes with:
xmlstarlet ed -u '//object[name="Foo"]/const1' -v 18 sample.xml
However, I'm struggling to merge the two, so that I have a single statement which matches <descr>="Torrent" and then updates the relevant <port> and <local-port> attributes.
Any help with a suitable xmlstarlet command would be much appreciated.
xmlstarlet ed \
-u '//rule[descr="Torrent"]/destination/port' -v 1111 \
-u '//rule[descr="Torrent"]/local-port' -v 2222 \
sample.xml
For my Symfony application I build a deployment script with ANT. In the installation manual you can see that you need to set permissions to the app/cache/ and app/logs directories. I therefore use the following approach on command line:
sudo chmod +a "_www allow delete,write,append,file_inherit,directory_inherit" app/cache app/logs
The approach with setfacl does not work for me (Mac OS) and the third option with umask is no option for me.
So I was thinking about how to do this with ANT and came up with that:
<chmod perm="+a '_www allow delete,write,append,file_inherit,directory_inherit'">
<fileset dir="app/cache" />
<fileset dir="app/logs" />
</chmod>
The problem is it does nothing. I don't even get an error message nor do the permissions change. Any suggestions for that? I can't use a workaround like perm="0775" or perm="0777" because then I can't delete it anymore with ANT (only with sudo ant, but that is not what I want).
Any suggestions how to handle this?
Ok, got it:
<exec executable="chmod" dir="webroot" failonerror="true">
<arg line="+a '_www allow delete,write,append,file_inherit,directory_inherit' app/cache app/logs" />
</exec>
My system (Debian Wheezy) doesn't seem to support the +a attribute, but this has worked for me (second option in the docs):
<exec executable="/bin/sh" dir="${basedir}/app" failonerror="true">
<arg value="-c"/>
<arg value="setfacl -dR -m u:www-data:rwX -m u:`whoami`:rwX cache logs"/>
</exec>
Using sh -c seems to be necessary as I couldn't use the backticks for whoami without it.
You'll probably need to change dir="${basedir}/app" to your own app path.