Executed the command "sudo -u oozie ./bin/oozie-setup.sh sharelib create -fs hdfs://localhost:54310 -locallib oozie-sharelib-4.1.0.tar.gz"
Iam getting the error "Error: E0902: Exception occured: [User: oozie is not allowed to impersonate oozie]" when trying to execute sharelib command.
In core-site.xml in hadoop conf, change value as
<property>
<name>hadoop.proxyuser.oozie.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.oozie.groups</name>
<value>*</value>
</property>
And restart hadoop services and try again
Related
I am trying to run an ansible playbook from Execute Shell section of Jenkins,But I am getting below errors.I have already install Ansible plugin and configured in global tool configuration section.
+ ansible-playbook -i inventory installapache.yml -vvvv
/tmp/jenkins1596894985578146945.sh: line 4: ansible-playbook: command not found
Build step 'Execute shell' marked build as failure
Finished: FAILURE
I am using below commands:
export ANSIBLE_FORCE_COLOR=true
cd /home/ec2-user
ansible-playbook -i hosts /home/ec2-user/installapache.yml -vvvv
I am trying to trigger a Jenkins job which is throwing below error.Jenkins is hosted on AWS ec2 instances
/tmp/jenkins7326195626020563202.sh: line 4: cd: /home/ec2-user: Permission denied
Build step 'Execute shell' marked build as failure
Finished: FAILURE
My Jenkins job has below shell command to execute:
export ANSIBLE_FORCE_COLOR=true
cd /home/ec2-user
ansible-playbook -i hosts /home/ec2-user/installapache.yml -vvvv
Please help here in what I need to fix for above error.
That issue happened when jenkins user doesnot have permission in /home/ec2-user.
So solution for this :
chmod -R 777 /home/ec2-user
OR add jenkins user to ec2-user group
OR grant jenkins user has a root permission.
How can we get status of Oozie jobs running daily? We have many jobs running in Oozie coordinator and currently we are monitoring through Hue/Oozie browser.
Is there any way we can get a single log file which contains coordinator name/workflow name with date and status? Can we write any program or script to achieve this?
You use the below command and put it into a script to run it daily/cron.
oozie jobs -oozie http://localhost:11000/oozie -filter status=RUNNING -len 2
oozie jobs -oozie http://localhost:11000/oozie -filter startCreatedTime=2016-06-28T00:00Z\;endcreatedtime=2016-06-28T10:00Z -len 2
Basically you are using oozie's jobs api and -filter command to get the information about workflow/coordinator/bundle. The -filter command supports couple of options to get the data based on status/startCreatedTime/name.
By default, it will bring the workflow record information, if you want to get the coordinator/bindle information. You can use the -jobtype parameter and value as coord/bundle.
Let me know if you need anything specifically. The oozie doc is little outdated for this feature.
Command to get status of all running oozie coordinators
oozie jobs -jobtype coordinator -filter status=RUNNING -len 1000 -oozie http://localhost:11000/oozie
Command to get status of all running oozie workflows
oozie jobs -filter status=RUNNING -len 1000 -oozie http://localhost:11000/oozie
Command to get status of all workflows for a specific coordinator ID
oozie job -info COORDINATOR_ID_HERE
Based on these queries you can write required scripts to get what you want.
Terms explanation:
oozie : Command to initiate oozie
job/jobs : API
len: No. of oozie workflows/coordinators to display
-oozie: Param to specify oozie url
-filter : Param to specify list of filters.
Complete documentation https://oozie.apache.org/docs/3.1.3-incubating/DG_CommandLineTool.html
Below command worked for me.
oozie jobs -oozie http://xx.xxx.xx.xx:11000/oozie -jobtype wf -len 300 | grep 2016-07-01 > OozieJobsStatus_20160701.txt
However we need to parse the file.
I want to know about failed oozie job execution.
Does it run from start or run from the failed point?
I have MR and Pig task to be executed.
If Pig job failed, does it start again from MR job execution?
How to re-run the failed oozie jobs?
It will start from the failed action.
oozie job --oozie <oozie_url> -rerun <job_id> -config <job.properties>
Add the following property in the job.properties if not present already. -config parameter is required only if you are updating the job.properties file.
oozie.wf.rerun.failnodes=true
If I run fuser -n tcp 80 on ubuntu 14.0.4 which is running in a docker image, I get:
Cannot stat file /proc/25/fd/0: Permission denied
Cannot stat file /proc/25/fd/1: Permission denied
Cannot stat file /proc/25/fd/2: Permission denied
Cannot stat file /proc/25/fd/3: Permission denied
Cannot stat file /proc/25/fd/4: Permission denied
Cannot stat file /proc/25/fd/5: Permission denied
etc.
80/tcp: 24
If I run whoami it returns root.
Why can I not run this command?
That should be because of AppArmor.
It is a Linux Security Module implementation of name-based access controls. AppArmor confines individual programs to a set of listed files and posix 1003.1e draft capabilities.
You can see various workarounds in issue 7276 or issue 6800.
docker run -itd --cap-add=SYS_PTRACE --security-opt=apparmor:unconfined mytomcat7image