I am using airflow and running a dag
In have following in airflow.cfg
[core]
# The folder where your airflow pipelines live, most likely a
# subfolder in a code repository. This path must be absolute.
dags_folder = /usr/local/airflow/dags
# The folder where airflow should store its log files
# This path must be absolute
base_log_folder = /usr/local/airflow/logs
I have a long running task in airflow.
Its very difficuly to use web interface to check the logs of such size.
Instead i want to check on the airflow server directly
But i dont see a log file is created till the task fails or completely succeeded
In between the task i cant see any 1.log file created in the local server at the path mentioned in cfg
So on the server if i have access, how to check the logs of the airflow task live
Related
I'm having a few problems with airflow. When I installed airflow and set the airflow home directory to be
my_home/Workspace/airflow_home
But when I start the webserver a new airflow directory is created
my_home/airflow
I thought maybe something in the airflow.cfg file needs to be changed but I'm not really sure. Has anyone had this problem before?
Try doing echo $AIRFLOW_HOME and see if it the correct path you set
you need to set AIRFLOW_HOME to the directory where you save airflow config file.
if the full path of airflow.cfg file is /home/test/bigdata/airflow/airflow.cfg
just run
export AIRFLOW_HOME=/home/test/bigdata/airflow
if AIRFLOW_HOME is not set, it will use ~/airflow as default.
you could also write a shell script to start airflow webserver
it might contain lines below
source ~/.virtualenvs/airflow/bin/activate # if your airflow is installed with virtualenv, this is not necessary
export AIRFLOW_HOME=/home/test/bigdata/airflow # path should be changed according to your environment
airflow webserver -D # start airflow webserver as daemon
I wanted to change the backup to a different disk. I mounted the disk to /mnt2 on centos and when I navigate to Admin > Backups > Backup Daily > Edit backup-daily Backup, I see an option Server Path For Backup. I tried the following two things.
I entered the mount directory /mnt2 and hit run now. The background job fails with the following error in logs.
An error occurred while performing a backup: Backup directory provided
in configuration: '/mnt2' cannot be created or is not a directory.
I also tried creating a tmp2 directory on local drive and entered /tmp2 and hit run now. The background job fails with the same error as above.
Note 1:
I restarted the docker container just to see if it's not picking up file system changes in real time. That did not work.
Note 2:
There is a browse button next to Sever Path for Backup and I dont see /mnt2 or /tmp2 directories I created. I couldnot find anything useful in the documentation either.
How do I change the backup directory for artifactory?
The setup is artifactory with docker.
For artifactory docker instance, a volume needs to be specified so it maps to the local folder, say, /opt/artifactory/.
In my case, /var/opt/jfrog/artifactory(docker) is mapped to /opt/artifactory(local)
I am supposed to create a folder here -- /opt/artifactory/backup_mount. Give read and write access for 1039 user and group. It shows up in artifactory UI as /var/opt/jfrog/artifactory/backup_mount.
Note:
If you create a directory, it shows up without any docker restart.
If you create a mount, restart docker so the mount is recognized.
Environment : Hortonworks Sandbox HDP 2.2.4
Issue : Unable to run the hadoop commands present in the shell scripts as a root user. The oozie job is getting triggered as a root user, but when the hadoop fs or any mapreduce command is executed, then it runs as yarn user. As yarn, doesn’t have access to some of the file system , so the shell script is failing to execute. Let me know what changes I need to do , for making it run the hadoop commands as root user.
It is an expected behaviour to get Yarn in place whenever we are invoking shell actions in oozie. Yarn user only have the capabilities to run shell actions. One thing we can do is to give access permissions to Yarn on the file system.
This is more like a shell script question than an Oozie question. In theory, Oozie job runs as the user who submits the job. In a kerberos' env, the user is whoever signed in with keytab/password.
Once job is running on Hadoop cluster, in order to change the ownership of command, you can use "sudo" within your shell script. In your case, you may also want to make sure user "yarn" is allowed to sudo to the commands you want to execute.
Add below property to workflow:
HADOOP_USER_NAME=${wf:user()}
I have a Google Compute Engine VM instance with a Asterisk Server running on it. I get this message when I try to run sudo:
sudo: parse error in /etc/sudoers near line 21
sudo: no valid sudoers sources found, quitting
sudo: unable to initialize policy plugin
Is there a password for root so I can try to change it there? Any suggestions on this?
It looks like you have manually edited the /etc/sudoers file so while you would normally have sudo access, due to the parse error, you won't be able to do this directly.
Here's how to fix this situation.
1. Save the current boot disk
go to to the instance view in Developers Console
find your VM instance and click on its name; you should now be looking at a URL such as
https://console.cloud.google.com/project/[PROJECT]/compute/instancesDetail/zones/[ZONE]/instances/[VM-NAME]
stop the instance
detach the boot disk from the instance
2. Fix the /etc/sudoers on the boot disk
create a new VM instance with its own boot disk; you should have sudo access here
attach the disk saved above as a separate persistent disk
mount the disk you just attached
fix the /etc/sudoers file on the disk
unmount the second disk
detach the second disk from the VM
delete the new VM instance (let it delete its boot disk, you won't need it)
3. Restore the original VM instance
re-attach the boot disk to the original VM
restart the original VM with its original boot disk, with fixed config
How to avoid this in the future
Always use the command visudo rather just any text editor directly to edit the /etc/sudoers file which will validate the contents of the file prior to saving it.
I ran into this issue as well and had the same issue Nakilon was reporting when trying the gcloud workaround.
What we ended up doing was configure a startup script that removed the broken sudoers file.
So in your metadata put something like:
#/bin/sh
rm "/etc/sudoers.d/broken-config-file"
echo "ok" > /tmp/ok.log
https://cloud.google.com/compute/docs/startupscript
As you probably figured out this requires the /etc/sudoers file to be fixed. As nobody has root access to the instance, you will not be able to do this from inside the instance.
The best way to solve this is to edit the disk from another instance. The basic steps to do this are:
Take a snapshot of your disk as a backup (!)
Shutdown your instance, taking care not to delete the boot disk.
Start a new "debugger" instance from one of the stock GCE images.
Attach the old boot disk to the new instance.
In the debugger instance, mount the disk.
In the debugger instance, fix the sudoers file on the mounted disk.
In the debugger instance, unmount the disk
Shutdown the debugger instance.
Create a new instance with the same specs as your original instance using the fixed disk as the boot disk.
The new disk will then have the fixed sudoers file.
Since i bumped into this issue too, if you have another instance or any place where you can run with gcloud privileges, you can run:
gcloud compute --project "<project id>" ssh --zone "europe-west1-b" "<servername>"
I ran this on a server which had gcloud as root, so you login to the other box as root too! Then fix your issue. (if you don't have a box, just spin a micro up with the correct gcloud privileges) saves the hassle of disk stuff etc.
As mentioned in above comments, I am getting the same error like below in gcp VM.
sudo: parse error in /etc/sudoers near line 21
sudo: no valid sudoers sources found, quitting
sudo: unable to initialize policy plugin
To solve this
I have ssh to another vm and become root then I ran gcloud ssh command to our main vm (where you are getting the sudo error.)
gcloud compute --project "<project id>" ssh --zone "europe-west1-b "<servername>"
And BOOM!, now are login as root in the VM.
Now you can access/change the /etc/sudoers file accordingly.
I found this hack better than recreating vm/disks.
Hope this helps to someone!
It is possible to connect to a VM as root from your developers console Google Cloud Shell. Make sure the VM is running, start the shell and use this command:
gcloud compute ssh root#<instance-name> --zone <zone> [--project <project-id>]
where instance-name is found in the Compute Engine VM Instances screen. project-id is optional but required if you are connecting to an instance in a different project from the project where you started the shell.
You can then fix this and other issues that may prevent you from using sudo.
I got a Permission denied error when trying to ssh to the problem instance via gcloud. Using a startup script as mentioned above by #Jorick works. Instructions for it are here. You will have to stop and restart the VM instance for the startup script to get executed. I modified the script slightly:
rm -f /etc/sudoers.d/google_sudoers >& /tmp/startup.log
After the restart, launch an SSH session from the cloud console and check that you are able to view the file contents (with sudo more /etc/sudoers.d/google_sudoers for example). If that works your problem has been solved.
I've setup a class library in VS and run functions using command line arguments.
I've built the project and copied the files in the 'debug' folder over to the server.
Copied over the database and changed the connection string to match.
On the server, I've setup a windows task scheduler to run the function every 15 mins:
In the 'create a basic task wizard' dialogue: start a program:
'Program/script:' chose the .exe file in the folder, and removed .config from the end of it.
'Added Argument': DownloadPOS
and 'start in':chose the same folder the project was in.
Problem being when the scheduler runs I get this error and have no idea why:
please advise