In an Amazon EC2 terminal, I type: `sudo nano crontab -e' to bring up the editor. I have the following (empty line at the end included):
#reboot echo "Running RMV scrape & R Shiny via: nano crontab -e"
#reboot nohup python /home/ec2-user/RMV/RMV_scrape.py &
#reboot nohup shiny-server &
#reboot service start httpd
#hourly cp -f /home/ec2-user/RMV/wait_times.csv /var/shiny-server/www/wait_times.csv
Here, I'm trying to run (a) my program, (b) apache, (c) R Shiny server and (d) a script that runs hourly to copy a file.
For some reason, this fails to run. pgrep chron does show chron runs upon startup. It shouldn't be a permissions issue because I ran crontab using sudo. I had one relative pathname in my .py script but I changed it to an absolute pathname.
I've consulted:
https://askubuntu.com/questions/23009/reasons-why-crontab-does-not-work
http://www.unix.com/answers-to-frequently-asked-questions/13527-cron-crontab.html
Any ideas why this may not be working?
I think your problems is with the command you used to edit the crontab sudo nano crontab -e does not edit the crontab you made a file named crontab in whatever directory you were working in, but crontab files are in /var and are not intended to be edited directly. For any given user crontab -e will edit the crontab using the editor specified in the environment variable EDITOR. So to edit root's crontab the command is sudo crontab -e.
That said adding entries to root's crontab is probably not what you want. You probably want to use the system crontab for some thing like this. In almost all cases the system crontab is /etc/crontab which can be edited using sudo nano /etc/crontab. Note that for the system crontab you need to add the user of the command between the time and command sections. e.g.
#reboot root echo "Running RMV scrape & R Shiny via: nano crontab -e"
Also note that crontab uses a very minimal PATH environment variable for security reasons. If a command you issue is not on the path it will not execute. Remember to either add the paths you need to the crontab PATH (specified in the particular crontab file) or use the full path to a given executable from the (filesystem) root directory.
Related
I'm currently struggling with running a .sh script I'm trying to trigger from Jenkins.
Within the Jenkins "execute shell" section, I'm connecting to a remote server (The Jenkins agent does not have right OS to build what I need.), using:
cp -r . /to/shared/drive/to/have/access/on/remote
ssh -t -t username#servername << EOF
cd /to/shared/drive/to/have/access/on/remote
source build.sh dev
exit
EOF
Inside build.sh, I'm exporting R_LIBS to build a package for different R versions.
...
for path in "${!rVersionPaths[#]}"; do
export R_LIBS="${path}"
Rscript -e 'install.packages(c("someDependency", "someOtherDependency"), repos="http://cran.r-project.org");'
...
Setting R_LIBS should functions here like setting lib within install.packages(...). For some reason the R_LIBS export doesn't get picked up. Also setting other env variables like http_proxy are ignored. This causes any requests outside the network to fail.
Is there any particular way of achieving this?
Maybe pass those variables with env, like
env R_LIBS="${path}" Rscript -e 'install.packages(c("someDependency", .....
Well i'm not able to comment on the question, so posting it as answer.
I had similar problem when calling remote shell script from Jenkins, the problem was somehow bash_profile variables were not loaded when called the script from Jenkins but locally it worked. Loading the bash profile in ssh connection solved it for me.
Add source to bash_profile in build.sh
. ~/.bash_profile OR source ~/.bash_profile
Or
Reload bash_profile in ssh connection
`ssh -t -t username#servername << EOF
. ~/.bash_profile
your commands here
exit
EOF
You can set that variable in the same command line like this:
R_LIBS="${path}" Rscript -e \
'install.packages(c("someDependency", "someOtherDependency"), repos="http://cran.r-project.org");'
It's possible to append more variables in this way. Note that this will set those environment variables only for the command being called after them (and its children processes as well).
You said that "R_LIBS export doesn't get picked up". Question Is the value UNSET? Or is it set to some other value & you are trying to override it?
It is possible that SSH may be invoking "/bin/sh -c". Based on the second answer to: Why does 'cd' command not work via SSH?, you can simplify the SSH command and explicitly invoke the build.sh script in Bash:
cp -r . /to/shared/drive/to/have/access/on/remote
ssh -t -t username#servername "cd /to/shared/drive/to/have/access/on/remote && bash -f build.sh dev"
This makes the SSH invocation more similar to invoking the command within a remote interactive shell. (You can avoid sourcing scripts and exporting variables.)
You don't need to export R_LIBSor env R_LIBS when it is possible to prefix any command with local environment variable overrides (agrees with Luis' answer):
...
for path in "${!rVersionPaths[#]}"; do
R_LIBS="${path}" Rscript -e 'install.packages(c("someDependency", "someOtherDependency"), repos="http://cran.r-project.org");'
...
The Rscript may be doing a lot with env vars. You can verify that you are setting the R_LIBS env var by replacing Rscript with the env command and observe the output:
...
for path in "${!rVersionPaths[#]}"; do
R_LIBS="${path}" env
...
According to this manual "Initialization at Start of an R Session", Rscript looks in several places to load "site and user files":
$R_PROFILE
$R_HOME/etc/Renviron
$R_HOME/etc/Renviron.site
$R_ENVIRON_USER
$R_PROFILE_USER
./.Rprofile
$HOME/.Rprofile
./.RData
The "Examples" section of that manual shows this:
## Not run:
## Example ~/.Renviron on Unix
R_LIBS=~/R/library
PAGER=/usr/local/bin/less
If you add the --vanilla command-line option to ignore all of these files, then you may get different results and know something in the site/init/environ files is affecting your R_LIBS! I cannot run this system myself. Hopefully we have given you some areas to investigate.
You probably don't want to source build.sh, just invoke it directly (i.e. remove the source command).
By source-ing the file your script is executed in the SSH shell (likely sh) rather than by bash, which it sounds like is what you intended.
Compiling R from the R-devel svn branch, I do
export CC='clang'
export CXX='clang++'
sudo ./configure
but the configuration script still tries to use gcc as the compiler. Why?
Because sudo reads environment variables of the root user, but export saves shell variables to your user environment. (within that terminal session only)
To fix this, you need to configure with sudo -E ./configure,
which reads environment variables from your user account (= login name) when executing ./configure with heightened privileges. Also have a look at the sudo -H flag (within man sudo).
Or you can first sudo su into the root account and export CC='clang' from within that root shell.
(the root shell prompt might begin with a # rather than a $, and be missing other config niceties—eg colourisation—from /home/user/.bashrc)
I have a shell script where I am calling the hana.scr script from within the main script. The hana.scr contains the below code.
chmod 777 /data/auto/SLT.out; rm -rf /data/auto/SLT.out; hdbsql -n plhesappr61 -i 00 -u USR -p $#^F#$GGG -o /data/auto/SLT.out "Select sum("ERPACC_RPPCLNT200"."VABD"."NETWR") FROM "ACC_CLNT"."VFKH" inner join "ACC_CLNT"."VNRO" on ("ACC_CLNT"."VNRO"."VBELN"="ACC_CLNT"."VFKH"."VBELN") where FKART in ('ZFP1','ZFP3') and FKDAT = (select ADD_DAYS (TO_DATE (current_date, 'YYYY-MM-DD'), -1) "add_days" from dummy) group by FKDAT";
When I run the main script manually, it calls this script fine and the SLT.out file is also generated.
But when I schedule it in cron, the main script executes just fine, except for this hana.scr which does not seem to execute because it does not does not even remove the old file as per the second command rm in the hana.scr.
The cron is the same user as the one I run the script manually with.
I read that if the cron does not get the same environment to run, these issues happen. I tried to import the UNIX profile of the user before executing as the hana.scr as well, but was not successful.
Below is the cron command which runs the main script which calls the hana.scr from within: Used absolute paths..
37 0,2,3,4,5,6 * * * /data/esb/auto/./main.sh R > /data/esb/auto/main.log
The hana.scr is executed in the below manner:
./hana.scr;
check6=$? ;
if [ $check6 = "1" ]
then
echo "***********HANA counts were not generated**********"
fi
After /data/esb/auto/./main.sh your current directory is not changed to /data/esb/auto/. I think you started main.sh from the commandline while your $PWD was the same as where hana.scr was.
Test it from the commandline with
cd /
/data/esb/auto/main.sh
How to fix?
The worst solution is changing the crontab line into
37 0,2,3,4,5,6 * * * cd /data/esb/auto; /data/esb/auto/main.sh R > /data/esb/auto/main.log
That is a workaround for the crontab but main.sh still fails when started from a different directory.
Slightly better is using the complete path in main.sh when you call hana.scr
myscriptdir=/data/esb/auto
..
${myscriptdir}/hana.scr
When you change the folders you need to edit the files and repair the settings.
You can try to use some config file with settings or let main.sh figure out what in which directory it is:
Getting the source directory of a Bash script from within
I found many similar questions on internet but no one could resolve my problem..I am working on Solaris 5.10 machine.. Here Inside a shell script a customized command is being run like below.
palf -f ${basePath}/palf_file.DAT -e ${basePath}/LOG/palf_file.log
This command only runs while logged in as "palf" user. Now this script & subsequently this command is running perfectly from command prompt. But crontab is not able run this command.
I tried few things.. I changed the entry of my crontab file like below which could not even run the script.
40 15 * * * palf bash /opt/bin/scripts/script.sh
Then I tried to edit a cronfile as "palf" user by using the below command but it gave me "invalid options" error.
crontab -u palf -e
I also tried
crontab -e palf
It opened a crontab file but it was same as the root's crontab file not the user's specific
Nothing worked for me. Could anyone please help here? Thanks.
palf -f ${basePath}/palf_file.DAT -e ${basePath}/LOG/palf_file.log
if [[ $? -ne 0 ]]; then
logger "palf command failed. Please check..." 1
else
logger "palf command successfully executed..." 0
fi
This is how I am checking the status of palf command.. and it prints "palf command failed. Please check..." using logger function every time it runs using cronjob.
I have a script that looks like this
#!/bin/bash
function something() {
echo "hello world!!"
}
something | tee logfile
I have set the execute permission on this file and when I try running the file like this
$./script.sh
it runs perfectly fine, but when I run it on the command line like this
$sh script.sh
It throws up an error. Why does this happen and what are the ways in which I can fix this.
Running it as ./script.sh will make the kernel read the first line (the shebang), and then invoke bash to interpret the script. Running it as sh script.sh uses whatever shell your system defaults sh to (on Ubuntu this is Dash, which is sh-compatible, but doesn't support some of the extra features of Bash).
You can fix it by invoking it as bash script.sh, or if it's your machine you can change /bin/sh to be bash and not whatever it is currently (usually just by symlinking it - rm /bin/sh && ln -s /bin/bash /bin/sh). Or you can just use ./script.sh instead if that's already working ;)
If your shell is indeed dash and you want to modify the script to be compatible, https://wiki.ubuntu.com/DashAsBinSh has a helpful guide to the differences. In your sample it looks like you'd just have to remove the function keyword.
if your script is at your present working directory and you issue ./script.sh, the kernel will read the shebang (first line) and execute the shell interpreter that is defined. you can also call your script.sh by specifying the path of the interpreter eg
/bin/bash myscript.sh
/bin/sh myscript.sh
/bin/ksh myscript.sh etc
By the way, you can also put your shebang like this (if you don't want to specify full path)
#!/usr/bin/env sh
sh script.sh forces the script to be executed within the sh - shell.
while simply starting it from command line uses the shell-environemnt you're in.
Please post the error message for further answers.
Random though on what the error may be:
path specified in first line /bin/bash is wrong -- maybe bash is not installed?