I am trying to submit a job to a cluster that may take up to a few days. Usually, for a shorter job, I simply do qsub Arun1_scr and then wait for the job to finish while monitoring its status with qstat. the Arun_scr is a basic script. If I want to be able to exit the shell and maybe even turn off the computer while the job is being done on the cluster all I have to do is nohup qsub Arun1_scr?
Thank you!
If you submit your job using qsub Arun1_scr you can exit the shell and it will still continue to run on the cluster. So, you do not need to change anything.
If you use nohup command and you want to run it into background the syntax is nohup command-name & ( without & your job will not be run in background and will be stopped after you close the shell).
Related
How can I run a shell script and immediately background it, however keep the ability to inspect its output any time by tailing /tmp/output.txt.
It would be nice if I can foreground the process too later.
P.S.
It would be really cool if you can also show me how to "send" the backgrounded process in to a GNU screen that may or may not have been initialized.
To 'background' a process when you start it
Simply add an ampersand (&) after the command.
If the program writes to standard out, it will still write to your console / terminal.
To foreground the process
Simply use the fg command. You can see a list of jobs in the background with jobs.
For example:
sh -c 'sleep 3 && echo I just woke up' & jobs
To background a currently running process
If you have already started the process in the foreground, but you want to move it to the background, you can do the following:
Press Ctrl+z to put the current process to sleep and return to your shell. This process will be paused until you send it another signal.
Run the bg command to resume the process, but have it run in the background instead of the foreground.
Another way is using the nohup command with & at the end of the line.
Something like this
nohup whatevercommandyouwant whateverparameters &
This will run it in the background and send its output to a nohup.log file.
One easy to use approach that allows managing multiple processes and has a nice terminal UI is hapless utility.
Install with pip install hapless (or python3 -m pip install hapless) and just run
$ hap run my-command # e.g. hap run python my_long_running_script.py
$ hap status # check all the launched processes
$ hap logs 4 # output logs for you 4th background process
$ hap logs -f 2 # continuously stream logs for the 2nd process
See docs for more info.
We run an interactive perl script which takes a set of inputs(an input which is time till which the script should run) from the terminal and runs till the completion. The job runs in the foreground. I have made the job to run in the background by hitting CTRL-Z followed by bg %- command.
Once the current job runs in the background, if I run the same perl script with a different set of inputs and try to put it into the background the first job is getting terminated when I hit CTRL-Z as follows.
^Z[1] Terminated scriptname
[2]+ Stopped scriptname
Please point out if I am making any mistake
When you press ^Z the running script is stopped and after bg it will continue (in the background).
You get the feedback of the bg job when you do something on the commandline.
With [1] Terminated scriptname you see that the first script is finished. With [2]+ Stopped scriptname you see the second job is paused and waiting for a bg/fg command.
Using ^Z + bg is about the same as appending & to your command (for not interactive programs). You can try things with
sleep 1 &
# now wait few seconds
ps
# you will see the terminated script
sleep 6 &
sleep 3; ps
sleep 4; ps
# and now repeat these cases with ^Z / bg instead of &
try nohup
nohup nice <your_script followed by parameters(separated by space)> &
It will give you process_id. Hit Ctrl+C to exit the command and verify by ps -ef that it is still running. I remember running multiple instance of same script with different parameter like this.
Ok, my first question so sorry for any errors in the way I've asked.
I am trying to create a script jenkins.sh which should call/launch another process (start_rhino.sh) in the background (if started normally it will take over the console and will not free the command line unless it is stopped).
So far I have tried:
./start-rhino.sh & -> starts the process the same as ./start-rhino.sh
nohup ./start-rhino.sh & -> Requires a key press to continue
and none of them immediately released the command line in order for the script to progress. In desperation I also tried with double && and /& but with no success. I think the nohup worked the best but it required an 'enter' key press in order to continue to command line (I've tried them directly from command line not by running the script)
nohup ./start-rhino.sh < /dev/null 2>&1 > /dev/null &
I am trying to create a Jenkins job that restarts a program that runs all the time on one of our servers.
I specify the following as the command to run:
cd /usr/local/tool && ./tool stop && ./tool start
The script 'tool' contains a line like:
nohup java NameOfClass &
The output of that ends up in my build console instead of in nohup.out, so the job never terminates unless I terminate it manually, which terminates the program.
How can I cause nohup to behave the same way it does from a terminal?
If I understood the question correctly, Jenkins is killing all processes at the end of the build and you would like some process to be left running after the build has finished.
You should read https://wiki.jenkins-ci.org/display/JENKINS/ProcessTreeKiller
Essentially, Jenkins searches for processes with some secret value in BUILD_ID environment variable. Just override it for the processes you want to be left alone.
In the new Pipeline jobs, setting BUILD_ID no longer prevents Jenkins from killing your processes once the job finishes. Instead, you need to set JENKINS_NODE_COOKIE:
sh 'JENKINS_NODE_COOKIE=dontKillMe nohup java NameOfClass &'
See the wiki on ProcessTreeKiller and this comment in the Jenkins Jira for more information.
Try adding the & in the Jenkins build step and redirecting the output using > nohup.out.
I had a similar problem with runnning a shell script from jenkins as a background process. I fixed it by using the below command:
BUILD_ID=dontKillMe nohup ./start-fitnesse.sh &
In your case,
BUILD_ID=dontKillMe nohup java NameOfClass &
In UNIX, I have a utility, say 'Test_Ex', a binary file. How can I write a job or a shell script(as a cron job) running always in the background which keeps checking if 'Test_Ex' is still running every 5 seconds(and probably hide this job). If it is running, do nothing. If not, delete a directory at the specified path.
Try this script:
pgrep Test_Ex > /dev/null || rm -r dir
If you don't have pgrep, use
ps -e -ocomm | grep Test_Ex || ...
instead.
Utilities like upstart, originally part of the Ubuntu linux distribution I believe, are good for monitoring running tasks.
The best way to do this is to not do it. If you want to know if Test_Ex is still running, then start it from a script that looks something like:
#!/bin/sh
Test_Ex
logger "Test_Ex died"
rm /p/a/t/h
or
#!/bin/sh
while ! Test_ex
do
logger "Test_Ex terminated unsuccesfully, restarting in 5 seconds"
sleep 5
done
Querying ps regularly is a bad idea, and trying to monitor it from cron is a horrible, horrible idea. There seems to be some comfort in the idea that crond will always be running, but you can no more rely on that than you can rely on the wrapper script staying alive; either one can be killed at any time. Waking up every 10 seconds to query ps is just a waste of resources.