I want a script that prints the memory space taken by the directory in every 5 minutes. I do not want to use a clone job. I wrote a bash script-
#! /bin/bash
echo "the script starts now"
sleep 60;
du -c > time&
But this does not work the way as expected.
I want to run this task in background and dump the space taken along with time to a log file.
You can achieve it by following code
du -c >> /var/log/messages &
sleep 60s
Try to use this command.
du -k path
Also, try to explore more options on du. Have you tried something on du?
Also refer to below question.
How do run a Unix command at a given time interval?
Try below commands.
watch -n 300 du -s path
OR
while true; do
du -s path
sleep 300
done
Related
I have a long list of remote hosts and I want to run a shell command on all of them. The command takes a very long time, so I want to run the command inside screen on the remote machine, disconnecting immediately from each, and I want the terminal output on the remote to be preserved after the command exits. There is a "tag" that should be supplied to each command as an argument. I tried to do this with parallel, something like this:
$ cat servers.txt
user1#server1.example.com/tag1
user2#server2.example.com/tag2
# ...
$ cat run.sh
grep -v '^#' servers.txt |
parallel ssh -tt '{//}' \
'tag={/}; exec screen slow_command --option1 --option2 $tag other args'
This doesn't work: all of the remote processes are launched, but they are not detached (so the ssh sessions remain live and I don't get my local shell back), and once each command finishes, its screen exits immediately and the output is lost.
How do I fix this script? Note: if this is easier to do with tmux and/or some other marshalling program besides parallel, I'm happy to hear answers that explain how to do it that way.
Something like this:
grep -v '^#' servers.txt |
parallel -q --colsep / ssh {1} "screen -d -m bash -c 'echo do stuff \"{2}\";sleep 1000000'"
The final sleep makes sure the screen does not die. You will have 1000000 seconds to attach to it and kill it.
There is an awful lot of quoting there - especially if do stuff is complex.
It may be easier to make a function that computes tag on the remote machine. You need GNU Parallel 20200522 for this:
env_parallel --session
f() {
sshlogin="$1"
# TODO given $sshlogin compute $tag (e.g. a table lookup)
do_stuff() {
echo "do stuff $tag"
sleep 1000000
}
export -f do_stuff
screen -d -m bash -c do_stuff "$#"
}
env_parallel --nonall --slf servers_without_tag f '$PARALLEL_SSHLOGIN'
env_parallel --endsession
I use crontask to regularly run Rscript. Unfortunately, I need to do this on a small instance of aws and the process may hang, building more and more processes on top of each other until the whole system is lagging.
I would like to write a crontask to kill all R processes lasting longer than one minute. I found another answer on Stack Overflow that I've adapted that I think would solve the problem. I came up with;
if [[ "$(uname)" = "Linux" ]];then killall --older-than 1m "/usr/lib/R/bin/exec/R --slave --no-restore --file=/home/ubuntu/script.R";fi
I copied the task directly from htop, but it does not work as I expect. I get the No such file or directory error but I've checked it a few times.
I need to kill all R processes that have lasted longer than a minute. How can I do this?
You may want to avoid killing processes from another user and try SIGKILL (kill -9) after SIGTERM (kill -15). Here is a script you could execute every minute with a CRON job:
#!/bin/bash
PROCESS="R"
MAXTIME=`date -d '00:01:00' +'%s'`
function killpids()
{
PIDS=`pgrep -u "${USER}" -x "${PROCESS}"`
# Loop over all matching PIDs
for pid in ${PIDS}; do
# Retrieve duration of the process
TIME=`ps -o time:1= -p "${pid}" |
egrep -o "[0-9]{0,2}:?[0-9]{0,2}:[0-9]{2}$"`
# Convert TIME to timestamp
TTIME=`date -d "${TIME}" +'%s'`
# Check if the process should be killed
if [ "${TTIME}" -gt "${MAXTIME}" ]; then
kill ${1} "${pid}"
fi
done
}
# Leave a chance to kill processes properly (SIGTERM)
killpids "-15"
sleep 5
# Now kill remaining processes (SIGKILL)
killpids "-9"
Why imply an additional process every minute with cron?
Would it not be easier to start R with timeout from coreutils, the processes will then be killed automatically after the time you chose.
timeout [option] duration command [arg]…
I think the best option is to do this with R itself. I am no expert, but it seems the future package will allow executing a function in a separate thread. You could run the actual task in a separate thread, and in the main thread sleep for 60 seconds and then stop().
Previous Update
user1747036's answer which recommends timeout is a better alternative.
My original answer
This question is more appropriate for superuser, but here are a few things wrong with
if [[ "$(uname)" = "Linux" ]];then
killall --older-than 1m \
"/usr/lib/R/bin/exec/R --slave --no-restore --file=/home/ubuntu/script.R";
fi
The name argument is either the name of image or path to it. You have included parameters to it as well
If -s signal is not specified killall sends SIGTERM which your process may ignore. Are you able to kill a long running script with this on the command line? You may need SIGKILL / -9
More at http://linux.die.net/man/1/killall
In OpenBSD, there's no logrotate in ports, and newsyslog seems to have limited features as far as monthly rotation of a huge number of log files is concerned.
I have a lot of domains, a huge number of nginx log-files names like /var/www/logs/*/*.{access,error}.log.
I'm thinking a small shell script and cronjob. What would be the easiest way to rotate them all monthly, and append the prior month to the filename?
I think the following crontab should work:
0 0 1 * * /etc/nginx/logrotate.monthly.sh
Where /etc/nginx/logrotate.monthly.sh should have the following content:
find /var/www/logs/ -name "*log" -exec \
mv -i {} {}.`sh -c 'date -r $(expr $(date +%s) - 1209600) +%Y-%m'` \; ; \
kill -USR1 `cat /var/run/nginx.pid`
The -i/--interactive ("prompt before overwrite") option to mv is important to ensure that files don't get overwritten. We get the date for the filename by moving today's date two weeks back (as per « tcsh: print date 2 weeks ago in shell »).
As documented, "NGINX will re-open its logs in response to the USR1 signal."
please check also this misc# thread.
(also keep in mind the caveat documented in the FAQ about privseped apache and the need for a small time window upon the move.)
I want to run a Unix command (e.g. ls) at 5 minute intervals through a script.
Explanation:
I have a Unix script. In that script I have a command called "ls".
I want that "ls" command to run every 5 minutes from that script.
Use watch. The -n flag specifies interval in seconds, so
watch -n 300 ls
while true; do
ls
sleep 300
done
?
Put your script into the Crontab via
crontab -e
More information about Cron you can find at wikipedia
You could use crontab for example. See http://en.wikipedia.org/wiki/Cron
for example i you want to run your script every five minutes via crontab it should look something like this:
#m h dom mon dow user command
*/5 * * * * root /path/to/script
crontab
http://en.wikipedia.org/wiki/Cron
or
svc
http://cr.yp.to/daemontools.html
I have a script that looks like this
#!/bin/bash
function something() {
echo "hello world!!"
}
something | tee logfile
I have set the execute permission on this file and when I try running the file like this
$./script.sh
it runs perfectly fine, but when I run it on the command line like this
$sh script.sh
It throws up an error. Why does this happen and what are the ways in which I can fix this.
Running it as ./script.sh will make the kernel read the first line (the shebang), and then invoke bash to interpret the script. Running it as sh script.sh uses whatever shell your system defaults sh to (on Ubuntu this is Dash, which is sh-compatible, but doesn't support some of the extra features of Bash).
You can fix it by invoking it as bash script.sh, or if it's your machine you can change /bin/sh to be bash and not whatever it is currently (usually just by symlinking it - rm /bin/sh && ln -s /bin/bash /bin/sh). Or you can just use ./script.sh instead if that's already working ;)
If your shell is indeed dash and you want to modify the script to be compatible, https://wiki.ubuntu.com/DashAsBinSh has a helpful guide to the differences. In your sample it looks like you'd just have to remove the function keyword.
if your script is at your present working directory and you issue ./script.sh, the kernel will read the shebang (first line) and execute the shell interpreter that is defined. you can also call your script.sh by specifying the path of the interpreter eg
/bin/bash myscript.sh
/bin/sh myscript.sh
/bin/ksh myscript.sh etc
By the way, you can also put your shebang like this (if you don't want to specify full path)
#!/usr/bin/env sh
sh script.sh forces the script to be executed within the sh - shell.
while simply starting it from command line uses the shell-environemnt you're in.
Please post the error message for further answers.
Random though on what the error may be:
path specified in first line /bin/bash is wrong -- maybe bash is not installed?