I want to use the xargs command to read the standard output from my date command. The following pipe works, creating a directory "2019-12-03" in the current directory.
date "+%Y-%m-%d" -r ../IDNumber/IDNumber.txt | xargs mkdir
What I would like it to do, however, is use the standard output from the data command to make a directory in a remote location, username#archivalstorage.university.edu:/remote/folder/path/, resulting in username#archivalstorage.university.edu:/remote/folder/path/2019-12-03.
Running the following command:
date "+%Y-%m-%d" -r ../IDNumber/IDNumber.txt | xargs mkdir -p username#archivalstorage.university.edu:/remote/folder/path/
This command does not give any kind of error, but no folder is actually created in the remote location I have specified.
mkdir doesn't support the remote directory. Use ssh instead, with command substitute:
ssh user#host 'cd /path/to/dir && mkdir `date +%Y-%m-%d`'
or with xargs:
date +%Y-%m-%d | xargs -I _ ssh user#host 'cd /tmp && mkdir _'
Related
When using rsync, it is possible to create the target directory on the server using the --rsync-path trick as follows:
rsync -av -e "ssh" --rsync-path "mkdir -p /home/user/new/new && rsync" ./file.txt user#10.0.2.60:/home/user/new/new
This however does not seem to work when using an ssh tunnel. The following command just hangs:
rsync -av -e "ssh -A user#10.0.2.61 ssh" --rsync-path "mkdir -p /home/user/new/new && rsync" ./file.txt user#10.0.2.60:/home/user
I have verified the last command works if I remove the --rsync-path argument and create the directory manually on the target device. But how to make rsync create the missing directory when using ssh tunneling?
Managed to solve it. The command inside --rsync-path must be wrapped in another level of quotes:
rsync -av -e "ssh -A user#10.0.2.61 ssh" --rsync-path "'mkdir -p /home/user/new/new && rsync'" ./file.txt user#10.0.2.60:/home/user
For the https://github.com/ellakcy/piwik-with-wordpress I am making a restore bash script in order to restore the backup generated from the https://github.com/ellakcy/piwik-with-wordpress/blob/master/scripts/pre-backup script
The main idea is to set a path with a tarball containing the backup and recreating the folders that volumes are mounted.
The script is the following:
#!/bin/bash
# Printing functions
black='\E[30;40m'
red='\E[31;40m'
green='\E[32;40m'
yellow='\E[33;40m'
blue='\E[34;40m'
magenta='\E[35;40m'
cyan='\E[36;40m'
white='\E[37;40m'
#Echo a string with color
cecho () # Color-echo.
# Argument $1 = message
# Argument $2 = color
{
local default_msg="No message passed."
# Doesn't really need to be a local variable.
message=${1:-$default_msg} # Defaults to default message.
color=${2:-$black} # Defaults to black, if not specified.
echo -e "$color"
echo "$message"
tput sgr0 # Reset to normal.
return
}
#Echo a string as error with color
cecho_err () # Color-echo.
# Argument $1 = message
# Argument $2 = color
{
local default_msg="No message passed."
# Doesn't really need to be a local variable.
message=${1:-$default_msg} # Defaults to default message.
color=${2:-$red} # Defaults to black, if not specified.
echo >&2 -e "$color"
echo >&2 "$message"
tput sgr0 # Reset to normal.
return
}
backup_file=${1}
cecho "Creating the correct folders" $cyan
cecho "Deleting data folder in order to recreate it" $red
sudo rm -rf ./data
mkdir ./data/
sudo chown root:root ./data/
sudo chmod 755 ./data/
if [ ! -f restore ]; then
mkdir ./restore/
fi
tar -xf ${backup_file} -C ./restore/
cecho "Restoring backup data for wordpress" $cyan
sudo mkdir ./data/wordpress
sudo chown root:root ./data/wordpress
sudo chmod 755 ./data/wordpress
sudo mv ./restore/wordpress/data/www ./data/wordpress/
sudo chown www-data:www-data ./data/wordpress/www
cecho "Restoring environment" $cyan
wordpress_env=$(tr '\n' ' ' <./restore/wordpress/env.txt)
echo ${wordpress_env}
cecho "Restoring database" $cyan
sudo mkdir ./data/wordpress/db
echo "sudo env ${wordpress_env} docker run --volume \"./data/wordpress/db\":/var/lib/mysql --volume ./restore/wordpress/db:/docker-entrypoint-initdb.d -e MYSQL_ROOT_PASSWORD=\$WORDPRESS_MYSQL_ROOT_PASSWORD -e MYSQL_DATABASE=\"wordpress\" -e MYSQL_USER=\$WORDPRESS_MYSQL_USER -e MYSQL_PASSWORD=\$WORDPRESS_MYSQL_PASSWORD mariadb" > ./restore_db.sh
chmod +x ./restore_db.sh
./restore_db.sh
# rm -rf ./restore_db.sh
rm -rf ./restore
And I get this error when I try to restore the database:
docker: Error response from daemon: create ./data/wordpress/db: "./data/wordpress/db" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed.
See 'docker run --help'.
As you can see it generates a temporary scripts (that later will be deleted) one example of generated script is:
sudo env WORDPRESS_MYSQL_ROOT_PASSWORD=passwd WORDPRESS_MYSQL_USER=wordpress WORDPRESS_MYSQL_PASSWORD=wordpress WORDPRESS_ADMIN_USER=admin WORDPRESS_ADMIN_PASSWORD=admin WORDPRESS_URL=http://0.0.0.0:8080 docker run --volume "./data/wordpress/db":/var/lib/mysql --volume ./restore/wordpress/db:/docker-entrypoint-initdb.d -e MYSQL_ROOT_PASSWORD=$WORDPRESS_MYSQL_ROOT_PASSWORD -e MYSQL_DATABASE="wordpress" -e MYSQL_USER=$WORDPRESS_MYSQL_USER -e MYSQL_PASSWORD=$WORDPRESS_MYSQL_PASSWORD mariadb
What is the best option in order to generate the correct volume data in ./data/wordpress/db that mounts on a container's /var/lib/mysql?
When we specify --volume <host_dir>:<container_dir>, host_dir must be an absolute path. If it is not an absolute path, then it considered to be the volume's name. Hence the message invalid characters for a local volume name. Try providing absolute path for the host directory.
I have the following two directories:
~/A
drawable/
imageb.png
new/`
newimage.png
~/B
drawable/
imagec.png
When I use the cp -r ~/A/* ~/B command newimage.png with its new/ folder is copied across to ~/B however imageb.png is not copied into ~/B/drawable.
Could you explain why this is the case and how I can get around this?
Use tar instead of cp:
(cd A ; tar cf - *) | (cd B ; tar xf -)
or more compactly (if you're using GNU tar):
tar cC A -f - . | tar xC B -f -
If you are on linux you can use the -r option.
eg: cp -r ~/A/. ~/B/
If you are on BSD you could use the -R option.
eg: cp -R ~/A/. ~/B/
For more information on exactly what option you should pass, refer man cp
Also note that, if you do not have permissions to the file you it would prevent copying files.
I have a makefile (below) for a project where I've been given a folder of "Raw Data" - a set of files from a colleague, and I've made an R script that does an analysis on some of those files. What I want to do with a the makefile then is assign the directory to a variable RAWDIR, and then use that variable in specifying the make dependencies of the R script, and as a command line argument for the script. Usually in the shell, directories with spaces are expanded when using double quotes and curly braces, but I do not know if this is also correct for make files, as with the following makefile I get the message make: *** No rule to make target""../Raw', needed by pulls'. Stop. So I do not think my file path assigned to RAWDIR is being expanded properly.
Thanks.
RAWDIR="../Raw Data/Fc Project Raw Data"
.PHONY: dirs
pulls: dirs "${RAWDIR}/pm_fc_dnds_cleandata.csv" "${RAWDIR}/fc1_seqs.fasta" "${RAWDIR}/fc2_seqs.fasta" "${RAWDIR}/pm1_seqs.fasta" "${RAWDIR}/pm2_seqs.fasta"
Rscript Allele_Pulling.R "${RAWDIR}/" "${RAWDIR}/pm_fc_dnds_cleandata.csv"
dirs:
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/PM
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/Both
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC1PM1
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC1PM2
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC2PM1
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC2PM2
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC
mkdir -p -v Pulled_Allelic_Pairs/Aligned/PM
mkdir -p -v Pulled_Allelic_Pairs/Aligned/Both
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC1PM1
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC1PM2
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC2PM1
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC2PM2
In general spaces in pathnames are not well supported by make. At least some functions in GNU make could handle spaces that are escaped by \.
The following should work in your use case:
RAWDIR="../Raw\ Data/Fc\ Project\ Raw\ Data"
Suppose I want to rsync file foo.txt on my local machine to file /home/me/somedirectory/bar.txt on a remote computer, and that somedirectory/ doesn't yet exist. How do I do this?
I tried rsync -e ssh -z foo.txt remotemachine:/home/me/somedirectory/bar.txt, but I get a rsync: push_dir#3 "/home/me/somedirectory" failed: No such file or directory (2) error.
(Copying the file without renaming it works, though. That is, this runs fine: rsync -e ssh -z foo.txt remotemachine:/home/me/somedirectory/`)
Just put a trailing slash on your target dir. Something like this:
rsync foo.txt remotemachine:somedirectory/
Assuming that "/home/me" is your home dir on the remote machine, there is no need to specify it in the command line. Also, you don't need to clutter up your rsync with the -e unless you just like to do that.
You can do this process successfully in 2 stepes:-
1] rsync -e ssh -z foo.txt remotemachine:/home/me/somedirectory/
this will copy the foo.txt and create directory somedirectory on destination.
then
2] rsync -e ssh -z --delete-after foo.txt remotemachine:/home/me/somedirectory/bar.txt
and here you can delete foo.txt on destination by using --delete-after option.
you can see it's usage from man pages. This option must be used with -r option
This serves your purpose.
or if second command doesn't work then use :-
rsync -e ssh -z foo.txt remotemachine:/home/me/somedirectory/bar.txt
and delete foo.txt manually.