I have a makefile (below) for a project where I've been given a folder of "Raw Data" - a set of files from a colleague, and I've made an R script that does an analysis on some of those files. What I want to do with a the makefile then is assign the directory to a variable RAWDIR, and then use that variable in specifying the make dependencies of the R script, and as a command line argument for the script. Usually in the shell, directories with spaces are expanded when using double quotes and curly braces, but I do not know if this is also correct for make files, as with the following makefile I get the message make: *** No rule to make target""../Raw', needed by pulls'. Stop. So I do not think my file path assigned to RAWDIR is being expanded properly.
Thanks.
RAWDIR="../Raw Data/Fc Project Raw Data"
.PHONY: dirs
pulls: dirs "${RAWDIR}/pm_fc_dnds_cleandata.csv" "${RAWDIR}/fc1_seqs.fasta" "${RAWDIR}/fc2_seqs.fasta" "${RAWDIR}/pm1_seqs.fasta" "${RAWDIR}/pm2_seqs.fasta"
Rscript Allele_Pulling.R "${RAWDIR}/" "${RAWDIR}/pm_fc_dnds_cleandata.csv"
dirs:
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/PM
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/Both
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC1PM1
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC1PM2
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC2PM1
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC2PM2
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC
mkdir -p -v Pulled_Allelic_Pairs/Aligned/PM
mkdir -p -v Pulled_Allelic_Pairs/Aligned/Both
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC1PM1
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC1PM2
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC2PM1
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC2PM2
In general spaces in pathnames are not well supported by make. At least some functions in GNU make could handle spaces that are escaped by \.
The following should work in your use case:
RAWDIR="../Raw\ Data/Fc\ Project\ Raw\ Data"
Related
I want to create a Debian package from a C program without the use of a build tool such as autotools or CMake. My debian/rules file:
#!/usr/bin/make -f
%:
dh $#
override_dh_auto_clean:
rm -f program
override_dh_auto_build:
gcc program.c -o program
override_dh_auto_install:
cp program /usr/local/bin
Upon running dpkg-buildpackage, I get:
dh: error: Unknown sequence application (choose from: binary binary-arch binary-indep build build-arch build-indep clean install install-arch install-indep)
It seems the issue was related to the fact that I was creating the file in a shell script heredoc that was expanding the $#, e.g.
cat <<EOF > debian/rules.temp
#!/usr/bin/make -f
%:
dh $#
EOF
Which should be:
all_symbol='$#'
cat <<EOF > debian/rules.temp
#!/usr/bin/make -f
%:
dh $all_symbol
EOF
An unrelated issue is to the location of the override_dh_auto_install To manually create Debian file hierarchy it should be:
override_dh_auto_install:
mkdir -p debian/PACKAGENAME/usr/bin
cp program debian/PACKAGENAME/usr/bin
Or, to have this done automatically:
cat <<EOF > debian/PACKAGENAME.install
program usr/bin
EOF
When using rsync, it is possible to create the target directory on the server using the --rsync-path trick as follows:
rsync -av -e "ssh" --rsync-path "mkdir -p /home/user/new/new && rsync" ./file.txt user#10.0.2.60:/home/user/new/new
This however does not seem to work when using an ssh tunnel. The following command just hangs:
rsync -av -e "ssh -A user#10.0.2.61 ssh" --rsync-path "mkdir -p /home/user/new/new && rsync" ./file.txt user#10.0.2.60:/home/user
I have verified the last command works if I remove the --rsync-path argument and create the directory manually on the target device. But how to make rsync create the missing directory when using ssh tunneling?
Managed to solve it. The command inside --rsync-path must be wrapped in another level of quotes:
rsync -av -e "ssh -A user#10.0.2.61 ssh" --rsync-path "'mkdir -p /home/user/new/new && rsync'" ./file.txt user#10.0.2.60:/home/user
I want to use the xargs command to read the standard output from my date command. The following pipe works, creating a directory "2019-12-03" in the current directory.
date "+%Y-%m-%d" -r ../IDNumber/IDNumber.txt | xargs mkdir
What I would like it to do, however, is use the standard output from the data command to make a directory in a remote location, username#archivalstorage.university.edu:/remote/folder/path/, resulting in username#archivalstorage.university.edu:/remote/folder/path/2019-12-03.
Running the following command:
date "+%Y-%m-%d" -r ../IDNumber/IDNumber.txt | xargs mkdir -p username#archivalstorage.university.edu:/remote/folder/path/
This command does not give any kind of error, but no folder is actually created in the remote location I have specified.
mkdir doesn't support the remote directory. Use ssh instead, with command substitute:
ssh user#host 'cd /path/to/dir && mkdir `date +%Y-%m-%d`'
or with xargs:
date +%Y-%m-%d | xargs -I _ ssh user#host 'cd /tmp && mkdir _'
I have a Dockerfile, it does dotnet publish and the dll's are copied to intermediate docker container. I would like to copy the dll's which are generated in container to my local system (Host) as well.
I believe we can use "cp" command to do that, but I am not able to find a solution to get the intermediate container Id to use the "cp" command.
syntax: docker cp CONTAINER:Container_Path Host_Path.
Please suggest me any other better solution for this scenario.
Dockerfile:
FROM microsoft/aspnetcore-build:1.1.4 as builder
COPY . /Code
RUN dotnet restore /Code/MyProj.csproj
RUN dotnet publish -c Release /Code/MyProj.csproj
RUN cp CONTAINER: /Code/bin/Release/netcoreapp1.1/publish /binaries
Thanks.
This answer is outside of the Dockerfile.
first your Dockerfile would have to have a volume.
[VOLUME] /my/path/in/container
To get files into and out of a volume, try using tar -cvf and tar -xvf to put and get files between a container and a host.
To put files from host's newfiles.tar in pwd to a container at /var/lib/neo4j/conf mount.
docker run --rm \
-v my-volume-data:/my/path/in/container -v $(pwd):/newfiles ubuntu bash -c \
"cd /my/path/in/container && tar -xf /newfiles/newfiles.tar"
To get files from into a container at /my/path/in/container mount to a host oldfiles.tar.
docker run --rm \
-v my-volume-data:/my/path/in/container -v $(pwd):/newfiles ubuntu bash -c \
"cd /my/path/in/container && tar -cf /newfiles/origfiles.tar"
The --user 1000:1000 is optional if your container has a user with uid of 1000.
For the https://github.com/ellakcy/piwik-with-wordpress I am making a restore bash script in order to restore the backup generated from the https://github.com/ellakcy/piwik-with-wordpress/blob/master/scripts/pre-backup script
The main idea is to set a path with a tarball containing the backup and recreating the folders that volumes are mounted.
The script is the following:
#!/bin/bash
# Printing functions
black='\E[30;40m'
red='\E[31;40m'
green='\E[32;40m'
yellow='\E[33;40m'
blue='\E[34;40m'
magenta='\E[35;40m'
cyan='\E[36;40m'
white='\E[37;40m'
#Echo a string with color
cecho () # Color-echo.
# Argument $1 = message
# Argument $2 = color
{
local default_msg="No message passed."
# Doesn't really need to be a local variable.
message=${1:-$default_msg} # Defaults to default message.
color=${2:-$black} # Defaults to black, if not specified.
echo -e "$color"
echo "$message"
tput sgr0 # Reset to normal.
return
}
#Echo a string as error with color
cecho_err () # Color-echo.
# Argument $1 = message
# Argument $2 = color
{
local default_msg="No message passed."
# Doesn't really need to be a local variable.
message=${1:-$default_msg} # Defaults to default message.
color=${2:-$red} # Defaults to black, if not specified.
echo >&2 -e "$color"
echo >&2 "$message"
tput sgr0 # Reset to normal.
return
}
backup_file=${1}
cecho "Creating the correct folders" $cyan
cecho "Deleting data folder in order to recreate it" $red
sudo rm -rf ./data
mkdir ./data/
sudo chown root:root ./data/
sudo chmod 755 ./data/
if [ ! -f restore ]; then
mkdir ./restore/
fi
tar -xf ${backup_file} -C ./restore/
cecho "Restoring backup data for wordpress" $cyan
sudo mkdir ./data/wordpress
sudo chown root:root ./data/wordpress
sudo chmod 755 ./data/wordpress
sudo mv ./restore/wordpress/data/www ./data/wordpress/
sudo chown www-data:www-data ./data/wordpress/www
cecho "Restoring environment" $cyan
wordpress_env=$(tr '\n' ' ' <./restore/wordpress/env.txt)
echo ${wordpress_env}
cecho "Restoring database" $cyan
sudo mkdir ./data/wordpress/db
echo "sudo env ${wordpress_env} docker run --volume \"./data/wordpress/db\":/var/lib/mysql --volume ./restore/wordpress/db:/docker-entrypoint-initdb.d -e MYSQL_ROOT_PASSWORD=\$WORDPRESS_MYSQL_ROOT_PASSWORD -e MYSQL_DATABASE=\"wordpress\" -e MYSQL_USER=\$WORDPRESS_MYSQL_USER -e MYSQL_PASSWORD=\$WORDPRESS_MYSQL_PASSWORD mariadb" > ./restore_db.sh
chmod +x ./restore_db.sh
./restore_db.sh
# rm -rf ./restore_db.sh
rm -rf ./restore
And I get this error when I try to restore the database:
docker: Error response from daemon: create ./data/wordpress/db: "./data/wordpress/db" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed.
See 'docker run --help'.
As you can see it generates a temporary scripts (that later will be deleted) one example of generated script is:
sudo env WORDPRESS_MYSQL_ROOT_PASSWORD=passwd WORDPRESS_MYSQL_USER=wordpress WORDPRESS_MYSQL_PASSWORD=wordpress WORDPRESS_ADMIN_USER=admin WORDPRESS_ADMIN_PASSWORD=admin WORDPRESS_URL=http://0.0.0.0:8080 docker run --volume "./data/wordpress/db":/var/lib/mysql --volume ./restore/wordpress/db:/docker-entrypoint-initdb.d -e MYSQL_ROOT_PASSWORD=$WORDPRESS_MYSQL_ROOT_PASSWORD -e MYSQL_DATABASE="wordpress" -e MYSQL_USER=$WORDPRESS_MYSQL_USER -e MYSQL_PASSWORD=$WORDPRESS_MYSQL_PASSWORD mariadb
What is the best option in order to generate the correct volume data in ./data/wordpress/db that mounts on a container's /var/lib/mysql?
When we specify --volume <host_dir>:<container_dir>, host_dir must be an absolute path. If it is not an absolute path, then it considered to be the volume's name. Hence the message invalid characters for a local volume name. Try providing absolute path for the host directory.