I wrote a script in R that has several arguments. I want to iterate over 20 directories and execute my script on each while passing in a substring from the file path as my -n argument using sed. I ran the following:
find . -name 'xray_data' -exec sh -c 'Rscript /Users/Caitlin/Desktop/DeMMO_Pubs/DeMMO_NativeRock/DeMMO_NativeRock/R/scipts/dataStitchR.R -f {} -b "{}/SEM_images" -c "{}/../coordinates.txt" -z ".tif" -m ".tif" -a "Unknown|SEM|Os" -d "overview" -y "overview" --overview "overview.*tif" -p FALSE -n "`sed -e 's/.*DeMMO.*[/]\(.*\)_.*[/]xray_data/\1/' "{}"`"' sh {} \;
which results in this error:
ubs/DeMMO_NativeRock/DeMMO_NativeRock/R/scipts/dataStitchR.R -f {} -b "{}/SEM_images" -c "{}/../coordinates.txt" -z ".tif" -m ".tif" -a "Unknown|SEM|Os" -d "overview" -y "overview" --overview "overview.*tif" -p FALSE -n "`sed -e 's/.*DeMMO.*[/]\(.*\)_.*[/]xray_data/\1/' "{}"`"' sh {} \;
sh: command substitution: line 0: syntax error near unexpected token `('
sh: command substitution: line 0: `sed -e s/.*DeMMO.*[/](.*)_.*[/]xray_data/1/ "./DeMMO1/D1T3rep_Dec2019_Ellison/xray_data"'
When I try to use sed with my pattern on an example file path, it works:
echo "./DeMMO1/D1T1exp_Dec2019_Poorman/xray_data" | sed -e 's/.*DeMMO.*[/]\(.*\)_.*[/]xray_data/\1/'
which produces the correct substring:
D1T1exp_Dec2019
I think there's an issue with trying to use single quotes inside the interpreted string but I don't know how to deal with this. I have tried replacing the single quotes around the sed pattern with double quotes as well as removing the single quotes, both result in this error:
sed: RE error: illegal byte sequence
How should I extract the substring from the file path dynamically in this case?
To loop through the output of find.
while IFS= read -ru "$fd" -d '' files; do
echo "$files" ##: do whatever you want to do with the files here.
done {fd}< <(find . -type f -name 'xray_data' -print0)
No embedded commands in quotes.
It uses a random fd just in case something inside the loop is eating/slurping stdin
Also -print0 delimits the files with null bytes, so it should be safe enough to handle spaces tabs and newlines on the path and file names.
A good start is always put an echo in front of every commands you want to do with the files, so you have an idea what's going to be executed/happen just in case...
This is the solution that ultimately worked for me due to issues with quotes in sed:
for dir in `find . -name 'xray_data'`;
do sampleID="`basename $(dirname $dir) | cut -f1 -d'_'`";
Rscript /Users/Caitlin/Desktop/DeMMO_Pubs/DeMMO_NativeRock/DeMMO_NativeRock/R/scipts/dataStitchR.R -f "$dir" -b "$dir/SEM_images" -c "$dir/../coordinates.txt" -z ".tif" -m ".tif" -a "Unknown|SEM|Os" -d "overview" -y "overview" --overview "overview.*tif" -p FALSE -n "$sampleID";
done
So I'm not exactly sure whether this is something wrong with optparse-applicative's script or if I'm using it wrong.
In the optparse-applicative readme, it states that programs are made available with automatic completion scripts, with options for zsh. For my program setup:
$> setup --zsh-completion-script `which setup`
Outputs:
#compdef setup
local request
local completions
local word
local index=$((CURRENT - 1))
request=(--bash-completion-enriched --bash-completion-index $index)
for arg in ${words[#]}; do
request=(${request[#]} --bash-completion-word $arg)
done
IFS=$'\n' completions=($( /Users/anrothan/.local/bin/setup "${request[#]}" ))
for word in $completions; do
local -a parts
# Split the line at a tab if there is one.
IFS=$'\t' parts=($( echo $word ))
if [[ -n $parts[2] ]]; then
if [[ $word[1] == "-" ]]; then
local desc=("$parts[1] ($parts[2])")
compadd -d desc -- $parts[1]
else
local desc=($(print -f "%-019s -- %s" $parts[1] $parts[2]))
compadd -l -d desc -- $parts[1]
fi
else
compadd -f -- $word
fi
done
I'm running the following in my zshrc (I use oh-my-zsh, but I removed it and this still happens in a bare-minimum config with only a small PATH addition to get the setup script).
autoload -U +X compinit && compinit
autoload -U +X bashcompinit && bashcompinit
source <(setup --zsh-completion-script `which setup`)
I get the following error several times:
/dev/fd/11:compadd:24: can only be called from completion function
I've run compinit, and the completion script seems to look right to me, and I've looked around but I can't seem to figure out why this error is happening...
You don't need to source zsh-completion scripts, they just need to be added to your fpath parameter.
So just place the output of setup --zsh-completion-script $(which setup) in a file call _setup in $HOME/.config/zsh/completions.
fpath=($HOME/.config/zsh/completions $fpath)
autoload -U compinit && compinit
I have this alias in my .zshrc file:
alias rmcons="docker rm -f $(docker ps -aq)"
But after trying to execute it, it removes only one container and then it prints
$rmcons
ef8197f147fb
zsh: command not found: c2ea2673f9e4
zsh: command not found: 4603059f1618
zsh: command not found: 40ad60328595
How can I remove all containers that docker ps -aq shows?
You need to use single quotes ('') instead of double quotes ("").
alias rmcons='docker rm -f $(docker ps -aq)'
If you use double quotes, than the command substitution $(docker ps -aq) will be evaluated when you define the alias. In your example this was equivalent to
alias rmcons="docker rm -f ef8197f147fb
c2ea2673f9e4
4603059f1618
40ad60328595"
As the newlines are command separators (like ;) this alias is substituted by four commands: docker rm -f ef8197f147fb, c2ea2673f9e4, 4603059f1618 and 40ad60328595. The last three of which do not exist on your system, hence "command not found". It also means that the same output of docker ps -aq - as it was on alias definiton - will be used and not as it would be when running the alias.
On the other hand, if you use single quotes, the alias will actually substituted by the exact command you defined: docker rm -f $(docker ps -aq). Although docker ps -aq will still return output with newlines, these newlines are now only parsed word separators between arguments.
Warning: untested. I don't use/have docker.
I think you should serialize the output first "escaping" the new lines.
You might also use the for loop, trying:
for id in `docker ps -aq`; do docker rm -f $id; done
Note the backquotes to parse the command's output.
You can also directly use $() instead of its shortcut backquote.
I recommend to test with echo first instead of removing with rm:
for id in `docker -ps -aq`; do echo rm -f $id; done
, and to use the rm with its -i switch to prompt for confirmation before deleting.
I hope docker's rm subcommand has one.
How can I insert the content of the variable $SALT in a specific point (line or string) of a file like wp-contet.php from wordpress using Bash script?
SALT=$(curl -L https://api.wordpress.org/secret-key/1.1/salt/)
I'm not an expert at parsing text files in bash but you should delete the lines that define the things you're downloading from the wordpress salt and then insert the variable at the end... something like:
#!/bin/sh
SALT=$(curl -L https://api.wordpress.org/secret-key/1.1/salt/)
STRING='put your unique phrase here'
printf '%s\n' "g/$STRING/d" a "$SALT" . w | ed -s wp-config.php
OK, now it's fixed... it should look for where the salt is supposed to go and it will replace it with the info retrieved from https://api.wordpress.org/secret-key/1.1/salt/
This version defines new keys if none exist, and also replaces existing keys:
#!/bin/bash
find . -name wp-config.php -print | while read line
do
curl http://api.wordpress.org/secret-key/1.1/salt/ > wp_keys.txt
sed -i.bak -e '/put your unique phrase here/d' -e \
'/AUTH_KEY/d' -e '/SECURE_AUTH_KEY/d' -e '/LOGGED_IN_KEY/d' -e '/NONCE_KEY/d' -e \
'/AUTH_SALT/d' -e '/SECURE_AUTH_SALT/d' -e '/LOGGED_IN_SALT/d' -e '/NONCE_SALT/d' $line
cat wp_keys.txt >> $line
rm wp_keys.txt
done
If you have csplit available, you can split the original wp-config.php file either side of the salt definitions, download new salts, then cat back together. This keeps the PHP define() statements at the same location in wp-config.php instead of than moving them to a different location within the file:
# Download new salts
curl "https://api.wordpress.org/secret-key/1.1/salt/" -o salts
# Split wp-config.php into 3 on the first and last definition statements
csplit wp-config.php '/AUTH_KEY/' '/NONCE_SALT/+1'
# Recombine the first part, the new salts and the last part
cat xx00 salts xx02 > wp-config.php
# Tidy up
rm salts xx00 xx01 xx02
How about using sed?
cat wp-config.php | sed 's/old_string/new_string/g' > wp-config.php
I think I got this one! its a bash script using only commands normally available at the command prompt and it does -everything- (assuming httpd is your web user) except create the databases. here you go.
#!/bin/bash
# wordpress latest auto-install script, by alienation 24 jan 2013. run as root.
# usage: ~/wp-install alien /hsphere/local/home/alien/nettrip.org alien_wpdbname alien_wpdbusername p#sSw0rd
# ( wp-install shell-user folder db-name db-user-name db-user-pw )
# download wordpress to temporary area
cd /tmp
rm -rf tmpwp
mkdir tmpwp
cd tmpwp
wget http://wordpress.org/latest.tar.gz
tar -xvzpf latest.tar.gz
# copy wordpress to where it will live, and go there, removing index placeholder if there is one
mv wordpress/* $2
cd $2
rm index.html
# create config from sample, replacing salt example lines with a real salt from online generator
grep -A 1 -B 50 'since 2.6.0' wp-config-sample.php > wp-config.php
wget -O - https://api.wordpress.org/secret-key/1.1/salt/ >> wp-config.php
grep -A 50 -B 3 'Table prefix' wp-config-sample.php >> wp-config.php
# put the appropriate db info in place of placeholders in our new config file
replace 'database_name_here' $3 -- wp-config.php
replace 'username_here' $4 -- wp-config.php
replace 'password_here' $5 -- wp-config.php
# change file ownership and permissions according to ideal at http://codex.wordpress.org/Hardening_WordPress#File_Permissions
touch .htaccess
chown $1:httpd .htaccess
chown -R $1:httpd *
find . -type d -exec chmod 755 {} \;
find . -type f -exec chmod 644 {} \;
chmod -R 770 wp-content
chmod -R g-w wp-admin wp-includes wp-content/plugins
chmod g+w .htaccess
# thats it!
echo ALL DONE
I built a simple CLI for just that. Try it out. It's called [WP-Salts-Update-CLI][1].
WP-Salts-Update-CLI
WPSUCLI downloads new salts from the WP API and replaces them with the ones in your wp-config.php file for every site on your server.
⚡️ Installation
Open command line terminal (I prefer iTerm2) and run the following command.
bash
sudo wget -qO wpsucli https://git.io/vykgu && sudo chmod +x ./wpsucli && sudo install ./wpsucli /usr/local/bin/wpsucli
This command will perform the following actions:
Use sudo permissions
Use wget to download WPSUCLI and rename it to wpsucli
Make the wpsucli executable
Install wpsucli inside /usr/local/bin/ folder.
🙌 Usage
Just run wpsucli and it will update the salts for every wp-config.php file on your server or PC.
This is the bash script that I came up with that works on my Ubuntu server. I modified the examples from above.
Its a bit of brute force in that it will only replace the 8 keys that currently are required and expects the server to return exactly the same length key every time. The script works well for my use case so I thought I would share it.
CONFIG_FILE=wp-config.php
SALT=$(curl -L https://api.wordpress.org/secret-key/1.1/salt/)
SRC="define('AUTH_KEY'"; DST=$(echo $SALT|cat|grep -o define\(\'AUTH_KEY\'.\\{70\\}); sed -i "/$SRC/c$DST" $CONFIG_FILE
SRC="define('SECURE_AUTH_KEY'"; DST=$(echo $SALT|cat|grep -o define\(\'SECURE_AUTH_KEY\'.\\{70\\}); sed -i "/$SRC/c$DST" $CONFIG_FILE
SRC="define('LOGGED_IN_KEY'"; DST=$(echo $SALT|cat|grep -o define\(\'LOGGED_IN_KEY\'.\\{70\\}); sed -i "/$SRC/c$DST" $CONFIG_FILE
SRC="define('NONCE_KEY'"; DST=$(echo $SALT|cat|grep -o define\(\'NONCE_KEY\'.\\{70\\}); sed -i "/$SRC/c$DST" $CONFIG_FILE
SRC="define('AUTH_SALT'"; DST=$(echo $SALT|cat|grep -o define\(\'AUTH_SALT\'.\\{70\\}); sed -i "/$SRC/c$DST" $CONFIG_FILE
SRC="define('SECURE_AUTH_SALT'"; DST=$(echo $SALT|cat|grep -o define\(\'SECURE_AUTH_SALT\'.\\{70\\}); sed -i "/$SRC/c$DST" $CONFIG_FILE
SRC="define('LOGGED_IN_SALT'"; DST=$(echo $SALT|cat|grep -o define\(\'LOGGED_IN_SALT\'.\\{70\\}); sed -i "/$SRC/c$DST" $CONFIG_FILE
SRC="define('NONCE_SALT'"; DST=$(echo $SALT|cat|grep -o define\(\'NONCE_SALT\'.\\{70\\}); sed -i "/$SRC/c$DST" $CONFIG_FILE
I tried the accepted solution:
#!/bin/sh
SALT=$(curl -L https://api.wordpress.org/secret-key/1.1/salt/)
STRING='put your unique phrase here'
printf '%s\n' "g/$STRING/d" a "$SALT" . w | ed -s wp-config.php
However it does not work perfectly as for some reason it induces the SALTS to "move down" 1 line in the wp-config.php file each time it is used... it is not ideal if you are going to change SALTS automatically like every week, months with cron for example...
A better solution for me was to create a little function that I call in my script.
This function creates a file with the SALTS (deletes it at the end), deletes every lines containing one of the SALTS then just inserts the SALTS contained in the file in place of the initial SALTS.
This works perfectly.
fct_update_salts() {
# Requires website name as target
curl http://api.wordpress.org/secret-key/1.1/salt/ > ~/SALTS.txt
var_initial_path1=`pwd`
cd ~ #going to home directory
# This scripts eliminates successively all SALT entries, replaces the last one by XXX as a marker, places SALTS.txt, below XXX and deletes XXX
sudo sed -i "/SECURE_AUTH_KEY/d" $1/wp-config.php
sudo sed -i "/LOGGED_IN_KEY/d" $1/wp-config.php
sudo sed -i "/NONCE_KEY/d" $1/wp-config.php
sudo sed -i "/AUTH_SALT/d" $1/wp-config.php
sudo sed -i "/SECURE_AUTH_SALT/d" $1/wp-config.php
sudo sed -i "/LOGGED_IN_SALT/d" $1/wp-config.php
sudo sed -i "/NONCE_SALT/d" $1/wp-config.php
sudo sed -i "/AUTH_KEY/cXXX" $1/wp-config.php
sudo sed -i '/XXX/r SALTS.txt' $1/wp-config.php
sudo sed -i "/XXX/d" $1/wp-config.php
echo "SALTS REPLACED BY:"
echo "====================="
cat ~/SALTS.txt
sudo rm -rf ~/SALTS.txt
cd $var_initial_path1
}
The function is to be called in the script like this:
# Reset SALTS
fct_update_salts $SITE_PATH
Where $SITE_PATH="/var/www/html/YOUR_WEBSITE" or whatever path works for you.
I was challenged with the same issue. Here is the script I wrote to replace the salts and keys from ones downloaded from WordPress. You can use it at any time to replace them if/when needed. I run it as sudo, and the script tests for that. If you use an account that can download to the directory and make updates to the wp-config.php file, then you can delete that part of the script.
#!/bin/sh
# update-WordPress-Salts: Updates WordPress Salts
# written by Wayne Woodward 2017
if [ $# -lt 1 ]; then
echo "Usage: update-WordPress-Salts directory"
exit
fi
if [ "$(whoami)" != "root" ]; then
echo "Please run as root (sudo)"
exit
fi
WPPATH=$1
# Update the salts in the config file
# Download salts from WordPress and save them locally
curl http://api.wordpress.org/secret-key/1.1/salt/ > /var/www/$WPPATH/wp-keys.txt
# Iterate through each "Saltname" and append 1 to it
# For a couple names that may match twice like "AUTH_KEY" adds extra 1s to the end
# But that is OK as when this deletes the lines, it uses the same matching pattern
# (Smarter people may fix this)
for SALTNAME in AUTH_KEY SECURE_AUTH_KEY LOGGED_IN_KEY NONCE_KEY AUTH_SALT SECURE_AUTH_SALT LOGGED_IN_SALT NONCE_SALT
do
sed -i -e "s/$SALTNAME/${SALTNAME}1/g" /var/www/$WPPATH/wp-config.php
done
# Find the line that has the updated AUTH_KEY1 name
# This is so we can insert the file in the same area
line=$(sed -n '/AUTH_KEY1/{=;q}' /var/www/$WPPATH/wp-config.php)
# Insert the file from the WordPress API that we saved into the configuration
sed -i -e "${line}r /var/www/$WPPATH/wp-keys.txt" /var/www/$WPPATH/wp-config.php
# Itererate through the old keys and remove them from the file
for SALTNAME in AUTH_KEY SECURE_AUTH_KEY LOGGED_IN_KEY NONCE_KEY AUTH_SALT SECURE_AUTH_SALT LOGGED_IN_SALT NONCE_SALT
do
sed -i -e "/${SALTNAME}1/d" /var/www/$WPPATH/wp-config.php
done
# Delete the file downloaded from Wordpress
rm /var/www/$WPPATH/wp-keys.txt
Many of the answers rely on the phrase 'put your unique phrase here' being present in the file, so they do not work when you want to change salts after the first time. There are also some that remove the old definitions and append the new ones at the end. While that does work, it's nice to keep the definitions where you would expect them, right after the comment documenting them. My solution addresses those issues.
I made a few attempts with sed, perl and regex, but there are special characters in the salts and the rest of the config file that tend to mess things up. I ended up using grep to search the document for the unique comment structure that opens and closes the salt definition block, which has the following format:
/**##+
<comment documentation>
*/
<salt definitions>
/**##-*/
Note that if that comment structure is removed or altered, this will no longer work. Here's the script:
#!/bin/bash -e
# Set Default Settings:
file='wp-config.php'
# set up temporary files with automatic removal:
trap "rm -f $file_start $file_end $salt" 0 1 2 3 15
file_start=$(mktemp) || exit 1
file_end=$(mktemp) || exit 1
salt=$(mktemp) || exit 1
function find_line {
# returns the first line number in the file which contains the text
# program exits if text is not found
# $1 : text to search for
# $2 : file in which to search
# $3 (optional) : line at which to start the search
line=$(tail -n +${3:-1} $2 | grep -nm 1 $1 | cut -f1 -d:)
[ -z "$line" ] && exit 1
echo $(($line + ${3:-1} - 1))
}
line=$(find_line "/**##+" "$file")
line=$(find_line "\*/" "$file" "$line")
head -n $line $file > $file_start
line=$(find_line "/**##-\*/" "$file" "$line")
tail -n +$line $file > $file_end
curl -Ls https://api.wordpress.org/secret-key/1.1/salt/ > $salt
(cat $file_start $salt; echo; cat $file_end) > $file
exit 0
Strings containing single asterisks, such as "*/" and "/**##-*/" want to expand to directory lists, so that is why those asterisks are escaped.
Here's a pure bash approach. This does not depend on wordpress.org.
I converted the original wp_generate_password() function used by WordPress to generate salt.
#!/bin/bash
set -e
#
# Generates a random password drawn from the defined set of characters.
# Inspired by WordPress function https://developer.wordpress.org/reference/functions/wp_generate_password/
#
# Parameters
# ----------
# $length
# (ing) (Optional) Length of password to generate.
# Default value: 12
# $special_chars
# (bool) (Optional) Whether to include standard special characters.
# Default value: true
# $extra_special_chars
# (bool) (Optional) Whether to include other special characters. Used when generating secret keys and salts.
# Default value: false
#
function wp_generate_password() {
# Args
length="$(test $1 && echo $1 || echo 12 )"
special_chars="$(test $2 && echo $2 || echo 1 )"
extra_special_chars="$(test $3 && echo $3 || echo 0 )"
chars='abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789'
[[ $special_chars != 0 ]] && chars="$chars"'!##$%^&*()'
[[ $extra_special_chars != 0 ]] && chars="$chars"'-_ []{}<>~`+=,.;:/?|'
password='';
for i in $(seq 1 $length); do
password="${password}${chars:$(( RANDOM % ${#chars} )):1}"
done
echo "$password"
}
You can then just run SALT="$(wp_generate_password 64 1 1)".
Update
I just published a standalone script to generate WP salt values. You can generate the salt values by running ./wp-generate-salt.sh.
If the wordpress.org API generated SALT values are not necessary for your use case, you can use the pwgen to generate keys on the server and insert those into wp-config.php.
for i in {1..8} ;do unique_key="`pwgen -1 -s 64`";sudo sed -i "0,/put your unique phrase here/s/put your unique phrase here/$unique_key/" /srv/www/wordpress/wp-config.php; done
You may need to fix the ownership of the file after using sudo. You can use a command similar to this for changing the ownership.
chown www-data:www-data /srv/www/wordpress/wp-config.php
I just installed Ruby Version Manager (RVM) which is working fine but it asked me to put the following line in my /.bash_profile and ~/.bashrc files:
if [[ -s /Users/tammam56/.rvm/scripts/rvm ]] ; then source /Users/tammam56/.rvm/scripts/rvm ; fi
looking at the content I see the following:
tammam56$ cat /.bash_profile
export PATH="/usr/local/bin:/usr/local/sbin:/usr/local/mysql/bin:$PATH"
if [[ -s /Users/tammam56/.rvm/scripts/rvm ]] ; then source /Users/tammam56/.rvm/scripts/rvm ; fi
tammam56$ cat ~/.bashrc
export PATH=/usr/local/bin:$PATH
export MANPATH=/usr/local/man:$MANPATH
if [[ -s /Users/tammam56/.rvm/scripts/rvm ]] ; then source /Users/tammam56/.rvm/scripts/rvm ; fi
However when I start new Terminal window it doesn't execute the command I know this as I set my default Ruby verion to 1.9 and if I execute the line manually I get to the correct version:
tammam56$ ruby -v
ruby 1.8.7 (2008-08-11 patchlevel 72) [universal-darwin10.0]
Macintosh-37:~ tammam56$ if [[ -s /Users/tammam56/.rvm/scripts/rvm ]] ; then source /Users/tammam56/.rvm/scripts/rvm ; fi
Macintosh-37:~ tammam56$ ruby -v
ruby 1.9.1p378 (2010-01-10 revision 26273) [i386-darwin10.2.0]
Any ideas how I can fix that?
Thanks,
Tam
If you have a ~/.profile try adding the following line to it.
if [[ -s /Users/tammam56/.rvm/scripts/rvm ]] ; then source /Users/tammam56/.rvm/scripts/rvm ; fi
Have you confirmed that other commands in .bashrc and/or .bash_profile get updated properly?
Next you should confirm whether Terminal is starting the shell as a login shell or not (see under Preferences -> Startup).
From the bash man page:
When bash is invoked as an interactive login shell, or as a non-inter-
active shell with the --login option, it first reads and executes com-
mands from the file /etc/profile, if that file exists. After reading
that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile,
in that order, and reads and executes commands from the first one that
exists and is readable. The --noprofile option may be used when the
shell is started to inhibit this behavior.
When an interactive shell that is not a login shell is started, bash
reads and executes commands from ~/.bashrc, if that file exists. This
may be inhibited by using the --norc option. The --rcfile file option
will force bash to read and execute commands from file instead of
~/.bashrc.
I found that the easiest solution to this issue was to move all of the contents from my ~/.bashrc file (so my ~/.bashrc is now empty) to a new ~/.bash_profile file. Now all Terminal.app tabs automatically run all lines included in ~/.bash_profile. This also applies to iTerm.