How to use variables in a command in sed? - unix

I have abc.sh:
exec $ROOT/Subsystem/xyz.sh
On a Unix box, if I print echo $HOME then I get /HOME/COM/FILE.
I want to replace $ROOT with $HOME using sed.
Expected Output:
exec /HOME/COM/FILE/Subsystem/xyz.sh
I tried, but I'm not getting the expected output:
sed 's/$ROOT/"${HOME}"/g' abc.sh > abc.sh.1
Addition:
If I have abc.sh
exec $ROOT/Subsystem/xyz.sh $ROOT/ystem/xyz1.sh
then with
sed "s|\$INSTALLROOT/|${INSTALLROOT}|" abc.sh
it is only replacing first $ROOT, i.e., output is coming as
exec /HOME/COM/FILE/Subsystem/xyz.sh $ROOT/ystem/xyz1.sh

Say:
sed "s|\$ROOT|${HOME}|" abc.sh
Note:
Use double quotes so that the shell would expand variables.
Use a separator different than / since the replacement contains /
Escape the $ in the pattern since you don't want to expand it.
EDIT: In order to replace all occurrences of $ROOT, say
sed "s|\$ROOT|${HOME}|g" abc.sh

This might work for you:
sed 's|$ROOT|'"${HOME}"'|g' abc.sh > abc.sh.1

This may also can help
input="inputtext"
output="outputtext"
sed "s/$input/${output}/" inputfile > outputfile

The safe for a special chars workaround from https://www.baeldung.com/linux/sed-substitution-variables with improvement for \ char:
#!/bin/bash
to="/foo\\bar#baz"
echo "str %FROM% str" | sed "s#%FROM%#$(echo ${to//\\/\\\\} | sed 's/#/\\#/g')#g"

Related

back reference with sed command getting unexpected result

File Content
abab102
cdcd103
efef105
I want the username and id separated. Here, abab is user and 102 is id.
I use the command
sed 's/\([a-z]\)\{4\}\([0-9]\)\{3\}/Username:\1 ID:\2/' file.txt
Get this
Username:b ID:2
Username:d ID:3
Username:f ID:5
But I am expecting
Username:abab ID:102
Username:cdcd ID:103
Username:efef ID:105
But using the command
sed -e 's/\([a-z]\)\{4\}/Username:&/' -e 's/\([0-9]\)\{3\}/ID:&/' file2.txt
Output
Username:ababID:102
Username:cdcdID:103
Username:efefID:105
This output is close to what I need, but still I am expecting a blank space between Username:abab ID:102.
I want to know why \1 or \2 is not working here.
\([a-z]\) - between ( ) is one letter. If you repeat the \(\) group, back reference goes to the last matched expression. Put all the repetition inside.
's/\([a-z]\{4\}\)\([0-9]\{3\}\)/Username:\1 ID:\2/'
Ugh, simpler with extended:
sed -E 's/([a-z]{4})([0-9]{3})/Username:\1 ID:\2/'
Using sed
$ sed -E 's/[[:alpha:]]{4}/Username:& ID:/' input_file
Username:abab ID:102
Username:cdcd ID:103
Username:efef ID:105

Unix: Remove the filename, get only the extension and rename the extension using sed

I have lot filenames which have this kind format:
118-edorf.sum.fil
118-edorf.sum.fil_1
118-edorf.sum.fil_11
i want to remove 118-edorf.sum. from the filename and get only the extension , fil , fil_1 and fil_11 and rename it to asc, asc_1 and asc_11.
So far, i can only remove 118-edorf.sum. using
sed 's/.*\.//'
The result will be
fil
fil_1
fil_11
So, how to rename it to
asc
asc_1
asc_11
To your solution only add additional substitution
sed 's/^.*\.//; s/fil/asc/'
To rename all files in directory using that criteria
rename 's/^.*\.//; s/fil/asc/' *
note: this rename command is untested
You can try this zsh foreach loop. Foreach can be somewhat slow for many files. You can also remove the echo statements for less noisier output.
foreach C (`ls 118-edorf.sum.fil*`)
f2=`echo $C|cut -d "." -f 3|cut -s -d "_" -f 2`
if [ "${f2}" -eq "" ]; then
echo "no underscore"
mv $C asc
elif
echo "_${f2}"
mv $C "asc_${f2}"
end
This might work for you (GNU sed):
sed -r 's/.*\.([^_]*(.*))/mv \1 asc\2/e' file
Use an evaluated substitution command. The substitution removes everything upto the last . and retains everything thereafter as the first backreference. Everything following the first _ within that backreference is also retained as the second backreference. The RHS of the substitution command the forms a mv commmand using the parts from the LHS.
If your sed does not have the e command/flag, use:
sed 's/.*\.\([^_]*\(.*\)\)/mv \1 asc\2/' file | shell
It might be safer to use:
sed -r 's/.*\.(fil(.*))/mv \1 asc\2/e' file

Delete Special Word Using sed

I would like to use sed to remove all occurances of this line if and only if it is this
<ab></ab>
If this line, I would not want to delete it
<ab>keyword</ab>
My attempt that's not working:
sed '/<ab></ab>/d'
Thanks for any insight. I'm not sure what's wrong as I should not have to escape anything?
I'm using a shell script named temp to execute this. My command is this:
cat foobar.html | ./temp
This is my temp shell script:
#!/bin/sh
sed -e '/td/!d' | sed '/<ab></ab>/d'
It looks like we have a couple of problems here. The first is with the / in the close-tag. sed uses this to delimit different parts of the command. Fortunately, all we have to do is escape it with \. Try:
sed '/<ab><\/ab>/d'
Here's an example on my machine:
$ cat test
<ab></ab>
<ab></ab>
<ab>test</ab>
$ sed '/<ab><\/ab>/d' test
<ab>test</ab>
$
The other problem is that I'm not sure what the purpose of sed -e '/td/!d' is. In it's default operating mode, you don't need to tell it not to delete something; just tell it exactly what you want to delete.
So, to do this on a file called input.html:
sed '/<ab><\/ab>/d' input.html
Or, to edit the file in-place, you can just do:
sed -i -e '/<ab><\/ab>/d' input.html
Additionally, sed lets you use any character you want as a delimiter; you don't have to use /. So if you'd prefer not to escape your input, you can do:
sed '\#<ab></ab>#d' input.html
Edit
In the comments, you mentioned wanting to delete lines that only contain </ab> and nothing else. To do that, you need to do what's called anchoring the match. The ^ character represents the beginning of the line for anchoring, and $ represents the end of the line.
sed '/^<\/ab>$/d' input.html
This will only match a line that contains (literally) </ab> and nothing else at all, and delete the line. If you want to match lines that contain whitespace too, but no text other than </ab>:
sed '/^[[:blank:]]*<\/ab>[[:blank:]]*$/d' input.html
[[:blank:]]* matches "0 or more whitespace characters" and is called a "POSIX bracket expression".

redirecting in a shell script

I'm trying to write a script to swap out text in a file:
sed s/foo/bar/g myFile.txt > myFile.txt.updated
mv myFile.txt.updated myFile.txt
I evoke the sed program, which swaps out text in myFile.txt and redirects the changed lines of text to a second file. mv then moves .updated txt file to myFile.txt, overwriting it. That command works in the shell.
I wrote:
#!/bin/sh
#First, I set up some descriptive variables for the arguments
initialString="$1"
shift
desiredChange="$1"
shift
document="$1"
#Then, I evoke sed on these (more readable) parameters
updatedDocument=`sed s/$initialString/$desiredChange/g $document`
#I want to make sure that was done properly
echo updated document is $updatedDocument
#then I move the output in to the new text document
mv $updatedDocument $document
I get the error:
mv: target `myFile.txt' is not a directory
I understand that it thinks my new file's name is the first word of the string that was sed's output. I don't know how to correct that. I've been trying since 7am and every quotation, creating a temporary file to store the output in (disastrous results), IFS...everything so far gives me more and more unhelpful errors. I need to clear my head and I need your help. How can I fix this?
Maybe try
echo $updatedDocument > $document
Change
updatedDocument=`sed s/$initialString/$desiredChange/g $document`
to
updatedDocument=${document}.txt
sed s/$initialString/$desiredChange/g $document
Backticks will actually put the entire piped output of the sed command into your variable value.
An even faster way would be to not use updatedDocument or mv at all by doing an in-place sed:
sed -i s/$initialString/$desiredChange/g $document
The -i flag tells sed to do the replacement in-place. This basically means creating a temp file for the output and replacing your original file with the temp file once it is done, pretty much exactly as you are doing.
#!/bin/sh
#First, I set up some descriptive variables for the arguments
echo "$1" | sed #translation of special regex char like . * \ / ? | read -r initialString
echo "$2" | sed 's|[\&/]|\\&|g' | read -r desiredChange
document="$3"
#Then, I evoke sed
sed "s/${initialString}/${desiredChange}/g" ${document} | tee ${document}
don't forget that initialString and desiredChange are pattern interpreted as regex, so a trnaslation is certainly needed
sed #translation of special regex char like . * \ / ? is to replace by the correct sed (discuss on several post on the site)

How do I replace a token with the result of `pwd` in sed?

I'm trying to do something like this:
sed 's/#REPLACE-WITH-PATH/'`pwd`'/'
Unfortunately, I that errors out:
sed: -e expression #1, char 23: unknown option to `s'
Why does this happen?
You need to use a different character instead of /, eg.:
sed 's?#REPLACE-WITH-PATH?'`pwd`'?'
because / appears in the pwd output.
in sed, you can't use / directly, you must use '/'.
#!/bin/bash
dir=$`pwd`/
ls -1 | sed "s/^/${dir//\//\\/}/g"
sed 's:#REPLACE-WITH-PATH:'`pwd`':' config.ini
The problem is one of escaping the output of pwd correctly. Fortunately, as in vim, sed supports using a different delimiter character. In this case, using the colon instead of slash as a delimiter avoids the escaping problem.
instead of fumbling around with quotes like that, you can do it like this
#!/bin/bash
p=`pwd`
# pass the variable p to awk
awk -v p="$p" '$0~p{ gsub("REPLACE-WITH-PATH",p) }1' file >temp
mv temp file
or just bash
p=`pwd`
while read line
do
line=${line/REPLACE-WITH-PATH/$p}
echo $line
done < file > temp
mv temp file

Resources