Replace a string in shell script using a variable - unix

I am using the below code for replacing a string
inside a shell script.
echo $LINE | sed -e 's/12345678/"$replace"/g'
but it's getting replaced with $replace instead of the value of that variable.
Could anybody tell what went wrong?

If you want to interpret $replace, you should not use single quotes since they prevent variable substitution.
Try:
echo $LINE | sed -e "s/12345678/${replace}/g"
Transcript:
pax> export replace=987654321
pax> echo X123456789X | sed "s/123456789/${replace}/"
X987654321X
pax> _
Just be careful to ensure that ${replace} doesn't have any characters of significance to sed (like / for instance) since it will cause confusion unless escaped. But if, as you say, you're replacing one number with another, that shouldn't be a problem.

you can use the shell (bash/ksh).
$ var="12345678abc"
$ replace="test"
$ echo ${var//12345678/$replace}
testabc

Not specific to the question, but for folks who need the same kind of functionality expanded for clarity from previous answers:
# create some variables
str="someFileName.foo"
find=".foo"
replace=".bar"
# notice the the str isn't prefixed with $
# this is just how this feature works :/
result=${str//$find/$replace}
echo $result
# result is: someFileName.bar
str="someFileName.sally"
find=".foo"
replace=".bar"
result=${str//$find/$replace}
echo $result
# result is: someFileName.sally because ".foo" was not found

Found a graceful solution.
echo ${LINE//12345678/$replace}

Single quotes are very strong. Once inside, there's nothing you can do to invoke variable substitution, until you leave. Use double quotes instead:
echo $LINE | sed -e "s/12345678/$replace/g"

Let me give you two examples.
Using sed:
#!/bin/bash
LINE="12345678HI"
replace="Hello"
echo $LINE | sed -e "s/12345678/$replace/g"
Without Using sed:
LINE="12345678HI"
str_to_replace="12345678"
replace_str="Hello"
result=${str//$str_to_replace/$replace_str}
echo $result
Hope you will find it helpful!

echo $LINE | sed -e 's/12345678/'$replace'/g'
you can still use single quotes, but you have to "open" them when you want the variable expanded at the right place. otherwise the string is taken "literally" (as #paxdiablo correctly stated, his answer is correct as well)

To let your shell expand the variable, you need to use double-quotes like
sed -i "s#12345678#$replace#g" file.txt
This will break if $replace contain special sed characters (#, \). But you can preprocess $replace to quote them:
replace_quoted=$(printf '%s' "$replace" | sed 's/[#\]/\\\0/g')
sed -i "s#12345678#$replace_quoted#g" file.txt

I had a similar requirement to this but my replace var contained an ampersand. Escaping the ampersand like this solved my problem:
replace="salt & pepper"
echo "pass the salt" | sed "s/salt/${replace/&/\&}/g"

use # if you want to replace things like /. $ etc.
result=$(echo $str | sed "s#$oldstr#$newstr#g")
the above code will replace all occurrences of the specified replacement term
if you want, remove the ending g which means that the only first occurrence will be replaced.

Use this instead
echo $LINE | sed -e 's/12345678/$replace/g'
this works for me just simply remove the quotes

I prefer to use double quotes , as single quptes are very powerful as we used them if dont able to change anything inside it or can invoke the variable substituion .
so use double quotes instaed.
echo $LINE | sed -e "s/12345678/$replace/g"

Related

Not able to assign sed output to a variable

I am not able to capture the output of sed & cut using together in a variable. Below is the code snippet of a script:
max=$(sed -n '1,${/$i/p;q;}' $file | cut -d "," -f2)
When I print the value of max it is showing blank. But the code line is working fine when I execute it in terminal only like below:
sed -n '1,${/$i/p;q;}' $file | cut -d "," -f2
I am not able to understand why the assignment is failing. Could anyone please help me out here?
Regards,
Sayantan
As stated in the comments (and worked out for OP):
In single quotes $i is not a variable, but end-of-line followed by the character i after the end-of-line (impossible).

Unix: Remove the filename, get only the extension and rename the extension using sed

I have lot filenames which have this kind format:
118-edorf.sum.fil
118-edorf.sum.fil_1
118-edorf.sum.fil_11
i want to remove 118-edorf.sum. from the filename and get only the extension , fil , fil_1 and fil_11 and rename it to asc, asc_1 and asc_11.
So far, i can only remove 118-edorf.sum. using
sed 's/.*\.//'
The result will be
fil
fil_1
fil_11
So, how to rename it to
asc
asc_1
asc_11
To your solution only add additional substitution
sed 's/^.*\.//; s/fil/asc/'
To rename all files in directory using that criteria
rename 's/^.*\.//; s/fil/asc/' *
note: this rename command is untested
You can try this zsh foreach loop. Foreach can be somewhat slow for many files. You can also remove the echo statements for less noisier output.
foreach C (`ls 118-edorf.sum.fil*`)
f2=`echo $C|cut -d "." -f 3|cut -s -d "_" -f 2`
if [ "${f2}" -eq "" ]; then
echo "no underscore"
mv $C asc
elif
echo "_${f2}"
mv $C "asc_${f2}"
end
This might work for you (GNU sed):
sed -r 's/.*\.([^_]*(.*))/mv \1 asc\2/e' file
Use an evaluated substitution command. The substitution removes everything upto the last . and retains everything thereafter as the first backreference. Everything following the first _ within that backreference is also retained as the second backreference. The RHS of the substitution command the forms a mv commmand using the parts from the LHS.
If your sed does not have the e command/flag, use:
sed 's/.*\.\([^_]*\(.*\)\)/mv \1 asc\2/' file | shell
It might be safer to use:
sed -r 's/.*\.(fil(.*))/mv \1 asc\2/e' file

redirecting in a shell script

I'm trying to write a script to swap out text in a file:
sed s/foo/bar/g myFile.txt > myFile.txt.updated
mv myFile.txt.updated myFile.txt
I evoke the sed program, which swaps out text in myFile.txt and redirects the changed lines of text to a second file. mv then moves .updated txt file to myFile.txt, overwriting it. That command works in the shell.
I wrote:
#!/bin/sh
#First, I set up some descriptive variables for the arguments
initialString="$1"
shift
desiredChange="$1"
shift
document="$1"
#Then, I evoke sed on these (more readable) parameters
updatedDocument=`sed s/$initialString/$desiredChange/g $document`
#I want to make sure that was done properly
echo updated document is $updatedDocument
#then I move the output in to the new text document
mv $updatedDocument $document
I get the error:
mv: target `myFile.txt' is not a directory
I understand that it thinks my new file's name is the first word of the string that was sed's output. I don't know how to correct that. I've been trying since 7am and every quotation, creating a temporary file to store the output in (disastrous results), IFS...everything so far gives me more and more unhelpful errors. I need to clear my head and I need your help. How can I fix this?
Maybe try
echo $updatedDocument > $document
Change
updatedDocument=`sed s/$initialString/$desiredChange/g $document`
to
updatedDocument=${document}.txt
sed s/$initialString/$desiredChange/g $document
Backticks will actually put the entire piped output of the sed command into your variable value.
An even faster way would be to not use updatedDocument or mv at all by doing an in-place sed:
sed -i s/$initialString/$desiredChange/g $document
The -i flag tells sed to do the replacement in-place. This basically means creating a temp file for the output and replacing your original file with the temp file once it is done, pretty much exactly as you are doing.
#!/bin/sh
#First, I set up some descriptive variables for the arguments
echo "$1" | sed #translation of special regex char like . * \ / ? | read -r initialString
echo "$2" | sed 's|[\&/]|\\&|g' | read -r desiredChange
document="$3"
#Then, I evoke sed
sed "s/${initialString}/${desiredChange}/g" ${document} | tee ${document}
don't forget that initialString and desiredChange are pattern interpreted as regex, so a trnaslation is certainly needed
sed #translation of special regex char like . * \ / ? is to replace by the correct sed (discuss on several post on the site)

ZSH subString extraction

Goal
In ZSH script, for a given args, I want to obtain the first string and the rest.
For instance, when the script is named test
sh test hello
supposed to extract h and ello.
ZSH manual
http://zsh.sourceforge.net/Doc/zsh_a4.pdf
says:
Subscripting may also be performed on non-array values, in which case the subscripts specify a
substring to be extracted. For example, if FOO is set to ‘foobar’, then ‘echo $FOO[2,5]’ prints
‘ooba’.
Q1
So, I wrote a shell script in a file named test
echo $1
echo $1[1,1]
terminal:
$ sh test hello
hello
hello[1,1]
the result fails. What's wrong with the code?
Q2
Also I don't know how to extract subString from n to the last. Perhaps do I have to use Array split by regex?
EDIT: Q3
This may be another question, so if it's proper to start new Thread, I will do so.
Thanks to #skishore Here is the further code
#! /bin/zsh
echo $1
ARG_FIRST=`echo $1 | cut -c1`
ARG_REST=`echo $1 | cut -c2-`
echo ARG_FIRST=$ARG_FIRST
echo ARG_REST=$ARG_REST
if $ARG_FIRST = ""; then
echo nullArgs
else
if $ARG_FIRST = "#"; then
echo #Args
else
echo regularArgs
fi
fi
I'm not sure how to compare string valuables to string, but for a given args hello
result:
command not found: h
What's wrong with the code?
EDIT2:
What I've found right
#! /bin/zsh
echo $1
ARG_FIRST=`echo $1 | cut -c1`
ARG_REST=`echo $1 | cut -c2-`
echo ARG_FIRST=$ARG_FIRST
echo ARG_REST=$ARG_REST
if [ $ARG_FIRST ]; then
if [ $ARG_FIRST = "#" ]; then
echo #Args
else
echo regularArgs
fi
else
echo nullArgs
fi
EDIT3:
As the result of whole, this is what I've done with this question.
https://github.com/kenokabe/GitSnapShot
GitSnapShot is a ZSH thin wrapper for Git commands for easier and simpler usage
A1
As others have said, you need to wrap it in curly braces. Also, use a command interpreter (#!...), mark the file as executable, and call it directly.
#!/bin/zsh
echo $1
echo ${1[1,1]}
A2
The easiest way to extract a substring from a parameter (zsh calls variables parameters) is to use parameter expansion. Using the square brackets tells zsh to treat the scalar (i.e. string) parameter as an array. For a single character, this makes sense. For the rest of the string, you can use the simpler ${parameter:start:length} notation instead. If you omit the :length part (as we will here), then it will give you the rest of the scalar.
File test:
#!/bin/zsh
echo ${1[1]}
echo ${1:1}
Terminal:
$ ./test Hello
H
ello
A3
As others have said, you need (preferably double) square brackets to test. Also, to test if a string is NULL use -z, and to test if it is not NULL use -n. You can just put a string in double brackets ([[ ... ]]), but it is preferable to make your intentions clear with -n.
if [[ -z "${ARG_FIRST}" ]]; then
...
fi
Also remove the space between #! and /bin/zsh.
And if you are checking for equality, use ==; if you are assigning a value, use =.
RE:EDIT2:
Declare all parameters to set the scope. If you do not, you may clobber or use a parameter inherited from the shell, which may cause unexpected behavior. Google's shell style guide is a good resource for stuff like this.
Use builtins over external commands.
Avoid backticks. Use $(...) instead.
Use single quotes when quoting a literal string. This prevents pattern matching.
Make use of elif or case to avoid nested ifs. case will be easier to read in your example here, but elif will probably be better for your actual code.
Using case:
#!/bin/zsh
typeset ARG_FIRST="${1[1]}"
typeset ARG_REST="${1:1}"
echo $1
echo 'ARG_FIRST='"${ARG_FIRST}"
echo 'ARG_REST='"${ARG_REST}"
case "${ARG_FIRST}" in
('') echo 'nullArgs' ;;
('#') echo '#Args' ;;
(*)
# Recommended formatting example with more than 1 sloc
echo 'regularArgs'
;;
esac
using elif:
#!/bin/zsh
typeset ARG_FIRST="${1[1]}"
typeset ARG_REST="${1:1}"
echo $1
echo 'ARG_FIRST='"${ARG_FIRST}"
echo 'ARG_REST='"${ARG_REST}"
if [[ -z "${ARG_FIRST}" ]]; then
echo nullArgs
elif [[ '#' == "${ARG_FIRST}" ]]; then
echo #Args
else
echo regularArgs
fi
RE:EDIT3
Use "$#" unless you really know what you are doing. Explanation.
You can use the cut command:
echo $1 | cut -c1
echo $1 | cut -c2-
Use $() to assign these values to variables:
ARG_FIRST=$(echo $1 | cut -c1)
ARG_REST=$(echo $1 | cut -c2-)
echo ARG_FIRST=$ARG_FIRST
echo ARG_REST=$ARG_REST
You can also replace $() with backticks, but the former is recommended and the latter is somewhat deprecated due to nesting issues.
So, I wrote a shell script in a file named test
$ sh test hello
This isn't a zsh script: you're calling it with sh, which is (almost certainly) bash. If you've got the shebang (#!/bin/zsh), you can make it executable (chmod +x <script>) and run it: ./script. Alternatively, you can run it with zsh <script>.
the result fails. What's wrong with the code?
You can wrap in braces:
echo ${1} # This'll work with or without the braces.
echo ${1[3,5]} # This works in the braces.
echo $1[3,5] # This doesn't work.
Running this: ./test-script hello gives:
./test-script.zsh hello
hello
llo
./test-script.zsh:5: no matches found: hello[3,5]
Also I don't know how to extract subString from n to the last. Perhaps do I have to use Array split by regex?
Use the [n,last] notation, but wrap in braces. We can determine how long our variable is with, then use the length:
# Store the length of $1 in LENGTH.
LENGTH=${#1}
echo ${1[2,${LENGTH}]} # Display from `2` to `LENGTH`.
This'll produce ello (prints from the 2nd to the last character of hello).
Script to play with:
#!/usr/local/bin/zsh
echo ${1} # Print the input
echo ${1[3,5]} # Print from 3rd->5th characters of input
LENGTH=${#1}
echo ${1[2,${LENGTH}]} # Print from 2nd -> last characters of input.
You can use the cut command:
But that would be using extra baggage - zsh is quite capable of doing all this on it's own without spawning multiple sub-shells for simplistic operations.

sed extract something from string

I have a string " r1/pkg/amd64/misc/hash/hash-r1.r5218.tbz"
but, I only want "hash-r1.r5218.tbz"
so, I try this
unix$ a="r1/pkg/amd64/misc/hash/hash-r1.r5218.tbz"
unix$ echo $a | sed 's/.*\/\([^\/]*\)\.tbz/\1/' //[1]
hash-r1.r5218 //I know this should work
unix$ echo $a | sed 's/.*\/\([^\/]+\)\.tbz/\1/' //[2]
r1/pkg/amd64/misc/hash/hash-r1.r5218.tbz //however I do not know why it does not work.
as far as I remember, + in regexp, means using previous regexp 1 or more times. * in regexp, means using previous regexp 0 or more times.
Could anyone explain why [2] fails, thanks a lot.
a="r1/pkg/amd64/misc/hash/hash-r1.r5218.tbz"
echo $a | sed 's:.*/::; s:.tbz$::'
hash-r1.r5218
You don't need to use '/' as the patern/repl marker, you can use other chars. The ':' is very popular.
Also, you don't have to use capture buffers, when you know the exact text on both sides of your target data.
I have substituted out all chars up to the last '/', relying on .* for all chars, and '/' to terminate the standard greedy search of sed. THe you sub out the trailing \.tbz with noting.
IHTH.
Not all versions of sed support + in the regex. Some that do support it require -r to be specified. But why use sed instead of basename or echo ${a##*/}?
Using this submatch via parentheses will grab everything after the last slash to the end of your line.
str="r1/pkg/amd64/misc/hash/hash-r1.r5218.tbz"
echo $str | sed -n -E -e 's/.+\/(.+)$/\1/p'
returns hash-r1.r5218.tbz
Oh, and your #2 fails because sed by default prints out each line that has a match. Using the -n flag suppresses that, and the trailing 'p' on this regex prints out the replace part of the substitution.

Resources