redirecting in a shell script - unix

I'm trying to write a script to swap out text in a file:
sed s/foo/bar/g myFile.txt > myFile.txt.updated
mv myFile.txt.updated myFile.txt
I evoke the sed program, which swaps out text in myFile.txt and redirects the changed lines of text to a second file. mv then moves .updated txt file to myFile.txt, overwriting it. That command works in the shell.
I wrote:
#!/bin/sh
#First, I set up some descriptive variables for the arguments
initialString="$1"
shift
desiredChange="$1"
shift
document="$1"
#Then, I evoke sed on these (more readable) parameters
updatedDocument=`sed s/$initialString/$desiredChange/g $document`
#I want to make sure that was done properly
echo updated document is $updatedDocument
#then I move the output in to the new text document
mv $updatedDocument $document
I get the error:
mv: target `myFile.txt' is not a directory
I understand that it thinks my new file's name is the first word of the string that was sed's output. I don't know how to correct that. I've been trying since 7am and every quotation, creating a temporary file to store the output in (disastrous results), IFS...everything so far gives me more and more unhelpful errors. I need to clear my head and I need your help. How can I fix this?

Maybe try
echo $updatedDocument > $document

Change
updatedDocument=`sed s/$initialString/$desiredChange/g $document`
to
updatedDocument=${document}.txt
sed s/$initialString/$desiredChange/g $document
Backticks will actually put the entire piped output of the sed command into your variable value.
An even faster way would be to not use updatedDocument or mv at all by doing an in-place sed:
sed -i s/$initialString/$desiredChange/g $document
The -i flag tells sed to do the replacement in-place. This basically means creating a temp file for the output and replacing your original file with the temp file once it is done, pretty much exactly as you are doing.

#!/bin/sh
#First, I set up some descriptive variables for the arguments
echo "$1" | sed #translation of special regex char like . * \ / ? | read -r initialString
echo "$2" | sed 's|[\&/]|\\&|g' | read -r desiredChange
document="$3"
#Then, I evoke sed
sed "s/${initialString}/${desiredChange}/g" ${document} | tee ${document}
don't forget that initialString and desiredChange are pattern interpreted as regex, so a trnaslation is certainly needed
sed #translation of special regex char like . * \ / ? is to replace by the correct sed (discuss on several post on the site)

Related

Changing a line in a file on UNIX

I have created a file named "asd.txt" on a UNIX based system.
I added four lines by using echo command.
Now, I would like to change the first line of this file.
I am not allowed to use any text editors, such as vi.
I have to do this by using only command line. Can anyone help?
Thanks.
Here is how you could do it with sed.
sed '1 s/search/replace/' asd.txt
If you are feeling up to it and have GNU sed, use the -i switch to do it in place.
If you want to replace the entire first line how about doing this?
echo "Here is my new first line" && sed '1d' asd.txt
For both of these commands you can redirect the output to a new file using the > operator.
#!/bin/bash
cat <(echo "Replacement") <(tail -n +2 foo.txt)

Unix replace a particular type of string in file through unix

I have a situation that I need to replace a particular type of string in a file.
Scenario is:
user input like this:
abc = 21
xyz=32;34;35
The user can input many numbers in xyz but format should be ";" separated values
Now I need to replace these values in a particular file suppose test.txt
This file has a format like this:
test.txt
cond0=abc
cond1=xyz
Cond2=abcxyz%
hence output should be like this
cond0=21
cond1=32;34;35
cond2=2132%;2134%;2135%
I am using below command to do this but from this I am not able to get right output in cond2
sed "s/abc/${abc}/g" "$TEST_DIR/$file" > "$TEST_DIR/$file.bak" && mv "$TEST_DIR/$file.bak" "$TEST_DIR/$file"
sed "s/xyz/${xyz}/g" "$TEST_DIR/$file" > "$TEST_DIR/$file.bak" && mv "$TEST_DIR/$file.bak" "$TEST_DIR/$file"
Can anyone have a look at this?
Why can't you pipe through all of your conditions?
sed "s/abc/$abc/g" <$TEST_DIR/$file | sed "s/xyz/$xyz/g" >$TEST_DIR/newfile
mv $TEST_DIR/newfile $TEST_DIR/$file
Note that you will have to make this two operations, i.e., writing to a temporary file and then renaming. Otherwise you'll end up wiping out the file.
The input "<" and output ">" redirections are handled by the shell, and the moment they see ">somefile", the "somefile" is truncated. So you can never do cat <file >file successfully.
Using bash,
abc=21
xyz='32;34;35'
abcxyz=$(sed -r "s/^|;/\0${abc}/g;s/;|$/%\0/g" <<< "${xyz}")
sed -i~ "s/abcxyz%/${abcxyz}/;s/abc/${abc}/;s/xyz/${xyz}/" inputFile

Interpret as fixed string/literal and not regex using sed

For grep there's a fixed string option, -F (fgrep) to turn off regex interpretation of the search string.
Is there a similar facility for sed? I couldn't find anything in the man. A recommendation of another gnu/linux tool would also be fine.
I'm using sed for the find and replace functionality: sed -i "s/abc/def/g"
Do you have to use sed? If you're writing a bash script, you can do
#!/bin/bash
pattern='abc'
replace='def'
file=/path/to/file
tmpfile="${TMPDIR:-/tmp}/$( basename "$file" ).$$"
while read -r line
do
echo "${line//$pattern/$replace}"
done < "$file" > "$tmpfile" && mv "$tmpfile" "$file"
With an older Bourne shell (such as ksh88 or POSIX sh), you may not have that cool ${var/pattern/replace} structure, but you do have ${var#pattern} and ${var%pattern}, which can be used to split the string up and then reassemble it. If you need to do that, you're in for a lot more code - but it's really not too bad.
If you're not in a shell script already, you could pretty easily make the pattern, replace, and filename parameters and just call this. :)
PS: The ${TMPDIR:-/tmp} structure uses $TMPDIR if that's set in your environment, or uses /tmp if the variable isn't set. I like to stick the PID of the current process on the end of the filename in the hopes that it'll be slightly more unique. You should probably use mktemp or similar in the "real world", but this is ok for a quick example, and the mktemp binary isn't always available.
Option 1) Escape regexp characters. E.g. sed 's/\$0\.0/0/g' will replace all occurrences of $0.0 with 0.
Option 2) Use perl -p -e in conjunction with quotemeta. E.g. perl -p -e 's/\\./,/gi' will replace all occurrences of . with ,.
You can use option 2 in scripts like this:
SEARCH="C++"
REPLACE="C#"
cat $FILELIST | perl -p -e "s/\\Q$SEARCH\\E/$REPLACE/g" > $NEWLIST
If you're not opposed to Ruby or long lines, you could use this:
alias replace='ruby -e "File.write(ARGV[0], File.read(ARGV[0]).gsub(ARGV[1]) { ARGV[2] })"'
replace test3.txt abc def
This loads the whole file into memory, performs the replacements and saves it back to disk. Should probably not be used for massive files.
If you don't want to escape your string, you can reach your goal in 2 steps:
fgrep the line (getting the line number) you want to replace, and
afterwards use sed for replacing this line.
E.g.
#/bin/sh
PATTERN='foo*[)*abc' # we need it literal
LINENUMBER="$( fgrep -n "$PATTERN" "$FILE" | cut -d':' -f1 )"
NEWSTRING='my new string'
sed -i "${LINENUMBER}s/.*/$NEWSTRING/" "$FILE"
You can do this in two lines of bash code if you're OK with reading the whole file into memory. This is quite flexible -- the pattern and replacement can contain newlines to match across lines if needed. It also preserves any trailing newline or lack thereof, which a simple loop with read does not.
mapfile -d '' < file
printf '%s' "${MAPFILE//"$pat"/"$rep"}" > file
For completeness, if the file can contain null bytes (\0), we need to extend the above, and it becomes
mapfile -d '' < <(cat file; printf '\0')
last=${MAPFILE[-1]}; unset "MAPFILE[-1]"
printf '%s\0' "${MAPFILE[#]//"$pat"/"$rep"}" > file
printf '%s' "${last//"$pat"/"$rep"}" >> file
perl -i.orig -pse 'while (($i = index($_,$s)) >= 0) { substr($_,$i,length($s), $r)}'--\
-s='$_REQUEST['\'old\'']' -r='$_REQUEST['\'new\'']' sample.txt
-i.orig in-place modification with backup.
-p print lines from the input file by default
-s enable rudimentary parsing of command line arguments
-e run this script
index($_,$s) search for the $s string
substr($_,$i,length($s), $r) replace the string
while (($i = index($_,$s)) >= 0) repeat until
-- end of perl parameters
-s='$_REQUEST['\'old\'']', -r='$_REQUEST['\'new\'']' - set $s,$r
You still need to "escape" ' chars but the rest should be straight forward.
Note: this started as an answer to How to pass special character string to sed hence the $_REQUEST['old'] strings, however this question is a bit more appropriately formulated.
You should be using replace instead of sed.
From the man page:
The replace utility program changes strings in place in files or on the
standard input.
Invoke replace in one of the following ways:
shell> replace from to [from to] ... -- file_name [file_name] ...
shell> replace from to [from to] ... < file_name
from represents a string to look for and to represents its replacement.
There can be one or more pairs of strings.

How do I replace a token with the result of `pwd` in sed?

I'm trying to do something like this:
sed 's/#REPLACE-WITH-PATH/'`pwd`'/'
Unfortunately, I that errors out:
sed: -e expression #1, char 23: unknown option to `s'
Why does this happen?
You need to use a different character instead of /, eg.:
sed 's?#REPLACE-WITH-PATH?'`pwd`'?'
because / appears in the pwd output.
in sed, you can't use / directly, you must use '/'.
#!/bin/bash
dir=$`pwd`/
ls -1 | sed "s/^/${dir//\//\\/}/g"
sed 's:#REPLACE-WITH-PATH:'`pwd`':' config.ini
The problem is one of escaping the output of pwd correctly. Fortunately, as in vim, sed supports using a different delimiter character. In this case, using the colon instead of slash as a delimiter avoids the escaping problem.
instead of fumbling around with quotes like that, you can do it like this
#!/bin/bash
p=`pwd`
# pass the variable p to awk
awk -v p="$p" '$0~p{ gsub("REPLACE-WITH-PATH",p) }1' file >temp
mv temp file
or just bash
p=`pwd`
while read line
do
line=${line/REPLACE-WITH-PATH/$p}
echo $line
done < file > temp
mv temp file

Inserting text to a file with Sed within shell Script

I tried to insert a text to the first line
of a file using sed. I do this inside a sh
script.
But why it hangs at the line of sed execution?
#! /bin/sh
# Command to execute
# ./mybashcode.sh test.nbq
nbqfile=$1
nbqbase=$(basename $nbqfile nbq)
taglistfiletemp="${nbqbase}taglist_temp"
taglistfile="${nbqbase}taglist"
./myccode $nbqfile |
sort |
uniq -c |
awk '{print $2}' > $taglistfiletemp
noftags=$(wc -l $taglistfiletemp | awk '{print $1}')
echo $noftags
# We want to append output of noftags
# to the first line of taglistfile
sed '1i\
$noftags' > $taglistfile
# why it hangs here
# the content of taglistfile is NIL
I'm not sure what you are trying to do with sed but it needs two inputs, the script (usually a search/replace) and the data you want to perform it on. If you only specify one it assumes it has got the regular expression and waits for data on stdin. As you haven't supplied anything on stdin it'll hang indefinitely.
In addition, you have '$noftags' rather than "$noftags". The prior will output $noftags and the latter the contents of the variable, as single quotes do not allow variable expansion.
Have I got something wrong here?
Or, all you want to do is insert some text at the start of another file?
# $NewInitialText
# $fileToInsertInto
echo $NewInitialText > temp.file.txt
cat $fileToInsertInto >> temp.file.txt
mv temp.file.txt $fileToInsertInto
Is that easier done than sed? -- Pun intended I guess.
it hangs because you forget to supply sed with the input file.
....
...
sed -i.bak "1i $noftags" $taglistfile
...

Resources