I tried to insert a text to the first line
of a file using sed. I do this inside a sh
script.
But why it hangs at the line of sed execution?
#! /bin/sh
# Command to execute
# ./mybashcode.sh test.nbq
nbqfile=$1
nbqbase=$(basename $nbqfile nbq)
taglistfiletemp="${nbqbase}taglist_temp"
taglistfile="${nbqbase}taglist"
./myccode $nbqfile |
sort |
uniq -c |
awk '{print $2}' > $taglistfiletemp
noftags=$(wc -l $taglistfiletemp | awk '{print $1}')
echo $noftags
# We want to append output of noftags
# to the first line of taglistfile
sed '1i\
$noftags' > $taglistfile
# why it hangs here
# the content of taglistfile is NIL
I'm not sure what you are trying to do with sed but it needs two inputs, the script (usually a search/replace) and the data you want to perform it on. If you only specify one it assumes it has got the regular expression and waits for data on stdin. As you haven't supplied anything on stdin it'll hang indefinitely.
In addition, you have '$noftags' rather than "$noftags". The prior will output $noftags and the latter the contents of the variable, as single quotes do not allow variable expansion.
Have I got something wrong here?
Or, all you want to do is insert some text at the start of another file?
# $NewInitialText
# $fileToInsertInto
echo $NewInitialText > temp.file.txt
cat $fileToInsertInto >> temp.file.txt
mv temp.file.txt $fileToInsertInto
Is that easier done than sed? -- Pun intended I guess.
it hangs because you forget to supply sed with the input file.
....
...
sed -i.bak "1i $noftags" $taglistfile
...
Related
I'm trying to figure out a command to parse the following file content:
Operation=GET
Type=HOME
Counters=CacheHit=0,Exception=1,Validated=0
I need to extract Exception=1 into its own line. I'm fiddling with awk, sed and grep but not making much progress. Does anyone have any tips on using any unix command to perform this?
Thanks
Since your file is close to bash syntax, there is a fun little trick you can do to make bash itself parse the file. First, use some program like tr to transform the input into a something bash can parse, and then "source" that, which will create shell variables you can expand later to get the values.
source <(tr , $'\n' < file_name_goes_here)
echo $Exception
Many ways to do this. Here is one assuming the file is called "file.txt". Grab the line you want, replace everything from the start of the line up to Except with just Except, then pull out the first field using comma as the delimiter.
$ grep Exception file.txt | sed 's/.*Except/Except/g' | cut -d, -f 1
Exception=1
If you wanted to use gawk:
$ grep Exception file.txt | sed 's/.*Except/Except/g' | gawk -F, '{print $1}'
Exception=1
or just using grep and sed:
$ grep Exception file.txt | sed 's/.*\(Exception=[0-9]*\).*/\1/g'
Exception=1
or as #sheltter reminded me:
$ egrep -o "Exception=[0-9]+" file.txt
Exception=1
No need to use a mix of commands.
awk -F, 'NR==2 {print RS$1}' RS="Exception" file
Exception=1
Here we split the line by the keyword we look for RS="Exception"
If the line has two record (only when keyword is found), then
print first field, separated using command, with Record selector.
PS This only works if you have one Exception field
I have a sed command that is capturing a single line with sometext. The line in the file it is capturing ends with a linefeed. I am trying to utilize this variable in a pipeline, however, when I attempt to echo, or use it with other commands requiring an input, the result is a blank. Ex:
sed '1,1!d' somefile.txt | echo "$1", I know the variable itself is not empty as I can replace echo "$1" with cat $1 and see the correct printout.
edit - I have tried piping to a tr -d and removing the newline. I have confirmed the newline character is gone, yet echos still show blank. Cats do not.
edit 2 - I piped the variable into an if statement ... | if [[ -z $1 ]]; then cat $1; fi it hits the if, is determined to be empty, so runs the cat, which prints a non-empty line to console. If the variable is empty why is cat still printing out information?
What is causing this inconsistency and how can I solve my problem? The ultimate goal is to run the output of one sed, through another to replace specific lines in a target file.
sed '1,1!d' somefile.txt | sed '2,1s/.*/'$1'/' targetfile.txt
Contents of somefile.txt:
these
are
words
Contents of targetfile.txt:
The next line should say these
This line should say these
The previous line should say these
Output of echo after sed:
<empty>
Output of cat after sed:
these
Output of 2nd sed, using input from 1st:
The next line should say these
the previous line should say these
You are confused about arguments and input data. Look at this:
$ echo "$1"
$ echo "foo" | if [[ -z $1 ]]; then cat $1; fi
foo
The first argument to my shell, $1 is empty so if [[ -z $1 ]] succeeds. The reason that cat $1 produces output is that you have a fundamental shell programming error in that statement - you aren't quoting your variable, $1. The correct syntax isn't cat $1, it's cat "$1". Look at the difference:
$ echo "foo" | if [[ -z $1 ]]; then cat "$1"; fi
cat: '': No such file or directory
We can simplify the code to make what's happening clearer:
$ echo "foo" | cat $1
foo
$ echo "foo" | cat "$1"
cat: '': No such file or directory
The reason that echo "foo" | cat $1 produces output is that the unquoted $1 is expanded by the shell to nothing before cat is called so that statement is equivalent to just echo "foo" | cat and so cat just copies the input coming in from the pipe to it's output.
On the other hand echo "foo" | cat "$1" generates an error because the shell expands "$1" to the null string before cat is called and so it's then asking cat to open a file named <null> and that of course does not exist, hence the error.
Always quote your shell variables unless you have a specific reason not to and fully understand all of the implications. Read a shell man page and/or google that if you're not sure what those implications are.
wrt another part of your code you have:
sed '1,1!d' somefile.txt | echo "$1"
but, unlike cat, echo neither reads it's input from a pipe nor from a file name passed as an argument. The input to echo is just the list of string arguments you provide it so while echo "foo" | cat will cause cat to read the input stream containing foo and output it, echo "foo" | echo will produce no output because echo isn't designed to read input from a pipe and so it'll just print a null string since you gave it no arguments.
It's not clear what you're really trying to accomplish but I think you might want to replace the 2nd line of targetfile.txt with the first line of somefile.txt. If so that's just:
awk '
NR==FNR { if (NR==1) new=$0; next }
FNR==2 { $0 = new }
{ print }
' somefile.txt targetfile.txt
Do not try to use sed to do it or you'll find yourself in escaping/quoting hell because, unlike awk, sed does not understand literal strings, see Is it possible to escape regex metacharacters reliably with sed.
You appear to want to extract the first line from file1 and use it to replace the second line in file2.
At the moment, you are extracting that value from the first file with your first sed but sending it to the second sed on its stdin rather than as a parameter ($1).
Your description is confusing so I will use this as file1:
File 1 Line 1
File 1 Line 2
File 1 Line 3
And this as file2:
File 2 Line 1
File 2 Line 2
File 2 Line 3
There are many ways to do this.
Method 1
# Extract line1 from file1
extracted=$(sed '1!d' file1)
# Replace line 2 in file2 with extracted value
sed "2s/.*/$extracted/" file2
Not sure why I feel like a dentist now :-)
If you want to put it all on one line, as some folks like to do:
x=$(sed '1!d' file1) && sed "2s/.*/$x/" file2
Method 2
This one is a bit tricky. It uses the first sed to write a script for the second sed:
sed 's/^/2s|.*|/;s/$/|/;q' file1 | sed -f /dev/stdin file2
If you look at the first sed on its own you will see it is generating a script for the second one:
sed 's/^/2s|.*|/;s/$/|/;q' file1
2s|.*|File 1 Line 1|
If you look at the second sed, you will see it is executing a script passed on its standard input:
sed -f /dev/stdin ...
Method 3
Easier still is awk:
awk 'FNR==NR{if(NR==1)saved=$0;next} FNR==2{$0=saved}1' file1 file2
File 2 Line 1
File 1 Line 1
File 2 Line 3
All you need to notice is that I am passing 2 files to awk and that FNR==NR inside the script means that awk is currently processing the first file, because FNR is the line number in the current file and NR is the total number of lines awk has processed from all files so far. So, when processing the second file, NR is greater than FNR by the number of lines in the first file.
I'm trying to write a script to swap out text in a file:
sed s/foo/bar/g myFile.txt > myFile.txt.updated
mv myFile.txt.updated myFile.txt
I evoke the sed program, which swaps out text in myFile.txt and redirects the changed lines of text to a second file. mv then moves .updated txt file to myFile.txt, overwriting it. That command works in the shell.
I wrote:
#!/bin/sh
#First, I set up some descriptive variables for the arguments
initialString="$1"
shift
desiredChange="$1"
shift
document="$1"
#Then, I evoke sed on these (more readable) parameters
updatedDocument=`sed s/$initialString/$desiredChange/g $document`
#I want to make sure that was done properly
echo updated document is $updatedDocument
#then I move the output in to the new text document
mv $updatedDocument $document
I get the error:
mv: target `myFile.txt' is not a directory
I understand that it thinks my new file's name is the first word of the string that was sed's output. I don't know how to correct that. I've been trying since 7am and every quotation, creating a temporary file to store the output in (disastrous results), IFS...everything so far gives me more and more unhelpful errors. I need to clear my head and I need your help. How can I fix this?
Maybe try
echo $updatedDocument > $document
Change
updatedDocument=`sed s/$initialString/$desiredChange/g $document`
to
updatedDocument=${document}.txt
sed s/$initialString/$desiredChange/g $document
Backticks will actually put the entire piped output of the sed command into your variable value.
An even faster way would be to not use updatedDocument or mv at all by doing an in-place sed:
sed -i s/$initialString/$desiredChange/g $document
The -i flag tells sed to do the replacement in-place. This basically means creating a temp file for the output and replacing your original file with the temp file once it is done, pretty much exactly as you are doing.
#!/bin/sh
#First, I set up some descriptive variables for the arguments
echo "$1" | sed #translation of special regex char like . * \ / ? | read -r initialString
echo "$2" | sed 's|[\&/]|\\&|g' | read -r desiredChange
document="$3"
#Then, I evoke sed
sed "s/${initialString}/${desiredChange}/g" ${document} | tee ${document}
don't forget that initialString and desiredChange are pattern interpreted as regex, so a trnaslation is certainly needed
sed #translation of special regex char like . * \ / ? is to replace by the correct sed (discuss on several post on the site)
Im trying to get the following script to work, but Im having some issues:
g++ -g -c $1
DWARF=echo $1 | sed -e `s/(^.+)\.cpp$/\1/`
and Im getting -
./dcompile: line 3: test3.cpp: command not found
./dcompile: command substitution: line 3: syntax error near unexpected token `^.+'
./dcompile: command substitution: line 3: `s/(^.+)\.cpp$/\1/'
sed: option requires an argument -- 'e'
and then bunch of stuff on sed usage. What I want to do is pass in a cpp file and then extract the file name without the .cpp and put it into the variable DWARF. I would also like to later use the variable DWARF to do the following -
readelf --debug-dump=info $DWARF+".o" > $DWARF+".txt"
But Im not sure how to actually do on the fly string concats, so please help with both those issues.
You actually need to execute the command:
DWARF=$(echo $1 | sed -e 's/(^.+)\.cpp$/\1/')
The error message is a shell error because your original statement
DWARF=echo $1 | sed -e `s/(^.+)\.cpp$/\1/`
is actually parsed like this
run s/(^.+)\.cpp$/\1/
set DWARF=echo
run the command $1 | ...
So when it says test3.cpp: command not found I assume that you are running with argument test3.cpp and it's literally trying to execute that file
You also need to wrap the sed script in single quotes, not backticks
In BASH you can crop off the extension from $1 by
${1%*.cpp}
if you need to set the DWARF var use
DWARF="${1%*.cpp}"
or just reference $1 as
readelf --debug-dump=info "${1%*.cpp}.o" > "${1%*.cpp}.txt"
which will chop off the rightmost .cpp so test.cpp.cpp will be test.cpp
You can use awk for this:
$ var="testing.cpp"
$ DWARF=$(awk -F. '{print $1}' <<< $var)
$ echo "$DWARF"
testing
I am having a log file a.log and i need to extract a piece of information from it.
To locate the start and end line numbers of the pattern i am using the following.
start=$(sed -n '/1112/=' file9 | head -1)
end=$(sed -n '/true/=' file9 | head -1)
i need to use the variables (start,end) in the following command:
sed -n '16q;12,15p' orig-data-file > new-file
so that the above command appears something like:
sed -n '($end+1)q;$start,$end'p orig-data-file > new-file
I am unable to replace the line numbers with the variables. Please suggest the correct syntax.
Thanks,
Rosy
When I realized how to do it, I was looking for anyway to get line number into a file containing the requested info, and display the file from that line to EOF.
So, this was my way.
with
PATTERN="pattern"
INPUT_FILE="file1"
OUTPUT_FILE="file2"
line number of first match of $PATTERN into $INPUT_FILE can be retrieved with
LINE=`grep -n ${PATTERN} ${INPUT_FILE} | awk -F':' '{ print $1 }' | head -n 1`
and the outfile will be the text from that $LINE to EOF. This way:
sed -n ${LINE},\$p ${INPUT_FILE} > ${OUTPUT_FILE}
The point here, is the way how can variables be used with command sed -n:
first witout using variables
sed -n 'N,$p' <file name>
using variables
LINE=<N>; sed -n ${LINE},\$p <file name>
Remove the single quotes thus. Single quotes turn off the shell parsing of the string. You need shell parsing to do the variable string replacements.
sed -n '('$end'+1)q;'$start','$end''p orig-data-file > new-file