using sed to replace with wildcards - unix
I am trying to replace this text (whatever the xxxx) :
{"type":"record","name":"xxxx","namespace":"example","fields":[{"name":"name","type":"string"}, {"name":"favorite_color","type":["string","null"]}]}
with this one:
{"type":"record","name":"yyyy","namespace":"example","fields":[{"name":"name","type":"string"},{"name":"favorite_color","type":["string","null"]}]}
I am using sed, but with this command:
sed 's/\"name\"\:\".*\"/\"name\"\:\"yyyy\"/' file.json
I am getting:
{"type":"record", "name":"yyyy"]}]}
because it replaces all the content until the last double quote.
How could I use the wildcards to replace only the text until the next double quote?
You can try with this:
sed 's/\"name\":\"[^\"]*\"/\"name\":\"yyyy\"/' file.json
Hence, modify the regular pattern so it looks until the next double quotes instead of any character.
If you want to write the changes to the file, you'll need -i flag, in addition to -r flag to use regular expressions. And if you want this to be applied to multiple occurrences, you can add a g to the sed command:
sed -i -r 's/\"name\":\"[^\"]*\"/\"name\":\"yyyy\"/g' file.json
Update: not needed to escape double quotes... The command could be just this:
sed -i -r 's/"name":"[^"]*"/"name":"yyyy"/g' file.json
Related
Replace Text using variables in Sed not working
I need to modify an xml file using Sed to replace the line url="jdbc:oracle:thin:#//ttpdbscan.axel.net:1521/axel.telco.net" with url="jdbc:oracle:thin:#//ttpdbscan.axeltelecom.net:1598/axelPRD.telco.net" I have stored the lines like this ACTUAL_DB=$(sed -n 's#^.*url="\(.*\).*"#\1#p' $FILE.xml) and NEW_DB="jdbc:oracle:thin:#//ttpdbscan.axeltelecom.net:1598/axelPRD.telco.net" And the replacing method is this one sed -i "s#$ACTUAL_DB#$NEW_DB#g" $File.xml The problem is that when I run the script the file stays the same. I have echoed the variables and all of them return the correct values.
Assuming the file you have is File.xml (if it is not a variable), you may use
sed -i "s#${ACTUAL_DB}#${NEW_DB}#g" File.xml
Try also with other delimiters:
sed -i "s~${ACTUAL_DB}~${NEW_DB}~g" File.xml
If your sed does not support -i use
sed "s~${ACTUAL_DB}~${NEW_DB}~g" File.xml 1<> File.xml
See sed edit file in place
So I saved the output into another file and found out that the string had an extra space
so it looked like this
ACTUAL_DB= "jdbc:oracle:thin:#//ttpdbscan.axel.net:1521/axel.telco.net "
I removed the extra space with
"$(echo -e "${ACTUAL_DB}" | tr -d '[:space:]')"
And now the sed is working as intended
Delete Special Word Using sed
I would like to use sed to remove all occurances of this line if and only if it is this <ab></ab> If this line, I would not want to delete it <ab>keyword</ab> My attempt that's not working: sed '/<ab></ab>/d' Thanks for any insight. I'm not sure what's wrong as I should not have to escape anything? I'm using a shell script named temp to execute this. My command is this: cat foobar.html | ./temp This is my temp shell script: #!/bin/sh sed -e '/td/!d' | sed '/<ab></ab>/d'
It looks like we have a couple of problems here. The first is with the / in the close-tag. sed uses this to delimit different parts of the command. Fortunately, all we have to do is escape it with \. Try: sed '/<ab><\/ab>/d' Here's an example on my machine: $ cat test <ab></ab> <ab></ab> <ab>test</ab> $ sed '/<ab><\/ab>/d' test <ab>test</ab> $ The other problem is that I'm not sure what the purpose of sed -e '/td/!d' is. In it's default operating mode, you don't need to tell it not to delete something; just tell it exactly what you want to delete. So, to do this on a file called input.html: sed '/<ab><\/ab>/d' input.html Or, to edit the file in-place, you can just do: sed -i -e '/<ab><\/ab>/d' input.html Additionally, sed lets you use any character you want as a delimiter; you don't have to use /. So if you'd prefer not to escape your input, you can do: sed '\#<ab></ab>#d' input.html Edit In the comments, you mentioned wanting to delete lines that only contain </ab> and nothing else. To do that, you need to do what's called anchoring the match. The ^ character represents the beginning of the line for anchoring, and $ represents the end of the line. sed '/^<\/ab>$/d' input.html This will only match a line that contains (literally) </ab> and nothing else at all, and delete the line. If you want to match lines that contain whitespace too, but no text other than </ab>: sed '/^[[:blank:]]*<\/ab>[[:blank:]]*$/d' input.html [[:blank:]]* matches "0 or more whitespace characters" and is called a "POSIX bracket expression".
Sed replace only exact match
I wan't to replace a string like Europe12 with Europe12_yesturday in a file. Without changing the Europe12-36 strings that also exists in the file.
I tried:
$basename=Europe12
sed -i 's/\b$basename\b/${basename}_yesterday/g' file.txt
but this also changed the Europe12-36 strings.
Require a space or end of line character: sed 's/Europe12\([ ]|$\)/Europe12_yesturday\1/g' input
Manually construct the delimiter list you want instead of using \b, \W or \<. - is not part of the word characters (alphanumericals), so that's why this also matches your other string. So try something like this, expanding the list as needed: [-a-zA-Z0-9].
You can do it in 2 times: sed -e 's/Europe12/Europe12_yesturday/g' -e 's/Europe12_yesturday-36/Europe12-36/g' file.txt
sed 's/\(Europe12[[:blank:]]\)/\1_yesturday/g;s/Europe12$/&_yesturday/' YourFile [[:blank:]] could be completeted with any boundary you accept also like .,;:/]) etc (be carrefull of regex meaning of this char in this case)
It is little late to reply..
It can be achieved easily by "word boundary" notation (\<..\>)
sed -i 's/\<$basename\>/${basename}_yesterday/g' file.txt
How to remove blank lines from a Unix file
I need to remove all the blank lines from an input file and write into an output file. Here is my data as below. 11216,33,1032747,64310,1,0,0,1.878,0,0,0,1,1,1.087,5,1,1,18-JAN-13,000603221321 11216,33,1033196,31300,1,0,0,1.5391,0,0,0,1,1,1.054,5,1,1,18-JAN-13,059762153003 11216,33,1033246,31300,1,0,0,1.5391,0,0,0,1,1,1.054,5,1,1,18-JAN-13,000603211032 11216,33,1033280,31118,1,0,0,1.5513,0,0,0,1,1,1.115,5,1,1,18-JAN-13,055111034001 11216,33,1033287,31118,1,0,0,1.5513,0,0,0,1,1,1.115,5,1,1,18-JAN-13,000378689701 11216,33,1033358,31118,1,0,0,1.5513,0,0,0,1,1,1.115,5,1,1,18-JAN-13,000093737301 11216,33,1035476,37340,1,0,0,1.7046,0,0,0,1,1,1.123,5,1,1,18-JAN-13,045802041926 11216,33,1035476,37340,1,0,0,1.7046,0,0,0,1,1,1.123,5,1,1,18-JAN-13,045802041954 11216,33,1035476,37340,1,0,0,1.7046,0,0,0,1,1,1.123,5,1,1,18-JAN-13,045802049326 11216,33,1035476,37340,1,0,0,1.7046,0,0,0,1,1,1.123,5,1,1,18-JAN-13,045802049383 11216,33,1036985,15151,1,0,0,1.4436,0,0,0,1,1,1.065,5,1,1,18-JAN-13,000093415580 11216,33,1037003,15151,1,0,0,1.4436,0,0,0,1,1,1.065,5,1,1,18-JAN-13,000781202001 11216,33,1037003,15151,1,0,0,1.4436,0,0,0,1,1,1.065,5,1,1,18-JAN-13,000781261305 11216,33,1037003,15151,1,0,0,1.4436,0,0,0,1,1,1.065,5,1,1,18-JAN-13,000781603955 11216,33,1037003,15151,1,0,0,1.4436,0,0,0,1,1,1.065,5,1,1,18-JAN-13,000781615746
sed -i '/^$/d' foo This tells sed to delete every line matching the regex ^$ i.e. every empty line. The -i flag edits the file in-place, if your sed doesn't support that you can write the output to a temporary file and replace the original: sed '/^$/d' foo > foo.tmp mv foo.tmp foo If you also want to remove lines consisting only of whitespace (not just empty lines) then use: sed -i '/^[[:space:]]*$/d' foo Edit: also remove whitespace at the end of lines, because apparently you've decided you need that too: sed -i '/^[[:space:]]*$/d;s/[[:space:]]*$//' foo
awk 'NF' filename awk 'NF > 0' filename sed -i '/^$/d' filename awk '!/^$/' filename awk '/./' filename The NF also removes lines containing only blanks or tabs, the regex /^$/ does not.
Use grep to match any line that has nothing between the start anchor (^) and the end anchor ($): grep -v '^$' infile.txt > outfile.txt If you want to remove lines with only whitespace, you can still use grep. I am using Perl regular expressions in this example, but here are other ways: grep -P -v '^\s*$' infile.txt > outfile.txt or, without Perl regular expressions: grep -v '^[[:space:]]*$' infile.txt > outfile.txt
sed -e '/^ *$/d' input > output Deletes all lines which consist only of blanks (or is completely empty). You can change the blank to [ \t] where the \t is a representation for tab. Whether your shell or your sed will do the expansion varies, but you can probably type the tab character directly. And if you're using GNU or BSD sed, you can do the edit in-place, if that's what you want, with the -i option. If I execute the above command still I have blank lines in my output file. What could be the reason? There could be several reasons. It might be that you don't have blank lines but you have lots of spaces at the end of a line so it looks like you have blank lines when you cat the file to the screen. If that's the problem, then: sed -e 's/ *$//' -e '/^ *$/d' input > output The new regex removes repeated blanks at the end of the line; see previous discussion for blanks or tabs. Another possibility is that your data file came from Windows and has CRLF line endings. Unix sees the carriage return at the end of the line; it isn't a blank, so the line is not removed. There are multiple ways to deal with that. A reliable one is tr to delete (-d) character code octal 15, aka control-M or \r or carriage return: tr -d '\015' < input | sed -e 's/ *$//' -e '/^ *$/d' > output If neither of those works, then you need to show a hex dump or octal dump (od -c) of the first two lines of the file, so we can see what we're up against: head -n 2 input | od -c Judging from the comments that sed -i does not work for you, you are not working on Linux or Mac OS X or BSD — which platform are you working on? (AIX, Solaris, HP-UX spring to mind as relatively plausible possibilities, but there are plenty of other less plausible ones too.) You can try the POSIX named character classes such as sed -e '/^[[:space:]]*$/d'; it will probably work, but is not guaranteed. You can try it with: echo "Hello World" | sed 's/[[:space:]][[:space:]]*/ /' If it works, there'll be three spaces between the 'Hello' and the 'World'. If not, you'll probably get an error from sed. That might save you grief over getting tabs typed on the command line.
grep . file grep looks at your file line-by-line; the dot . matches anything except a newline character. The output from grep is therefore all the lines that consist of something other than a single newline.
with awk awk 'NF > 0' filename
To be thorough and remove lines even if they include spaces or tabs something like this in perl will do it: cat file.txt | perl -lane "print if /\S/" Of course there are the awk and sed equivalents. Best not to assume the lines are totally blank as ^$ would do. Cheers
You can sed's -i option to edit in-place without using temporary file: sed -i '/^$/d' file
How do I replace a token with the result of `pwd` in sed?
I'm trying to do something like this: sed 's/#REPLACE-WITH-PATH/'`pwd`'/' Unfortunately, I that errors out: sed: -e expression #1, char 23: unknown option to `s' Why does this happen?
You need to use a different character instead of /, eg.: sed 's?#REPLACE-WITH-PATH?'`pwd`'?' because / appears in the pwd output.
in sed, you can't use / directly, you must use '/'.
#!/bin/bash
dir=$`pwd`/
ls -1 | sed "s/^/${dir//\//\\/}/g"
sed 's:#REPLACE-WITH-PATH:'`pwd`':' config.ini The problem is one of escaping the output of pwd correctly. Fortunately, as in vim, sed supports using a different delimiter character. In this case, using the colon instead of slash as a delimiter avoids the escaping problem.
instead of fumbling around with quotes like that, you can do it like this
#!/bin/bash
p=`pwd`
# pass the variable p to awk
awk -v p="$p" '$0~p{ gsub("REPLACE-WITH-PATH",p) }1' file >temp
mv temp file
or just bash
p=`pwd`
while read line
do
line=${line/REPLACE-WITH-PATH/$p}
echo $line
done < file > temp
mv temp file