Surround every line with single quote except empty lines - unix

My goal is to add a single apostrophe to every line in the file and skip empty lines.
file.txt:
Quote1
Quote2
Quote3
So far I have used sed:
sed -e "s/\(.*\)/'\1'/"
Which does the job but creates apostrophes also in empty lines:
'Quote1'
'Quote2'
''
'Quote3'
My goal:
'Quote1'
'Quote2'
'Quote3'
How could I achieve this by using sed, or maybe it should it be awk.

.* means zero or more characters, you want 1 or more characters which in any sed would be ..*:
$ sed "s/..*/'&'/" file
'Quote1'
'Quote2'
'Quote3'
You can also write that regexp as .\+ in GNU sed, .\{1,\} in POSIX seds, and .+ in GNU or OSX/BSD sed when invoked with -E.
The above assumes lines of all blanks should be quoted. If that's wrong then:
$ sed "s/.*[^[:blank:]].*/'&'/" file
'Quote1'
'Quote2'
'Quote3'
In any awk assuming lines of all blanks should be quoted:
$ awk '/./{$0="\047" $0 "\047"}1' file
'Quote1'
'Quote2'
'Quote3'
otherwise:
$ awk 'NF{$0="\047" $0 "\047"}1' file
'Quote1'
'Quote2'
'Quote3'
You can see the difference between the above with this:
$ printf ' \n' | sed "s/..*/'&'/"
' '
$ printf ' \n' | sed "s/.*[^[:blank:]].*/'&'/"
$ printf ' \n' | awk '/./{$0="\047" $0 "\047"}1'
' '
$ printf ' \n' | awk 'NF{$0="\047" $0 "\047"}1'
$

One way:
awk '$1{$0 = q $0 q}1' q="'" file
Add quotes only if 1st column($1) has some value. 1 to print every line.

Assuming you want to add the single quotes to lines that contain nothing but whitespace:
sed -E "/./s/(.*)/'\1'/"

Another sed
sed '/^$/!{s/^/\x27/;s/$/\x27/}' file
The above script says
Look for an empty line - /^$/ - pattern.
For those lines that doesn't follow the above pattern(!), substitute start(^) and end($) with single quotes(\x27).

You can use perl to check for a negative lookbehind, asserting that you won't match an "empty" line:
perl -pe 's/(?<!$)(.*)/"\1"/' file
Another alternative is being more specific in your regex, as #edmorton suggested in his answer.

tried on gnu sed
sed -E "s/\S+/'&'/" file.txt

Related

arrange a file with sed

Okay I have a file which look like this :
>S000632122
Bacteria;domain;"Actinobacteria";Actinobacteria;Acidimicrobidae;Acidimicrobiales;order;"Acidimicrobineae";Acidimicrobiaceae;Acidimicrobium;
>S000632121
Bacteria;domain;"Actinobacteria";Actinobacteria;Acidimicrobidae;Acidimicrobiales;order;"Acidimicrobineae";Acidimicrobiaceae;Acidimicrobium;
>S000541758
Bacteria;domain;"Actinobacteria";Actinobacteria;Acidimicrobidae;Acidimicrobiales;order;"Acidimicrobineae";Acidimicrobiaceae;Acidimicrobium;
But I want something like this
>S000632122\tBacteria; domain; actinobacteria\n
>S000548245\tBacteria; domain; actinobacteria\n
I tried with sed but I'm a bit lost...
Here is what I've done :
sed ':a;N;$!ba;s/\n/\t/g' my file
But it returns me that:
>S000632122 \tBacteria;domain;"Actinobacteria";Actinobacteria;Acidimicrobidae;Acidimicrobiales;order;"Acidimicrobineae";Acidimicrobiaceae;Acidimicrobium;\t>S000632121\tBacteria;domain;"Actinobacteria";Actinobacteria;Acidimicrobidae;Acidimicrobiales;order;"Acidimicrobineae";Acidimicrobiaceae;Acidimicrobium;\t>S000541758
Thank you in advance for your help
This sort of thing is almost always simpler with awk:
$ cat input
>S000632122
Bacteria;domain;"Actinobacteria";Actinobacteria;Acidimicrobidae;Acidimicrobiales;order;"Acidimicrobineae";Acidimicrobiaceae;Acidimicrobium;
>S000632121
Bacteria;domain;"Actinobacteria";Actinobacteria;Acidimicrobidae;Acidimicrobiales;order;"Acidimicrobineae";Acidimicrobiaceae;Acidimicrobium;
>S000541758
Bacteria;domain;"Actinobacteria";Actinobacteria;Acidimicrobidae;Acidimicrobiales;order;"Acidimicrobineae";Acidimicrobiaceae;Acidimicrobium;
$ awk '/^>S[0-9]*$/{ printf "%s\t", $0; next} {printf "%s; %s; %s\n", $1, $2, tolower($4)}' FS=\; input
>S000632122 Bacteria; domain; actinobacteria
>S000632121 Bacteria; domain; actinobacteria
>S000541758 Bacteria; domain; actinobacteria
It's not clear to me from the question if you actually want literal text \t and \n in the output. If you do, you could do:
$ awk '/^>S[0-9]*$/{ printf "%s\\t", $0; next} {printf "%s; %s; %s\\n\n", $1, $2, tolower($4)}' FS=\; input
>S000632122\tBacteria; domain; actinobacteria\n
>S000632121\tBacteria; domain; actinobacteria\n
>S000541758\tBacteria; domain; actinobacteria\n
In each of these, the first clause matches the regex ^>S[0-9]*$ and prints those lines with a trailing tab. (Removing the newline effectively joins the next line in the output.) Every other line is printed according to the format string.
This might work for you (GNU sed):
sed -E 'N;s/\n(([^;]*;){3}).*/\t\L\1/;s/;/\n/3;s//& /g;s/"//g' file
Or if the tab and newline are literal:
sed -E 'N;s/\n(([^;]*;){3}).*/\\t\L\1/;s/;/\\n/3;s//& /g;s/"//g' file
Append the following line.
Replace the newline by a tab and remove all but the first 3 fields of the second line (also lowercase the second line at the same time).
Replace the 3rd ; by a newline.
Put a space after all remaining ;'s.
Remove any "'s.

using sed or awk to double quote comma separate and concatenate a list

I have the following list in a text file:
10.1.2.200
10.1.2.201
10.1.2.202
10.1.2.203
I want to encase in "double quotes", comma separate and join the values as one string.
Can this be done in sed or awk?
Expected output:
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203","10.1.2.204"
The easiest is something like this (in pseudo code):
Read a line;
Put the line in quotes;
Keep that quoted line in a stack or string;
At the end (or while constructing the string), join the lines together with a comma.
Depending on the language, that is fairly straightforward to do:
With awk:
$ awk 'BEGIN{OFS=","}{s=s ? s OFS "\"" $1 "\"" : "\"" $1 "\""} END{print s}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
Or, less 'wall of quotes' to define a quote character:
$ awk 'BEGIN{OFS=",";q="\""}{s=s ? s OFS q$1q : q$1q} END{print s}' file
With sed:
$ sed -E 's/^(.*)$/"\1"/' file | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g'
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
(With Perl and Ruby, with a join function, it is easiest to push the elements onto a stack and then join that.)
Perl:
$ perl -lne 'push #a, "\"$_\""; END{print join(",", #a)}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
Ruby:
$ ruby -ne 'BEGIN{#arr=[]}; #arr.push "\"#{$_.chomp}\""; END{puts #arr.join(",")}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
here is another alternative
sed 's/.*/"&"/' file | paste -sd,
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
awk -F'\n' -v RS="\0" -v OFS='","' -v q='"' '{NF--}$0=q$0q' file
should work for given example.
Tested with gawk:
kent$ cat f
10.1.2.200
10.1.2.201
10.1.2.202
10.1.2.203
kent$ awk -F'\n' -v RS="\0" -v OFS='","' -v q='"' '{NF--}$0=q$0q' f
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
$ awk '{o=o (NR>1?",":"") "\""$0"\""} END{print o}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"

Update value within a unix flat file

Kindly help. I want to make .0 to be 0.0 within a UNIX file yyyyy.csv :
603905209;47.824;USD
603905477;57.199;USD
603938657;3.2281;USD
603949388;.00191;USD
603937274;.00563;USD
603911160;.00287;USD
I want the result to be
603905209;47.824;USD
603905477;57.199;USD
603938657;3.2281;USD
603949388;0.00191;USD
603937274;0.00563;USD
603911160;0.00287;USD
but I got this result:
603905209;0.4.7824;USD
603905477;0.5.7199;USD
603938657;0.3.2281;USD
603949388;0.00191;USD
603937274;0.00563;USD
603911160;0.00287;USD
Below is my command:
sed 's/;.0/;0.0/g' yyyyy.csv | sed 's/;.2/;0.2/g' | sed 's/;.1/;0.1/g' | sed 's/;.3/;0.3/g' | sed 's/;.4/;0.4/g' | sed 's/;.5/;0.5/g'| sed 's/;.6/;0.6/g' | sed 's/;.7/;0.7/g' | sed 's/;.8/;0.8/g' | sed 's/;.9/;0.9/g' > xxxxx.csv
You need to escape all the dots present in your regex or otherwise it would match any character. That is, . is a special meta character in regex which matches any character. To match a literal dot, you need to escape the ..
sed 's/;\.0/;0.0/g' yyyyy.csv
And this would be enough.
$ sed 's/;\.\([0-9]\)/;0.\1/g' file
603905477;57.199;USD
603938657;3.2281;USD
603949388;0.00191;USD
603937274;0.00563;USD
603911160;0.00287;USD
In basic sed, \(...\) called capturing group, which is used to capture the characters matched by the pattern present inside that group. So the pattern present inside the group is [0-9] which matches a digit from 0-9. We could refer the captured characters through back-referencing ie, \1. \1 at the replacement part refers to the characters which are present inside the group index 1.
Change your command to:
sed 's/;\./;0./g' File
i.e, just substitute ;. with ;0..
Just print your file using a tool that understands numbers:
$ awk -F';' '{printf "%d;%f;%s\n", $1,$2,$3}' file
603905209;47.824000;USD
603905477;57.199000;USD
603938657;3.228100;USD
603949388;0.001910;USD
603937274;0.005630;USD
603911160;0.002870;USD
awk -F';' '{printf "%d;%.5f;%s\n", $1,$2,$3}' file
603905209;47.82400;USD
603905477;57.19900;USD
603938657;3.22810;USD
603949388;0.00191;USD
603937274;0.00563;USD
603911160;0.00287;USD
$ awk -F';' '{printf "%d;%.2f;%s\n", $1,$2,$3}' file
603905209;47.82;USD
603905477;57.20;USD
603938657;3.23;USD
603949388;0.00;USD
603937274;0.01;USD
603911160;0.00;USD
Whatever precision you like...

Unix- Sed replacing substring

I am new to sed . I want to replace a substring
for example:
var1=server1:game1,server2:game2,sever3:game1
output should be
server1 server2 server3 (with just spaces)
I have tried this.
echo $var1 | sed 's/,/ /g' | sed 's/:* / /g'
This is not working. Please suggest a solution.
You can try this sed,
echo $var1 | sed 's/:[^,]\+,\?/ /g'
Explanation:
:[^,]\+, - It will match the string from : to ,
\? - Previous may occur or may not ( Since end of line don't have , )
echo $var1 | sed s/:game[0-9],*/\ /
Assuming your sub string has game followed by a number([0-9]*)
An awk variation using same regex as sed
awk '{gsub(/:[^,]+,?/," ")}1' <<< "$var1"
PS Its always good custom to "quote" variables
Just for info, you are really only matching, not replacing, so grep can be your friend (with -P):
grep -oP '[^:,=]+(?=:)'
That matches a number of characters that aren't :,= followed by a : using lookahead.
This will put the servers on different lines, which may be what you want anyway. You can put them on one line by adding tr:
grep -oP '[^:,=]+(?=:)' | tr '\n' ' '

how to search line using sed and replace string at the end of the line

My Requirement is to search line in a file which should have pattern #$SQL_DIRand add string $IDSS_LA_SCHEMA at the end of the line. Can any one help me.. ??
Regards,
Rajesh
Use awk, awk '/PATTERN/ {print $0, "STRING"}' /path/to/your/file
Maybe like this:
sed '/#\$SQL_DIR/ s/$/\$IDSS_LA_SCHEMA/'
What I've tried to do there is match lines containing your specified string, and for each one of those, replace the ending with the second string.
$ cat myfile.txt
some line
some other line, with #$SQL_DIR in it
$ sed -i 's/^\(.*#$SQL_DIR.*\)/\1$IDSS_LA_SCHEMA/' myfile.txt
$ cat myfile.txt
some line
some other line, with #$SQL_DIR in it$IDSS_LA_SCHEMA
> cat temp
ngkerng
egerger
erger
ghethe #$SQL_DIRand
jgnv
erger
>
see below:
> perl -pi -e 'if(/\#\$SQL_DIRand/){$_=~s/$/\$IDSS_LA_SCHEMA/;}' temp
>cat temp
ngkerng
egerger
erger
ghethe #$SQL_DIRand$IDSS_LA_SCHEMA
jgnv
erger
>
Note: This will do an inplace appending.

Resources