Need you help on the below query.
$cat sample.txt
414d51204d5150525447575648415050529a6af298e,,,002330618820140707134048 ,+0657,45675
414d51204d514e55533732355051543051696344201645c6,002330618820140707134048 ,+0657,tgre,ghty
Need all fields after last comma "," in field 1.As you can see there are lots of (414d51204d5150525447575648415050529a6af298e,,,)
The output has to be like as below.
Expected Output:
002330618820140707134048 ,+0657,45675
002330618820140707134048 ,+0657,tgre,ghty
Thanks
In case the others do not work here is a perl oneliner:
perl -lne 'print $1 if /.*?,+(.*)/'
Output:
002330618820140707134048 ,+0657,45675
002330618820140707134048 ,+0657,tgre,ghty
You can use the shell program .
cat sample.txt | sed -n 1'p' | tr ',' '\n' | while read word; do
echo $word
done
You can use sed and cut :
sed to remove empty fields
cut to select the required fields
Example:
sed -r 's/,+/,/g' < text.txt | cut -d',' -f2-
Output:
002330618820140707134048 ,+0657,45675
002330618820140707134048 ,+0657,tgre,ghty
Using egrep or grep -E:
egrep -o '[0-9]{24}\s+\S+' your_file
Using normal grep:
grep -o '[0-9]\{24\}[[:blank:]]\+[^[:blank:]]\+' your_file
Output:
002330618820140707134048 ,+0657,45675
002330618820140707134048 ,+0657,tgre,ghty
Or using sed:
sed -e 's/^[^[:blank:]]\+,//' your_file
Or
sed -e 's/^[^ \t]\+,//' your_file
Or just about space:
sed -e 's/^[^ ]\+,//' your_file
Related
I have the following list in a text file:
10.1.2.200
10.1.2.201
10.1.2.202
10.1.2.203
I want to encase in "double quotes", comma separate and join the values as one string.
Can this be done in sed or awk?
Expected output:
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203","10.1.2.204"
The easiest is something like this (in pseudo code):
Read a line;
Put the line in quotes;
Keep that quoted line in a stack or string;
At the end (or while constructing the string), join the lines together with a comma.
Depending on the language, that is fairly straightforward to do:
With awk:
$ awk 'BEGIN{OFS=","}{s=s ? s OFS "\"" $1 "\"" : "\"" $1 "\""} END{print s}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
Or, less 'wall of quotes' to define a quote character:
$ awk 'BEGIN{OFS=",";q="\""}{s=s ? s OFS q$1q : q$1q} END{print s}' file
With sed:
$ sed -E 's/^(.*)$/"\1"/' file | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g'
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
(With Perl and Ruby, with a join function, it is easiest to push the elements onto a stack and then join that.)
Perl:
$ perl -lne 'push #a, "\"$_\""; END{print join(",", #a)}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
Ruby:
$ ruby -ne 'BEGIN{#arr=[]}; #arr.push "\"#{$_.chomp}\""; END{puts #arr.join(",")}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
here is another alternative
sed 's/.*/"&"/' file | paste -sd,
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
awk -F'\n' -v RS="\0" -v OFS='","' -v q='"' '{NF--}$0=q$0q' file
should work for given example.
Tested with gawk:
kent$ cat f
10.1.2.200
10.1.2.201
10.1.2.202
10.1.2.203
kent$ awk -F'\n' -v RS="\0" -v OFS='","' -v q='"' '{NF--}$0=q$0q' f
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
$ awk '{o=o (NR>1?",":"") "\""$0"\""} END{print o}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
i wanted to keep only the 10.100.52.11 and delete everything else, 10.100.52.11 keeps changing so i don't want to hard code it
The original output was as below
"PrivateIpAddress": "10.100.52.111",
I tried the below command and removed "PrivateIpAddress": "
sudo aws ec2 describe-instances --filter Name=tag:Name,Values=bip-spark-es-worker1 |grep PrivateIpAddress |head -1|sed 's/^[ ^t]*\"PrivateIpAddress\"[:]* \"//g'
so the output for the above command now is
10.100.52.111",
I want to delete even the ending quotes and comma.
I tried with ["].$ and also \{2\}.$ did not work.
Please help.
Let sed do all the work. You don't need grep or head:
sed -n '/"PrivateIpAddress": /{s///; s/[",]//g; p; q}'
If content within " do not have " themselves,
grep PrivateIpAddress |head -1|sed 's/^[ ^t]*\"PrivateIpAddress\"[:]* \"//g'
can be replaced with
awk -F\" '/PrivateIpAddress/{print $4; exit}'
-F\" use " as field separator
/PrivateIpAddress/ if line matches this string
print $4 print 4th field which is 10.100.52.111 for given sample
exit will quit as only first match is required
some awk proposals
echo '"PrivateIpAddress": "10.100.52.111",'| awk -F: '{print substr($2,3,13)}'
10.100.52.111
echo '"PrivateIpAddress": "10.100.52.111",'| awk -F\" '{print $4}'
10.100.52.111
Alternative :
$ echo "\"PrivateIpAddress\": \"10.100.52.111\", "
"PrivateIpAddress": "10.100.52.111",
$ echo "\"PrivateIpAddress\": \"10.100.52.111\", " |grep -Po '(\d+[.]){3}\d+'
10.100.52.111
$ echo "\"PrivateIpAddress\": \"10.100.52.111\", " |grep -Eo '([[:digit:]]+[.]){3}[[:digit:]]+'
10.100.52.111
I'm trying to extract headers from emails and create a JSON fragment from them. I'm using sed to pull out the keys and values, but it's failing to put the trailing quote on each of the lines:
$ cat email1 | grep -i -e "^subject:" -e "^from:" -e "^to:" | \
sed -n 's/\^([^:]*\):[ ]*\(.*\)$/"\1":"\2"/gp'
"From":"Blah Blech <blah.blech#blahblech.com>
"To":"foo#bar.com
"Subject":"Yeah
I don't understand why the replacement pattern isn't working.
awk to the rescue!
$ awk -F": *" -vOFS=":" -vq="\"" 'tolower($0)~/^from|to|subject/
{print q$1q,q$2q}' email1
which combines cat or grep steps as well.
Stripping the carriage returns as #tripleee suggested fixed the issue with sed (using ctrl-v ctrl-m to capture the literal carriage return):
$ cat email1 | tr -d '^M' | grep -i -e "^subject:" -e "^from:" -e "^to:" | \
sed -n 's/^\([^:]*\):[ ]*\(.*\)$/"\1":"\2"/gp'
"From":"Blah Blech <blah.blech#blahblech.com>"
"To":"foo#bar.com"
"Subject":"Yeah"
I have one file:
file.txt
101|aaa {rating=1, dept=10, date=10/02/2013, com=11}
106|bbb {rating=2, dept=11, date=10/03/2013, com=11}
103|vvv {rating=3, dept=12, date=10/03/2013, com=11}
102|aaa {rating=1, dept=10, date=10/04/2013, com=11}
109|bbb {rating=2, dept=11, date=10/05/2013, com=11}
104|bbb {rating=2, dept=11, date=10/07/2013, com=11}
I am greping it based on:
for i in `cat file.txt | grep -i "|aaa "`
do
echo `echo $i|cut -d' ' -f1`"|" `sed -n '/date=/,/, com/p' $i` >> output.txt
done
This error occurs
"/sysdate=/,/systime/p: No such file or directory"
Please help me?
The output should be:
output.txt
101|aaa|10/02/2013
102|aaa|10/04/2013
awk is way better for these cases:
$ awk -F"[ =,]" -v OFS="|" '/aaa/{print $1, $9}' a
101|aaa|10/02/2013
102|aaa|10/04/2013
This sets field separators to either space, = or , and fetches the first and 9th fields, whenever the text aaa is found in the line.
I want to print the fields in specific format ,
Input :
col1|col2|col3|col4
I used cat file | cut -d '|' -f 3,1,4
output :
col1|col3|col4
But my expected output is:
col3|col1|col4
Can anyone help me with this?
From man cut:
Selected input is written in the same order that it is read, and is written exactly once
You should do:
$ awk -F'|' -vOFS='|' '{print $3,$1,$4}' <<< "col1|col2|col3|col4"
col3|col1|col4
even though awk is good,here is a perl solution:
perl -F"\|" -ane 'print join "|",#F[2,0,3]'
tested:
> echo "col1|col2|col3|col4" | perl -F"\|" -ane 'print join "|",#F[2,0,3]'
col3|col1|col4