How to extract text from a file which appears one or more times in each line? - unix

Question

I have a text file which have 1 or more email ids in each line. E.g.
id:123, name:test, id: 5678, name john, address:new york
id:567, name:bob
id:3643, name:meg, id: 6721, name kate, address:la
Now, the problem is id:value may appear one or more times in a single line. How do I extract all id:value pairs so that the output is,
id:123, id:5678
id:567
id:3643, id:6721
I tried egrep -o but that is putting each id:value pair in a separate line.
sed/awk should do the trick but I am a noob
Do not want to use Perl as that will require a Perl installation.
EDIT:
On further analysis of the data files, I am seeing inconsistent separators, i.e. not all lines are , separated. Some are even separated with : and |. Also, , is appearing within the address value field. i.e. address:52nd st, new york. Can this be done in awk using a regex expression?

Answer 1

If your content is in the file test.txt then the following command:
cat test.txt | sed 's/ *: */:/g' | grep -o 'id:[0-9]*'
will return:
id:123
id:5678
id:567
id:3643
id:6721
The sed command is to remove any spaces adjacent to the colon, yielding an output of:
id:123, name:test, id:5678, name john, address:new york
id:567, name:bob
id:3643, name:meg, id:6721, name kate, address:la
and the grep -o command finds all matches to id: proceeded by zero or more numbers, with the -o to return only the matching part of the input string.
Per the man page:
-o, --only-matching Print only the matched (non-empty) parts of a matching
line, with each such part on a separate output line.
(FYI, the grep and sed commands are using regular expressions.)
EDIT:
Sorry, I didn't read carefully. I see that you object to the -o output format of one value per line. Back to the drawing board...
Note: If the reason you are opposed to the -o output is to preserve line numbers, using grep -no will give the following output (where the first number is the line number):
1:id:123
1:id:5678
2:id:567
3:id:3643
3:id:6721
Maybe that helps?

Answer 2

This might work for you (GNU sed):
sed -r 's/\<id:\s*/\n/g;s/,[^\n]*//g;s/\n/, id:/g;s/^, //' file
Convert the words id: and any following spaces to a unique token (in this case \n). Delete anyting following a , upto a \n. Replace the \n by the token , id: and then delete the leading ,.

Answer 3

This should work:
awk -F, '{id=0;for(i=1;i<=NF;i++) if($i~/id:/) id=id?id FS $i:$i; print id}' file
Test:
$ cat file
id:123, name:test, id: 5678, name john, address:new york
id:567, name:bob
id:3643, name:meg, id: 6721, name kate, address:la
$ awk -F, '{id=0;for(i=1;i<=NF;i++) if($i~/id:/) id=id?id FS $i:$i; print id}' file
id:123, id: 5678
id:567
id:3643, id: 6721

Answer 4

perl -lne 'push #a,/id:[^,]*/g;print "#a";undef #a' your_file
Tested Below:
> cat temp
id:123, name:test, id: 5678, name john, address:new york
id:567, name:bob
id:3643, name:meg, id: 6721, name kate, address:la
> perl -lne 'push #a,/id:[^,]*/g;print "#a";undef #a' temp
id:123 id: 5678
id:567
id:3643 id: 6721
>

Answer 5

This is just a variation of an answer allready given..I personaly prefere the script verion in a file more than the command line (better control, readability)
id.txt
id:1, name:test, id:2, name john, address:new york
id:3, name:bob
id:4, name:meg, id:5, name kate, address:la
id.akw
{
i=0
for(i=1;i<=NF;i++)
{ if($i~/id:/)
id=id?id $i:$i;}
print id
id=""
}
call: awk -f id.awk id.txt
output:
id:1, id:2,
id:3,
id:4, id:5,