I have two sets of files test.csv data.xml.
I am trying to grep a specific field from test.csv and search the string in data.xml. If string is found then print the corresponding line in test.csv file.
Example
search string is field 3 server name
test.csv
111,xxx,serversugar,port90
222,yyy,servertorque,port190
333,aaa,serverastrix,port8080
422,yxy,servertorque,port290
data.xml
<group>
<hostname>servertorque</hostname>
<hostname>serverastrix</hostname></group>
Output expected
222,yyy,servertorque,port190
333,aaa,serverastrix,port8080
422,yxy,servertorque,port290
with GNU sed & grep in 2 steps
sed '/>\w\+</!d;s/.*>\(\w\+\).*/\1/' data.xml>pattern.txt
grep -wf pattern.txt test.csv
..output:
222,yyy,servertorque,port190
333,aaa,serverastrix,port8080
422,yxy,servertorque,port290
One way with awk
awk -v FS="[><,]" 'NR==FNR{a[$3]++;next}$3 in a' data.xml test.csv
Test:
$ cat data.xml
<group>
<hostname>servertorque</hostname>
<hostname>serverastrix</hostname></group>
$ cat test.csv
111,xxx,serversugar,port90
222,yyy,servertorque,port190
333,aaa,serverastrix,port8080
422,yxy,servertorque,port290
$ awk -v FS="[><,]" 'NR==FNR {a[$3]++;next} $3 in a' data.xml test.csv
222,yyy,servertorque,port190
333,aaa,serverastrix,port8080
422,yxy,servertorque,port290
I assume you need:
awk -F',' '$3==<string_you_need> { print $0 }' test.csv
Related
My requirement is to split a file based on the first field.
My file looks something like this :
aaa|12345
bbb|45679
aaa|334564
ccc|qewqw
Now my awk command works just fine and splits the file with respect to the first field value.
awk -F\| '{f=($1); print $2 > f}' /myfile
Result : File name aaa has the below rows :
12345
334564
Now , I want to make it input driven , that is from a script i will enter the value aaa to this awk command and it will match with $1 and create the file like above. I tried with awk -v to take the value as input, but its not working the way I want it to.
Any help would be much appreciated.
Thanks in advance.
Like this?:
$ awk -v f=aaa -F\| 'f==$1{print $2 > f}' file
$ ls
aaa file
$ cat aaa
12345
334564
Explained:
$ awk -v f=aaa -F\| ' # set input field separator and var f="aaa"
f==$1 { # if f ie. "aaa" equals to first field value
print $2 > f # write second field value to file f ie. "aaa"
}' file
awk -v var=$input_value -F\| '{print $2 > var}' myfile
In your code in comments the problem is with f=($var). I think you wanted f=var.
cat file | awk -v inp=aaa -F\| '$1 ~ inp { print $2 > inp }'
Below are the full file names.
qwertyuiop.abcdefgh.1234567890.txt
qwertyuiop.1234567890.txt
trying to use
awk -F'.' '{print $1}'
How can i use awk command to extract below output.
qwertyuiop.abcdefgh
qwertyuiop
Edit
i have a list of files in a directory
i am trying to extract time,size,owner,filename into seperate variables.
for filenames.
NAME=$(ls -lrt /tmp/qwertyuiop.1234567890.txt | awk -F'/' '{print $3}' | awk -F'.' '{print $1}')
$ echo $NAME
qwertyuiop
$
NAME=$(ls -lrt /tmp/qwertyuiop.abcdefgh.1234567890.txt | awk -F'/' '{print $3}' | awk -F'.' '{print $1}')
$ echo $NAME
qwertyuiop
$
expected
qwertyuiop.abcdefgh
With GNU awk and other versions that allow manipulation of NF
$ awk -F. -v OFS=. '{NF-=2} 1' ip.txt
qwertyuiop.abcdefgh
qwertyuiop
NF-=2 will effectively delete last two fields
1 is an awk idiom to print contents of $0
Note that this assumes there are at least two fields in every line, otherwise you'd get an error
Similar concept with perl, prints empty line if number of fields in the line is less than 3
$ perl -F'\.' -lane 'print join ".", #F[0..$#F-2]' ip.txt
qwertyuiop.abcdefgh
qwertyuiop
With sed, you can preserve lines if number of fields is less than 3
$ sed 's/\.[^.]*\.[^.]*$//' ip.txt
qwertyuiop.abcdefgh
qwertyuiop
EDIT: Taking inspiration from Sundeep sir's solution and adding this following too in this mix.
awk 'BEGIN{FS=OFS="."} {$(NF-1)=$NF="";sub(/\.+$/,"")} 1' Input_file
Could you please try following.
awk -F'.' '{for(i=(NF-1);i<=NF;i++){$i=""};sub(/\.+$/,"")} 1' OFS="." Input_file
OR
awk 'BEGIN{FS=OFS="."} {for(i=(NF-1);i<=NF;i++){$i=""};sub(/\.+$/,"")} 1' Input_file
Explanation: Adding explanation for above code too here.
awk '
BEGIN{ ##Mentioning BEGIN section of awk program here.
FS=OFS="." ##Setting FS and OFS variables for awk to DOT here as per OPs sample Input_file.
} ##Closing BEGIN section here.
{
for(i=(NF-1);i<=NF;i++){ ##Starting for loop from i value from (NF-1) to NF for all lines.
$i="" ##Setting value if respective field to NULL.
} ##Closing for loop block here.
sub(/\.+$/,"") ##Substituting all DOTs till end of line with NULL in current line.
}
1 ##Mentioning 1 here to print edited/non-edited current line here.
' Input_file ##Mentioning Input_file name here.
I have the following list in a text file:
10.1.2.200
10.1.2.201
10.1.2.202
10.1.2.203
I want to encase in "double quotes", comma separate and join the values as one string.
Can this be done in sed or awk?
Expected output:
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203","10.1.2.204"
The easiest is something like this (in pseudo code):
Read a line;
Put the line in quotes;
Keep that quoted line in a stack or string;
At the end (or while constructing the string), join the lines together with a comma.
Depending on the language, that is fairly straightforward to do:
With awk:
$ awk 'BEGIN{OFS=","}{s=s ? s OFS "\"" $1 "\"" : "\"" $1 "\""} END{print s}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
Or, less 'wall of quotes' to define a quote character:
$ awk 'BEGIN{OFS=",";q="\""}{s=s ? s OFS q$1q : q$1q} END{print s}' file
With sed:
$ sed -E 's/^(.*)$/"\1"/' file | sed -e ':a' -e 'N' -e '$!ba' -e 's/\n/,/g'
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
(With Perl and Ruby, with a join function, it is easiest to push the elements onto a stack and then join that.)
Perl:
$ perl -lne 'push #a, "\"$_\""; END{print join(",", #a)}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
Ruby:
$ ruby -ne 'BEGIN{#arr=[]}; #arr.push "\"#{$_.chomp}\""; END{puts #arr.join(",")}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
here is another alternative
sed 's/.*/"&"/' file | paste -sd,
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
awk -F'\n' -v RS="\0" -v OFS='","' -v q='"' '{NF--}$0=q$0q' file
should work for given example.
Tested with gawk:
kent$ cat f
10.1.2.200
10.1.2.201
10.1.2.202
10.1.2.203
kent$ awk -F'\n' -v RS="\0" -v OFS='","' -v q='"' '{NF--}$0=q$0q' f
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
$ awk '{o=o (NR>1?",":"") "\""$0"\""} END{print o}' file
"10.1.2.200","10.1.2.201","10.1.2.202","10.1.2.203"
I want to grep version number in one file and replace it in another file. I want to grep 4.3.0.5 in file 1 and replace it in File 2 at 4.3.0.2. I have the below command to get the number , but how can I cut/replace it in second file??
File1 :
App :4.3.0.5 (or) App: 4.3.0.5-SNAPSHOT
File2: Before editing
grid_application_distribution_url=nexus://com.abcd.efge.ce/App/4.3.0.2/tar.gz/config
File 2 : after editing (Desired Result:)
If $VERISON in File is WITHOUT the word SNAPSHOT then in file 2
grid_application_distribution_url=nexus://com.abcd.efge.ce/App/4.3.0.5/tar.gz/config
If $VERSION has SNAPSHOT then line in file 2 should be
grid_application_distribution_url=nexus-snapshot://com.abcd.efge.ce/App/4.3.0.5/tar.gz/config
VER=$(awk -F: '/^App/{sub(/ .*$/, "", $2); print $2}'/path/file1.txt)
echo $VER
if ($vER ~ /SNAPSHOT/)
/usr/bin/ssh -t -t server2.com "sub("=nexus:", ":=nexus-snapshot") /path/file2" && sub(/[^\/]+\/tar\.gz/, $VER"/tar.gz") /path/file2
Something like this is all you need:
awk -F': +' 'NR==FNR{v=$2;next} {sub(/[^/]+\/tar.gz/,v"/tar.gz")} 1' File1 File2 > tmp && mv tmp File2
This awk script can do the job (this is an enhancement of above answer from #EDMorton):
Splitting the command in 2 as per OP's request
VER=$(awk -F' *: *' '/^App/{print $2}' file1)
awk -v v="$VER" '{
split(v, arr, "-");
sub(/[^\/]+\/tar\.gz/, arr[1]"/tar.gz");
if (arr[2] ~ /SNAPSHOT/)
sub("=nexus:", ":=nexus-snapshot")
}1' file2 > tmpFile
mv tmpFile > file2
You can try with this:
VERSION=($(grep -r "App:" /path/File1| awk '{print ($2)}'))
sed -i "s/4.3.0.2/$VERSION/" File2
it will look for "4.3.0.2" and change by value in $VERSION. File2 will be updated with this change.
If you want the file to keep the same, delete the flag -i:
sed "s/4.3.0.2/$VERSION/" File2
You will get the result in stdout.
As indicated in comments, 4.3.0.2 is not like this every time. Adapted for format X.Y.Z.W:
sed "s/\/[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]\(\/tar.gz\)/\/$VERSION\1/" File2
File is
a#gmail.com,b#yahoo.com
xyz#gmail.com
abc#gmail.com
ff#yahoo.co.in
jf#rediff.com
oop#hotmail.com
Output should be:
U|a#gmail.com,b#yahoo.com
D|xyz#gmail.com
R|abc#gmail.com
U|ff#yahoo.co.in
D|jf#rediff.com
R|oop#hotmail.com
I want to append specific string after every 3rd character.
#!/usr/bin/sed -f
s/^/U|/
n
s/^/D|/
n
s/^/R|/
Useful one-line scripts for sed
$ awk 'BEGIN {split("UDR",p,"")} {print p[((NR-1)%3)+1] "|" $0}' a.txt
U|a#gmail.com,b#yahoo.com
D|xyz#gmail.com
R|abc#gmail.com
U|ff#yahoo.co.in
D|jf#rediff.com
R|oop#hotmail.com