Unix File Merge - If key exists replace line with value - unix

I have 2 files table.cols and table.rules
table.cols:
column_1
column_2
column_3
column_4
column_5
table.rules:
column_3 my_function(column_3, another_parameter)
column_4 my_other_function(column_3, a_different_parameter)
I want to merge these files to produce:
column_1,
column_2,
my_function(column_3, another_parameter),
my_other_function(column_3, a_different_parameter),
column_5
Notice the commas at the end of each line except the last.

Try this -
$ head file?
==> file1 <==
column_3 my_function(column_3, another_parameter)
column_4 my_other_function(column_3, a_different_parameter)
==> file2 <==
column_1
column_2
column_3
column_4
column_5
$ awk -v line=$(wc -l < file2) 'NR==FNR{a[$1]=$2FS$3;next} {print (a[$1] ?a[$1]OFS :((FNR<line)?$0 OFS:$0))}' OFS=, file1 file2
column_1,
column_2,
my_function(column_3, another_parameter),
my_other_function(column_3, a_different_parameter),
column_5
Explained :
((FNR<line)?$0 OFS:$0)) : to ignore the comma from last line.

This script works as expected:
#! /bin/sh
OIFS="$IFS"
IFS=$'\n'
NL=$(cat table.cols| wc -l)
CL=1
rm -f /tmp/tablecols /tmp/tablerules
for LINE in $(cat table.cols)
do
echo -ne "${LINE}" >> /tmp/tablecols
if [ $CL -lt $NL ]; then
echo "," >> /tmp/tablecols
CL=$((CL + 1))
else
echo "" >> /tmp/tablecols
fi
done
for LINE in $(cat table.rules)
do
KEY=$(echo $LINE|cut -f 1 -d " ")
VAL=$(echo $LINE| cut -f 2-5 -d " ")
sed -i "s/${KEY}/${VAL}/g" /tmp/tablecols
done
mv /tmp/tablecols result
Hope it helps! :)

join -a1 table.cols table.rules | sed -e 1,4s/$/,/
join to combine the files
-a FILENUM print unpairable lines
sed commands have an optional range
substitute end-of-line with ,

Related

How to run awk on a file with cedella as delimiter

I have a file with below contents
cat file1.dat
anuÇ89Çhyd
binduÇ45Çchennai
I would like to print the second column with Ç as delimiter.
output should be
89
45
The manpage of awk mentions the following:
-F fs
--field-separator fs
Use fs for the input field separator (the value of the FS predefined variable).
So, this command does what you want:
cat file1.dat | awk -F'Ç' '{print $2}'
Given:
$ cat file
anuÇ89Çhyd
binduÇ45Çchennai
You can use cut:
$ cut -f 2 -d 'Ç' file
awk:
$ awk -F'Ç' '{print $2}' file
sed:
$ sed -E 's/^[^Ç]*Ç([^Ç]*).*/\1/' file
GNU grep:
$ grep -oP '^[^Ç]*Ç\K[^Ç]+(?=Ç)' file
Perl:
$ perl -lnE 'print $1 if /^[^Ç]*Ç([^Ç]+)Ç/' file
All those print:
89
45

No results while making precise matching using awk

I am having rows like this in my source file:
"Sumit|My Application|PROJECT|1|6|Y|20161103084527"
I want to make a precise match on Column 3 i.e. I do not want to use '~' operator while writing my awk command. However the command:
awk -F '|' '($3 ~ /'"$Var_ApPJ"'/) {print $3}' ${Var_RDR}/${Var_RFL};
is fetching me correct result but the command:
awk -F '|' '($3 == "${Var_ApPJ}") {print $3}' ${Var_RDR}/${Var_RFL};
fails to do so. Can anyone help in explaining why it happens and I am willing to use '==' because I do not want to match if the value is "PROJECT1" in source file.
Parameter Var_ApPJ="PROJECT"
${Var_RDR}/${Var_RFL} -> Refers to source file.
Refer to this part of documentation to know how to pass variable to awk.
I found an alternative way of '==' with '~':
awk -F '|' '($3 ~ "^${Var_ApPJ}"$) {print $3}' ${Var_RDR}/${Var_RFL};
here is the problem -
try below command -
awk -F '|' '$3 == Var_ApPJ {print $3}' ${Var_RDR}/${Var_RFL};
Remove curly braces and bracket.
vipin#kali:~$ cat kk.txt
a 5 b cd ef gh
vipin#kali:~$ awk -v var1="5" '$2 == var1 {print $3}' kk.txt
b
vipin#kali:~$
OR
#cat kk.txt
a 5 b cd ef gh
#var1="5"
#echo $var1
5
#awk '$2 == "'"$var1"'" {print $3}' kk.txt ### With "{}"
b
#
#awk '$2 == "'"${var1}"'" {print $3}' kk.txt ### without "{}"
b
#

AWK Include Whitespaces in Command

I have String: "./Delivery Note.doc 1" , where:
$1 = ./Delivery
$2 = Note.doc
$3 = 1
I need to execute sum command concatenating $1 and $2 but keeping white space (./Delivery Note.doc). I try this but it trim whitespaces:
| '{ command="sum -r "$1 $2"
Result: ./DeliveryNote.doc
To execute the sum command
echo "./Delivery Note.doc 1" | awk '{ command="sum -r \""$1" "$2"\""; print command}' | bash
$ echo "./Delivery Note.doc 1" | awk '{ command="sum -r "$1" "$2; print command}'
sum -r ./Delivery Note.doc

How to add a value/data to end of each row in Unix

I have fileA, fileB data as shown below
fileA
,,"user1","email"
,,"user2","email"
,,"user3","email"
,,"user4","email"
fileB
,,user2,location
,,user4,location
,,user1,location
,,user3,location
I want to search fileA user on fileB and get only location and add that one to fileA/or other file
Output expecting like
,,"user1","email",location
,,"user2","email",location
,,"user3","email",location
,,"user4","email",location
I'm trying the logic, using while get the fileA username and search that on fileB to get the location. but getting failed to add that with fileA back
Your help much appreciated
This should work:
for user in `awk -F\" '{print $2}' fileA`
do
loc=`grep ${user} fileB | awk -F',' '{print $4}'`
sed -i "/${user}/ s/$/,${loc}/" fileA
done
Adding the example:
$ cat fileA
,,"user1","email"
,,"user2","email"
,,"user3","email"
,,"user4","email"
$ cat fileB
,,user2,location2
,,user4,location4
,,user1,location1
,,user3,location3
$ for user in `awk -F\" '{print $2}' fileA`; do echo ${user}; loc=`grep ${user} fileB | awk -F',' '{print $4}'`; echo ${loc}; sed -i "/${user}/ s/$/,${loc}/" fileA; done
$ cat fileA
,,"user1","email",location1
,,"user2","email",location2
,,"user3","email",location3
,,"user4","email",location4
The description is not clear but based on the question you can use the following command to append a value/data to end of each row in Unix
sed -i '/search_pattern/ s/$/string_to_be_appended/' filename
You can do this entirely in awk
awk -F, '
NR==FNR{a[$3]=$4;next}
{for(x in a) if(index($3,x)>0) print $0","a[x]}' file2 file1
Test:
$ cat file1
,,"user1","email"
,,"user2","email"
,,"user3","email"
,,"user4","email"
$ cat file2
,,user2,location2
,,user4,location4
,,user1,location1
,,user3,location3
$ awk -F, 'NR==FNR{a[$3]=$4;next}{for(x in a) if(index($3,x)>0) print $0","a[x]}' file2 file1
,,"user1","email",location1
,,"user2","email",location2
,,"user3","email",location3
,,"user4","email",location4

Unix cut command taking an unordered list as arguments

The Unix cut command takes a list of fields, but not the order that I need it in.
$ echo 1,2,3,4,5,6 | cut -d, -f 1,2,3,5
1,2,3,5
$ echo 1,2,3,4,5,6 | cut -d, -f 1,3,2,5
1,2,3,5
However, I would like a Unix shell command that will give me the fields in the order that I specify.
Use:
pax> echo 1,2,3,4,5,6 | awk -F, 'BEGIN {OFS=","}{print $1,$3,$2,$5}'
1,3,2,5
or:
pax> echo 1,2,3,4,5,6 | awk -F, -vOFS=, '{print $1,$3,$2,$5}'
1,3,2,5
Or just use the shell
$ set -f
$ string="1,2,3,4,5"
$ IFS=","
$ set -- $string
$ echo $1 $3 $2 $5
1 3 2 5
Awk based solution is elegant. Here is a perl based solution:
echo 1,2,3,4,5,6 | perl -e '#order=(1,3,2,5);#a=split/,/,<>;for(#order){print $a[$_-1];}'

Resources