I know ARGV[i] can be used for storing user input. However, I want to use it in awk script and get the ARGV[i] to compare the field from another text file. If the ARGV[i] matches the field or the field contains ARGV[i] which is the user input, then I want to return the other fields of that line.
Let say I have a file, testing.txt
123123123:Walter White:1:2:3:4
123123124:Jesse Pinkman:1:3:4:1
This is my awk script, awkScript.awk
#!/usr/bin/awk -f
BEGIN{FS = ":"; print "ENTER A NAME: "}
{
for (i = 1; i < ARGC; i++)
{
if ($2 ~ /'ARGV[i]'/)
{
print "Member ID: " $1
}
}
}
It just prints ENTER A NAME: when I execute the script file. It doesn't even get the input from the user.
From awk manual
ARGV is an array of command line arguments
That is the list of arguments passed while calling the awk sript.
you may want something like
$echo 'ENTER A NAME'
$read Name
Jesse Pinkman
$awk -v name="$Name" -F: '$2=name{print $1}' filename
123123123
123123124
Here -v option creates a variable named name in awk script and assigns the value of $Name variable from shell to name
$2=name{print $1} the $2=name selects all lines where $2 is name and prints the first column
Not sure what you're thinking about wrt using ARGV[] but here's one way to do what you want in awk:
$ awk 'BEGIN{FS=":"; msg="ENTER A NAME:"; print msg} NR==FNR{a[$2]=$0; next} $0 in a{print a[$0]} {print msg}' file -
ENTER A NAME:
Jesse Pinkman
123123124:Jesse Pinkman:1:3:4:1
ENTER A NAME:
Walter White
123123123:Walter White:1:2:3:4
ENTER A NAME:
Related
I have a shell script which is trying to trim a file from end of the line but I always get some error.
Shell Script:
AWK_EXPRESSION='{if(length>'"$RANGE1"'){ print substr('"$0 "',0, length-'"$RANGE2"'}) } else { print '"$0 "'} }'
for report in ${ACTUAL_TARGET_FOLDER}/* ; do
awk $AWK_EXPRESSION $report > $target_file
done
If I trigger the AWK command, I get unexpected newline or end of string near print.
What am I missing?
Why are you trying to store the awk body in a shell variable? Just use awk and the -v option to pass a shell value into an awk variable:
awk -v range1="$RANGE1" -v range2="$RANGE2" '{
if (length > range1) {
print substr($0,0, length-range2)
} else {
print
}
}' "$ACTUAL_TARGET_FOLDER"/* > "$target_file"
Add a few newlines to help readability.
Get out of the habit of using ALLCAPS variable names, leave those as reserved by the shell. One day you'll write PATH=something and then wonder why your script is broken.
Unquoted variables are subject to word splitting and glob expansion. Use double quotes for all your variables unless you know what specific side-effect you want to use.
I would recommend writing the AWK program using AWK variables instead of interpolating variables into it from the shell. You can pass variable into awk on the command line using the -v command line option.
Also, awk permits using white space to make the program readable, just like other programming languages. Like this:
AWK_EXPRESSION='{
if (length > RANGE1) {
print substr($0, 1, length-RANGE2)
} else {
print
}
}'
for report in "${ACTUAL_TARGET_FOLDER}"/* ; do
awk -v RANGE1="$RANGE1" -v RANGE2="$RANGE2" "$AWK_EXPRESSION" "$report" > "$target_file"
done
I need to loop over environment variables and get their names and values in POSIX sh (not bash). This is what I have so far.
#!/usr/bin/env sh
# Loop over each line from the env command
while read -r line; do
# Get the string before = (the var name)
name="${line%=*}"
eval value="\$$name"
echo "name: ${name}, value: ${value}"
done <<EOF
$(env)
EOF
It works most of the time, except when an environment variable contains a newline. I need it to work in that case.
I am aware of the -0 flag for env that separates variables with nul instead of newlines, but if I use that flag, how do I loop over each variable? Edit: #chepner pointed out that POSIX env doesn't support -0, so that's out.
Any solution that uses portable linux utilities is good as long as it works in POSIX sh.
There is no way to parse the output of env with complete confidence; consider this output:
bar=3
baz=9
I can produce that with two different environments:
$ env -i "bar=3" "baz=9"
bar=3
baz=9
$ env -i "bar=3
> baz=9"
bar=3
baz=9
Is that two environment variables, bar and baz, with simple numeric values, or is it one variable bar with the value $'3\nbaz=9' (to use bash's ANSI quoting style)?
You can safely access the environment with POSIX awk, however, using the ENVIRON array. For example:
awk 'END { for (name in ENVIRON) {
print "Name is "name;
print "Value is "ENVIRON[name];
}
}' < /dev/null
With this command, you can distinguish between the two environments mentioned above.
$ env -i "bar=3" "baz=9" awk 'END { for (name in ENVIRON) { print "Name is "name; print "Value is "ENVIRON[name]; }}' < /dev/null
Name is baz
Value is 9
Name is bar
Value is 3
$ env -i "bar=3
> baz=9" awk 'END { for (name in ENVIRON) { print "Name is "name; print "Value is "ENVIRON[name]; }}' < /dev/null
Name is bar
Value is 3
baz=9
Maybe this would work?
#!/usr/bin/env sh
env | while IFS= read -r line
do
name="${line%%=*}"
indirect_presence="$(eval echo "\${$name+x}")"
[ -z "$name" ] || [ -z "$indirect_presence" ] || echo "name:$name, value:$(eval echo "\$$name")"
done
It is not bullet-proof, as if the value of a variable with a newline happens to have a line beginning that looks like an assignment, it could be somewhat confused.
The expansion uses %% to remove the longest match, so if a line contains several = signs, they should all be removed to leave only the variable name from the beginning of the line.
Here an example based on the awk approach:
#!/bin/sh
for NAME in $(awk "END { for (name in ENVIRON) { print name; }}" < /dev/null)
do
VAL="$(awk "END { printf ENVIRON[\"$NAME\"]; }" < /dev/null)"
echo "$NAME=$VAL"
done
I am trying to search for a particular string in a Unix file from each and every line and error out those records. Can someone let me how can I improve my code which is as below. Also please share your thoughts if you have a better solution.
v_filename=$1;
v_new_file="new_file";
v_error_file="error_file";
echo "The input file name is $var1"
while read line
do
echo "Testing $line"
v_cnt_check=`grep ',' $line | wc -l`
echo "Testing $v_cnt_check"
# if [ $v_cnt_check > 2 ]; then
# echo $line >> $v_error_file
# else
# echo $line >> $v_new_file
# fi
done < $v_filename
Input:
1,2,3
1,2,3,4
1,2,3
Output:
(New file)
1,2,3
1,2,3
(Error file)
1,2,3,4
awk -F ',' -v new_file="$v_new_file" -v err_file="$v_error_file" \
'BEGIN { OFS="," }
NF == 3 { print >new_file }
NF != 3 { print >err_file }' $v_filename
The first line sets the file name variables and sets the field separator to comma. The second line sets the output field separator to comma too. The third line prints lines with 3 fields to the new file; the fourth line prints lines with other than 3 fields to the error file.
Note that your code would be excruciatingly slow on big files because it executes two processes per line. This code has only one process operating on the whole file — which will be really important if the input grow to thousand or millions or more lines.
From the grep manpage:
General Output Control
-c, --count
Suppress normal output; instead print a count of matching lines for each input file. With the -v, --invert-match option (see below), count non-
matching lines. (-c is specified by POSIX.)
You could do something like:
grep --count "your pattern" v_filename
to get the number of occurrences. If you just want the number of lines with your pattern, replace the grep shown above with:
grep "your pattern" v_filename | wc -l
I have a function
xyz()
{
x=$1*2
echo x
}
then I want to use it to replace a particular column in a csv file by awk.
File input.csv:
abc,2,something
def,3,something1
I want output like:
abc,4,somthing
def,6,something1
Command used:
cat input.csv|awk -F, -v v="'"`xyz "$2""'" 'BEGIN {FS=","; OFS=","} {$2=v1; print $0}'
Open file input.csv, calling function xyz by passing file 2nd filed as argument and result is stored back to position 2 of file, but is not working!
If I put constant in place of $2 while calling function it works:
Please help me to do this.
cat input.csv|awk -F, -v v="'"`xyz "14""'" 'BEGIN {FS=","; OFS=","} {$2=v1; print $0}'
This above line of code is working properly by calling the xyz function and putting the result back to 2nd column of file input.csv, but with only 14*2, as 14 is taken as constant.
There's a back-quote missing from your command line, and a UUOC (Useless Use of Cat), and a mismatch between variable v on the command line and v1 in the awk program:
cat input.csv|awk -F, -v v="'"`xyz "$2""'" 'BEGIN {FS=","; OFS=","} {$2=v1; print $0}'
^ Here ^ Here ^ Here
That should be written using $(…) instead:
awk -F, -v v="'$(xyz "$2")'" 'BEGIN {FS=","; OFS=","} {$2=v; print $0}' input.csv
This leaves you with a problem, though; the function xyz is invoked once by the shell before you start your awk script running, and is never invoked by awk. You simply can't do it that way. However, you can define your function in awk (and on the fly):
awk -F, 'BEGIN { FS = ","; OFS = "," }
function xyz(a) { return a * 2 }
{ $2 = xyz($2); print $0 }' \
input.csv
For your two-line input file, it produces your desired output.
i want to set the value of inputLineNumber to 20. I tried checking if no value is given by user by [[-z "$inputLineNumber"]] and then setting the value by inputLineNumber=20. The code gives this message ./t.sh: [-z: not found as message on the console. How to resolve this? Here's my full script as well.
#!/bin/sh
cat /dev/null>copy.txt
echo "Please enter the sentence you want to search:"
read "inputVar"
echo "Please enter the name of the file in which you want to search:"
read "inputFileName"
echo "Please enter the number of lines you want to copy:"
read "inputLineNumber"
[[-z "$inputLineNumber"]] || inputLineNumber=20
for N in `grep -n $inputVar $inputFileName | cut -d ":" -f1`
do
LIMIT=`expr $N + $inputLineNumber`
sed -n $N,${LIMIT}p $inputFileName >> copy.txt
echo "-----------------------" >> copy.txt
done
cat copy.txt
Changed the script after suggestion from #Kevin. Now the error message ./t.sh: syntax error at line 11: `$' unexpected
#!/bin/sh
truncate copy.txt
echo "Please enter the sentence you want to search:"
read inputVar
echo "Please enter the name of the file in which you want to search:"
read inputFileName
echo Please enter the number of lines you want to copy:
read inputLineNumber
[ -z "$inputLineNumber" ] || inputLineNumber=20
for N in $(grep -n $inputVar $inputFileName | cut -d ":" -f1)
do
LIMIT=$((N+inputLineNumber))
sed -n $N,${LIMIT}p $inputFileName >> copy.txt
echo "-----------------------" >> copy.txt
done
cat copy.txt
Try changing this line from:
[[-z "$inputLineNumber"]] || inputLineNumber=20
To this:
if [[ -z "$inputLineNumber" ]]; then
inputLineNumber=20
fi
Hope this helps.
Where to start...
You are running as /bin/sh but trying to use [[. [[ is a bash command that sh does not recognize. Either change the shebang to /bin/bash (preferred) or use [ instead.
You do not have a space between [[-z. That causes bash to read it as a command named [[-z, which clearly doesn't exist. You need [[ -z $inputLineNumber ]] (note the space at the end too). Quoting within [[ doesn't matter, but if you change to [ (see above), you will need to keep the quotes.
Your code says [[-z but your error says [-z. Pick one.
Use $(...) instead of `...`. The backticks are deprecated, and $() handles quoting appropriately.
You don't need to cat /dev/null >copy.txt, certainly not twice without writing to it in-between. Use truncate copy.txt or just plain >copy.txt.
You seem to have inconsistent quoting. Quote or escape (\x) anything with special characters (~, `, !, #, $, &, *, ^, (), [], \, <, >, ?, ', ", ;) or whitespace and any variable that could have whitespace. You don't need to quote string literals with no special characters (e.g. ":").
Instead of LIMIT=`expr...`, use limit=$((N+inputLineNumber)).