I have a file test.sh from which I am executing the following awk command.
awk -f x.awk < result/output.txt >>difference.txt
x.awk
while (getline < result/$bld/$DeviceType)
the variable DeviceType and bld are available in test.sh.
I have declared them as export type.
export DeviceType=$line
Even then while executing test.sh file, the script stops at following line
awk -f x.awk < result/output.txt >>difference.txt
and I am getting
awk: x.awk:4: (FILENAME=- FNR=116) fatal: division by zero attempted
error.
The awk script is read by awk, not touched by the shell. Inside an awk script, $bld means 'the field designated by the number in the variable bld' (that's the awk variable bld).
You can set awk variables on the command line (officially with the -v option):
awk -v bld="$bld" -v dev="$DeviceType" -f x.awk < result/output.txt >> difference.txt
Whether that does what you want is still debatable. Most likely you need x.awk to contain something like:
BEGIN { file = sprintf("result/%s/%s", bld, dev); }
{ while ((getline < file) > 0) print }
awk is not shell just like C is not shell. You should not expect to be able to access shell variables within an awk program any more than you can access shell variables within a C program.
To pass the VALUE of shell variables to an awk script, see http://cfajohnson.com/shell/cus-faq-2.html#Q24 for details but essentially:
awk -v awkvar="$shellvar" '{ ... use awkvar ...}'
is usually the right approach.
Having said that, whatever you're trying to do it looks like the wrong approach. If you are considering using getline, make sure to read http://awk.freeshell.org/AllAboutGetline first and understand all of the caveats but if you tell us what it is you're trying to do with sample input and expected output we can almost certainly help you come up with a better approach that has nothing to do with getline.
Related
I want a unix command (that I will call in a ControlM job) that changes the value of the first column of my .csv file (not the header line), with the date of the previous day (expected format : YYYY-MM-DD).
I tried many commands but none of them do want I want :
tmp=$(mktemp) && awk -F\| -v val=`date -d yesterday +%F` 'NR>1 {gsub($1,val)}' file.csv > "$tmp" && mv "$tmp" file.csv
or :
awk -F\| -v val=`date -d yesterday +%F` '{gsub($1, val)}1' file.csv
even tried gensub but not working.
Example of what I want :
Input :
VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-05;2017-11-15;BRIDGE;HELLO
2019-03-05;2018-03-17;WORK;DATA
Output I want (as today is 2019-03-07):
VALUE_DATE;TRADE_DATE;DESCR1;DESCR2
2019-03-06;2017-11-15;BRIDGE;HELLO
2019-03-06;2018-03-17;WORK;DATA
Can you help please and give me examples of commands that should work, I'm not finding a solution.
Thanks a lot
Could you please try following first?(not saving output into file.csv itself it will print output on terminal once happy then you could use answer
provided at last of this post)
awk -v val=$(date -d yesterday +%F) 'BEGIN{FS=OFS=";"}FNR>1{$1=val} 1' file.csv
Problems identified in OP's code(and fixed in my suggestion):
1- Use of backtick is depreciated now to save shell variable's values, so instead use val=$(date....) for declaring awk's variable named val.
2- Use of -F, you have set your field separator as \| which is pipe but when we see your provided sample Input_file carefully it is delimited with ;(semi colon) NOT | so that is also one of the reason why it is not reflecting in output.
3- Since use of gsub($1,val), replaces whole line to only with value of variable val
because
syntax of gsub is something like: gsub(your_regex/value_needs_to_be_replaced,"new_value"/variable_which_should_be_there_after_replacement,current_line/variable). Since you have defined wrong field separator so whole line being treated as $1 and thus when you print it by doing awk -F\| -v val=$(date -d yesterday +%F) 'NR>1 {gsub($1,val)} 1' file.csv it will only print previous dates.
4- 4th and main issue is you have NOT printed anything, so even you did mistakes you will NOT see any output either on terminal or in output file.
If happy then you could run your own command to make changes into Input_file itself.(I am assuming that you are having propervaluein your tmp variable here)
tmp=$(mktemp) && awk -v val=$(date -d yesterday +%F) 'BEGIN{FS=OFS=";"}FNR>1{$1=val} 1' file.csv > "$tmp" && mv "$tmp" file.csv
On my FreeBSD 10.1 I'm writing a little piece of code that basically calls ls and automatically breaks the results down into something like this:
directory:
2.4M .git
528K src
380K dist
184K test
file:
856K CONDUCT.md
20K README.md
........
You will only need to list out directories and regular files, and you don't have to list out . .., but you have to list out hidden files, and sort them from largest to smallest separately.
The challenge is to complete it as a one-line command without using $(cmd), &&, ||, >, >>, <, ;, & and within 12 pipes (back quotes count as well).
Currently my progress is:
ls -Alh | sort -d -h -r |
awk 'BEGIN {print "Directories:"}
NR>1 {if(substr($1,1,1)~"d")print" "$5" "$9}'
which prints out only until the last directory item. But since the entire command will output once every record, I can't find a way to print files: only once, and then print out the remaining output.
Well, you may have to store the files in an array and print at the end:
ls -Alh|sed 1d|
sort -h -k5r|
awk 'BEGIN {print "Directories:"}
/^d/{print "\t"$5"\t"$9}
/^-/{f[n++]=sprintf("\t"$5"\t"$9)}
END{print "Files:";
for(i=0;i<n;++i)print f[i]}'
One additional problem you'll need to work out: files and dirs may have spaces in the name, and the simple $9 will be insufficient for that case.
I have an input file "myfile" and a shell variable "VAR" whose
contents are shown below:
# cat myfile
Hello World
Hello Universe
# echo $VAR
World
I need to process all lines in "myfile" which contains pattern "World"
using awk. It works if i do as shown below:
# awk '/World/ { print $0 }' myfile
But i could not use VAR to do the same operation. I tried the
following even knowing that these will not work:
# awk '/$VAR/ { print $0 }' myfile
and
# awk -v lvar=$VAR '/lvar/ { print $0 }' myfile
and
# awk -v lvar=$VAR 'lvar { print $0 }' myfile
Please let me know how to match the contents of VAR in awk.
Thanks in advance
You could use:
awk -v lvar="$VAR" '$0~lvar {print}' myfile
Or (works but not recommended):
awk "/$VAR/ {print}" myfile
This is probably what you want:
awk -v lvar="$VAR" '$0~lvar' myfile
Never do
awk "/$VAR/" myfile
or any other syntax that expands shell variables inline within the awk script such that they become part of the script as it can produce all sorts of bizarre failures and error messages depending on how the shell variable is populated since with that syntax there is no variable inside awk but the shell variable contents are instead integrated into the script as if you'd hard-coded it.
Always do the above using -v or
awk 'BEGIN{lvar=ARGV[1]; delete ARGV[1]} $0~lvar' "$VAR" myfile
depending on your requirements for the shell expanding backslashes. See http://cfajohnson.com/shell/cus-faq-2.html#Q24 for details.
Only do
awk '$0~lvar' lvar="$VAR" myfile
if you are processing multiple files, need to change initial values of the variable between files, and do not have gawk for BEGINFILE or need to populate those values from environment variables.
Note that all of the above do a regexp match on lvar, if you need a string match instead than use awk -v lvar="$VAR" 'index($0,lvar)'.
If you export your variable, it becomes a simpler solution:
export VAR
awk '$0~ENVIRON["VAR"]' myfile
Takes out any issues of reprocessing the variable's value.
I've looked around for awhile and found only either questions touching on the subject or providing me with an answer that does not work. Here's the question:
I'm working on an assignment for school that requires me to read in command line arguments for an awk script (which seems odd to begin with, but eh). We're using an older version of Unix and I'm running Bash. This awk only has the -f and -Fc options. Basically, I keep trying to do "awk -f awk_script arg1 arg2 arg3 arg4 arg5 arg6" but each time awk attempts to open arg1 as a file, which it isn't. An example I saw elsewhere addressing this was:
awk 'BEGIN { print "ARGV[1] = ", ARGV[1] }' foo bar
It was supposed to print "foo", but on this system I only get the output "ARGV[1] = awk: can't open foo". So, in summary, is there any way around this? Can an awk this old read command line arguments and use them for anything other than input files? The instructors notes file hinted at the above usage (of printing foo), but his program doesn't even run, so...
Any help would be greatly appreciated.
After Edit: Using SunOS 5.10 and this awk does not support the -v option, ONLY the -f and -Fc
You can decrement ARGC after reading arguments so that only the first(s) argument(s) is(are) considered by awk as input file(s) :
#!/bin/awk -f
BEGIN {
for (i=ARGC; i>2; i--) {
print ARGV[ARGC-1];
ARGC--;
}
}
…
Or alternatively, you can reset ARGC after having read all arguments :
#!/bin/awk -f
BEGIN {
for (i=0; i<ARGC; i++) {
print ARGV[ARGC-1];
}
ARGC=2;
}
…
Both methods will correctly process myawkscript.awk foobar foo bar … as if foobar was the only file to process (of course you can set ARGC to 3 if you want the two first arguments as files, etc.). In your particular case, it seems you don't want to process any file, so you would set ARGC to 1.
Use nawk or /usr/xpg4/bin/awk. These are newer versions of awk that support more features.
Alternatively, you can install another version of awk like mawk or GNU awk.
A possible work around - maybe not acceptable - would be to use the -v option of awk.
awk -v arg1=foo 'BEGIN { print arg1; }'
i am trying to create a shell script to search for a specific index in a multiline csv file.
the code i am trying is:
#!/bin/sh
echo "please enter the line no. to search: "
read line
echo "please enter the index to search at: "
read index
awk -F, 'NR=="$line"{print "$index"}' "$1"
the awk command I try to use on the shell works absolutely fine. But when I am trying to create a shell script out of this command, it fails and gives no output. It reads the line no. and index. and then no output at all.
is there something I am doing wrong?
I run the file at the shell by typing:
./fetchvalue.sh newfile.csv
Your quoting is not going to work. Try this:
awk -F, 'NR=="'$line'"{print $'$index'}' "$1"
Rather than going through quoting hell, try this:
awk -F, -v line=$line -v myindex=$index 'NR==line {print $myindex}' "$1"
(Index is a reserved word in awk, so I gave it a slightly differet name)