My main question is how to split strings on the command line into parameters using a terminal command in Linux?
For example
on the command line:
./my program hello world "10 20 30"
The parameters are set as:
$1 = hello
$2 = world
$3 = 10 20 30
But I want:
$1 = hello
$2 = world
$3 = 10
$4 = 20
$5 = 30
How can I do it correctly?
You can reset the positional parameters $# by using the set builtin. If you do not double-quote $#, the shell will word-split it producing the behavior you desire:
$ cat my_program.sh
#! /bin/sh
i=1
for PARAM; do
echo "$i = $PARAM";
i=$(( $i + 1 ));
done
set -- $#
echo "Reset \$# with word-split params"
i=1
for PARAM; do
echo "$i = $PARAM";
i=$(( $i + 1 ));
done
$ sh ./my_program.sh foo bar "baz buz"
1 = foo
2 = bar
3 = baz buz
Reset $# with word-split params
1 = foo
2 = bar
3 = baz
4 = buz
As an aside, I find it mildly surprising that you want to do this. Many shell programmers are frustrated by the shell's easy, accidental word-splitting — they get "John", "Smith" when they wanted to preserve "John Smith" — but it seems to be your requirement here.
Use xargs:
echo "10 20 30" | xargs ./my_program hello world
xargs is a command on Unix and most Unix-like operating systems used
to build and execute command lines from standard input. Commands such as
grep and awk can accept the standard input as a parameter, or argument
by using a pipe. However, others such as cp and echo disregard the
standard input stream and rely solely on the arguments found after the
command. Additionally, under the Linux kernel before version 2.6.23,
and under many other Unix-like systems, arbitrarily long lists of
parameters cannot be passed to a command,[1] so xargs breaks the list
of arguments into sublists small enough to be acceptable.
(source)
Related
I am following this thread: https://stackoverflow.com/a/19742842/5057251
for typeset (or declare) in ZSH, not BASH.
#Declare (or typeset) an array of integers
#declare -ai int_array
typeset -ai int_array
int_array=(1 2 3)
echo "${int_array[#]}"
Then
# Attempt to change 1st element to string. (expect to fail)
int_array[1]="Should fail" || echo "error: ${LINENO}"
echo "${int_array[#]}"
Bash finds the error, gracefully reports error and lineno, prints:
1 2 3
But Zsh accepts, prints:
Should fail 2 3
Not sure why different.
There are two problems here:
In bash, and zsh, assigning a string to an integer variable causes that string to be evaluated as an arithmetic expression. Thus, this is not an error:
$ typeset -i foo
$ foo="bar"
If bar was a variable previously set to an arithmetic expression, then bar's expansion would be evaluated as such:
$ bar=10+2
$ typeset -i foo
$ foo="bar"
$ echo "$foo"
12
The error in your assignment, of course, is that there's no way to expand Should fail like that. If it were, say, Should - fail (an arithmetic expression subtracting the value of the two variables Should and fail, for example, it would still work:
$ foo="Should - fail"
$ echo "$foo"
0
The second problem is that nothing in the zsh docs indicate that -i may be set for an entire array, and so the -a in -ai is ignored:
bash-5.0$ typeset -ai foo
bash-5.0$ declare -p foo
declare -ai foo=([0]="0") # the previous value was retained in the array
vs zsh:
% typeset -ai foo
% foo[1]=10
% foo[2]=20
% declare -p foo
typeset -i foo=20 # treated as a normal variable, not array
What you're seeing is essentially int_array being redeclared as an array (without any qualifiers) when you do int_array=(1 2 3):
% foo=(1 2 3)
% declare -p foo
typeset -a foo=( 1 2 3 )
Using zsh typeset can produce a few possible outcomes:
- no errors, works (yeah!).
- errors, script fails (fix!).
- no errors, but unexpected behavior. (scratch head)
As an example of last category, this produces no errors, but the typeset -p reveals -i is ignored.
{
unset int_array
typeset -ia int_array
int_array=(1 2 3)
echo $? "-Point A"
typeset -p int_array
} always {
echo $? "-Point B"
typeset -p int_array
(( TRY_BLOCK_ERROR=0 ))
}
echo $? "-Point C"
echo "survived"
produces
0 -Point A
typeset -a int_array=( 1 2 3 )
0 -Point B
typeset -a int_array=( 1 2 3 )
0 -Point C
survived
The first line unsets int_array. The typeset command declares
int_array to be both an array and int, which is not what zsh allows. The next
line assigns int_array to a value. There is no error as the $? tells us,
but close examination of final typeset -p int_array reveals what actually
happened.
With a small change, we can produce errors and use the always block and
typeset -p to find more details.
{
unset int_array
typeset -ia int_array=(1 2 3) # error
echo $? "-Point A"
typeset -p int_array
} always {
echo $? "-Point B"
typeset -p int_array
(( TRY_BLOCK_ERROR=0 ))
}
echo $? "-Point C"
echo "survived"
040_declare_version2.sh:typeset:135: int_array: inconsistent type for assignment
1 -Point B
040_declare_version2.sh:typeset:140: no such variable: int_array
1 -Point C
survived
The only difference is int_array was given a value in the faulty typeset -ia statement.
This produces errors, and the script jumps to the always block.
The (( TRY_BLOCK_ERROR=0)) allows the script to continue
and not terminate, but the error is still reported at "Point C".
To check shell version:
$SHELL --version
zsh 5.4.2 (x86_64-ubuntu-linux-gnu)
I want to count number of characters until the pattern 030 in megarow (do not read data forward from that point) such that you do not read the whole megarow in in memory.
It should return 28.
Megastring Data
48000000fe5a1eda480000000d00030001000000cd010000020000000000000000000000000000000000000000000000000000000200000001000000ffffffff57ea5e55ff640c00585e0000fe5a1eda480000000d00030007000000cd010000010000000000000002000000000000800000000000000000000000
My initial idea was to split at first instance of 030 but I did not succeed with this.
I am also not familiar with split command's capability to read only until the end of the pattern.
How can you count quickly until the first match?
If your megarow is in a file named megarow_file you could do the following:
#!/bin/bash
INPUT=megarow_file
SEARCH_STRING="030"
comp_string=""
while IFS= read -r -n1 char
do
char_count=`expr $char_count + 1`
comp_string="${comp_string}${char}"
comp_string_length=${#comp_string}
if [ $comp_string_length -eq 3 ]; then
# echo comparing value $comp_string
if [ $comp_string = $SEARCH_STRING ]; then
# echo match
break
fi
fi
if [ $comp_string_length -gt 3 ]; then
# echo its bigger than 3, strip 1st char
comp_string="${comp_string:1:3}"
# echo comparing value $comp_string
if [ $comp_string = $SEARCH_STRING ]; then
# echo match
break
fi
fi
done < "$INPUT"
count_up_to_comp_string=`expr $char_count - ${#SEARCH_STRING}`
echo "Length up to ${SEARCH_STRING} was ${count_up_to_comp_string} characters"
Comparing GNU awk and BSD AWK initiated by BlueMoon's comment
$ time cat megaRow | awk '{print index($0, "fafafafa")-1}'
48584
real 1m13.489s
user 1m11.608s
sys 0m4.685s
$ time cat megaRow | gawk '{print index($0, "fafafafa")-1}'
48584
real 1m12.792s
user 1m8.845s
sys 0m4.933s
where GNU AWK little faster but not enough significantly, because within uncertainty.
i am trying to solve this bash script which reads an arithmetic expression from user and echoes it to the output screen with round up of 3 decimal places in the end.
sample input
5+50*3/20 + (19*2)/7
sample output
17.929
my code is
read x
echo "scale = 3; $x" | bc -l
when there is an input of
5+50*3/20 + (19*2)/7
**my output is **
17.928
which the machine wants it to be
17.929
and due to this i get the solution wrong. any idea ?
The key here is to be sure to use printf with the formatting spec of "%.3f" and printf will take care of doing the rounding as you wish, as long as "scale=4" for bc.
Here's a script that works:
echo -e "please enter math to calculate: \c"
read x
printf "%.3f\n" $(echo "scale=4;$x" | bc -l)
You can get an understanding of what is going on with the above solution, if you run this command at the commandline: echo "scale=4;5+50*3/20 + (19*2)/7" | bc the result will be 17.9285. When that result is provided to printf as an argument, the function takes into account the fourth decimal place and rounds up the value so that the formatted result displays with precisely three decimal places and with a value of 17.929.
Alternatively, this works, too without a pipe by redirecting the here document as input for bc, as follows which avoids creating a sub-shell:
echo -e "please enter math to calculate: \c"
read x
printf "%.3f\n" $(bc -l <<< "scale=4;$x")
You are not rounding the number, you are truncating it.
$ echo "5+50*3/20 + (19*2)/7" | bc -l
17.92857142857142857142
$ echo "scale = 3; 5+50*3/20 + (19*2)/7" | bc -l
17.928
The only way I know to round a number is using awk:
$ awk 'BEGIN { rounded = sprintf("%.3f", 5+50*3/20 + (19*2)/7); print rounded }'
17.929
So, in you example:
read x
awk 'BEGIN { rounded = sprintf("%.3f", $x; print rounded }'
I entirely agree with jherran that you are not rounding the number, you are truncating it. I would go on to say that scale is probably just not behaving at all the way you want it, possibly in a way that noone would want it to behave.
> x="5+50*3/20 + (19*2)/7"
> echo "$x" | bc -l
17.92857142857142857142
> echo "scale = 3; $x" | bc -l
17.928
Furthermore, because of the behaviour of scale, you are rounding each multiplication/division separately from the additions. Let me prove my point with some examples :
> echo "scale=0; 5/2" | bc -l
2
> echo "scale=0; 5/2 + 7/2" | bc -l
5
> echo "5/2 + 7/2" | bc -l
6.00000000000000000000
However scale without any operation doesn't work either. There is an ugly work-around :
> echo "scale=0; 5.5" | bc -l
5.5
> echo "scale=0; 5.5/1" | bc -l
5
So tow things come out of this.
If you want to use bc's scale, do it only for the final result already computed, and even then, beware.
Remember that rounding is the same as truncating a number + half of the desired precision.
Let us take the example of rounding to the nearest integer, if you add .5 to a number that should be rounded up, its integer part will take the next integer value and truncation will give the desired result. If that number should have been rounded down, then adding .5 will not change its integer value and truncation will yield the same result as when nothing was added.
Thus my solution follows :
> y=$(echo "$x" | bc -l)
> echo "scale=3; ($y+0.0005)/1" | bc -l # scale doesn't apply to the +, so we get the expected result
17.929
Again, note that the following doesn't work (as explained above), thus breaking it up in two operations is really needed :
> echo "scale=3; ($x+0.0005)/1" | bc -l
17.928
Usually grep command is used to display the line contaning the specified pattern. Is there any way to display n lines before and after the line which contains the specified pattern?
Can this will be achieved using awk?
Yes, use
grep -B num1 -A num2
to include num1 lines of context before the match, and num2 lines of context after the match.
EDIT:
Seems the OP is using AIX. This has a different set of options which doesn't include -B and -A
this link describes grep on AIX 4.3 (it doesn't look promising)
Matt's perl script might be a better solution.
Here is what I usually do on AIX:
before=2 << The number of lines to be shown Before >>
after=2 << The number of lines to be shown After >>
grep -n <pattern> <filename> | cut -d':' -f1 | xargs -n1 -I % awk "NR<=%+$after && NR>=%-$before" <filename>
If you do not want the extra 2 varialbles you can always use it an a one line:
grep -n <pattern> <filename> | cut -d':' -f1 | xargs -n1 -I % awk 'NR<=%+<<after>> && NR>=%-<<before>>' <filename>
Suppose I have a pattern 'stack' and the filename is flow.txt
I want 2 lines before and 3 lines after. The the command will be like:
grep -n 'stack' flow.txt | cut -d':' -f1 | xargs -n1 -I % awk 'NR<=%+3 && NR>=%-2' flow.txt
I want 2 lines before and only - the the command will be like:
grep -n 'stack' flow.txt | cut -d':' -f1 | xargs -n1 -I % awk 'NR<=% && NR>=%-2' flow.txt
I want 3 lines after and only - the the command will be like:
grep -n 'stack' flow.txt | cut -d':' -f1 | xargs -n1 -I % awk 'NR<=%+3 && NR>=%' flow.txt
Multiple Files - change it for Awk & grep. From above for the pattern 'stack' with the filename is flow.* - 2 lines before and 3 lines after. The the command will be like:
awk 'BEGIN {
before=1; after=3; pattern="stack";
i=0; hold[before]=""; afterprints=0}
{
#Print the lines from the previous Match
if (afterprints > 0)
{
print FILENAME ":" FNR ":" $0
afterprints-- #keep a track of the lines to print after - this can be reset if a match is found
if (afterprints == 0) print "---"
}
#Look for the pattern in current line
if ( match($0, pattern) > 0 )
{
# print the lines in the hold round robin buffer from the current line to line-1
# if (before >0) => user wants lines before avoid divide by 0 in %
# and afterprints => 0 - we have not printed the line already
for(j=i; j < i+before && before > 0 && afterprints == 0 ; j++)
print hold[j%before]
if (afterprints == 0) # print the line if we have not printed the line already
print FILENAME ":" FNR ":" $0
afterprints=after
}
if (before > 0) # Store the lines in the round robin hold buffer
{ hold[i]=FILENAME ":" FNR ":" $0
i=(i+1)%before }
}' flow.*
From the tags, it's likely that the system has a grep that may not support providing context (Solaris is one system that doesn't and I can't remember about AIX). If that is the case, there's a perl script that may help at http://www.sun.com/bigadmin/jsp/descFile.jsp?url=descAll/cgrep__context_grep.
If you have sed you could use this shell script
BEFORE=2
AFTER=3
FILE=file.txt
PATTERN=pattern
for i in $(grep -n $PATTERN $FILE | sed -e 's/\:.*//')
do head -n $(($AFTER+$i)) $FILE | tail -n $(($AFTER+$BEFORE+1))
done
What it does is, grep -n prefixes each match with the line it was found at, the sed strips all but the line it was found at. Then you use head to get the lines up to the line it was found on plus an additional $AFTER lines. That's then piped to tail to just get $BEFORE + $AFTER + 1 lines (that is, your matching line plus the number of lines before and after)
Sure there is (from the grep man page):
-B NUM, --before-context=NUM
Print NUM lines of leading context before matching lines.
Places a line containing a group separator (--) between
contiguous groups of matches. With the -o or --only-matching
option, this has no effect and a warning is given.
-A NUM, --after-context=NUM
Print NUM lines of trailing context after matching lines.
Places a line containing a group separator (--) between
contiguous groups of matches. With the -o or --only-matching
option, this has no effect and a warning is given.
and if you want the same amount of lines before AND after the match, use:
-C NUM, -NUM, --context=NUM
Print NUM lines of output context. Places a line containing a
group separator (--) between contiguous groups of matches. With
the -o or --only-matching option, this has no effect and a
warning is given.
you can use awk
awk 'BEGIN{t=4}
c--&&c>=0
/pattern/{ c=t; for(i=NR;i<NR+t;i++)print a[i%t] }
{ a[NR%t]=$0}
' file
output
$ more file
1
2
3
4
5
pattern
6
7
8
9
10
11
$ ./shell.sh
2
3
4
5
6
7
8
9
I am working on a UNIX box, and trying to run an application, which gives some debug logs to the standard output. I have redirected this output to a log file, but now wish to get the lines where the error is being shown.
My problem here is that a simple
cat output.log | grep FAIL
does not help out. As this shows only the lines which have FAIL in them. I want some more information along with this. Like the 2-3 lines above this line with FAIL. Is there any way to do this via a simple shell command? I would like to have a single command line (can have pipes) to do the above.
grep -C 3 FAIL output.log
Note that this also gets rid of the useless use of cat (UUOC).
grep -A $NUM
This will print $NUM lines of trailing context after matches.
-B $NUM prints leading context.
man grep is your best friend.
So in your case:
cat log | grep -A 3 -B 3 FAIL
I have two implementations of what I call sgrep, one in Perl, one using just pre-Perl (pre-GNU) standard Unix commands. If you've got GNU grep, you've no particular need of these. It would be more complex to deal with forwards and backwards context searches, but that might be a useful exercise.
Perl solution:
#!/usr/perl/v5.8.8/bin/perl -w
#
# #(#)$Id: sgrep.pl,v 1.6 2007/09/18 22:55:20 jleffler Exp $
#
# Perl-based SGREP (special grep) command
#
# Print lines around the line that matches (by default, 3 before and 3 after).
# By default, include file names if more than one file to search.
#
# Options:
# -b n1 Print n1 lines before match
# -f n2 Print n2 lines following match
# -n Print line numbers
# -h Do not print file names
# -H Do print file names
use strict;
use constant debug => 0;
use Getopt::Std;
my(%opts);
sub usage
{
print STDERR "Usage: $0 [-hnH] [-b n1] [-f n2] pattern [file ...]\n";
exit 1;
}
usage unless getopts('hnf:b:H', \%opts);
usage unless #ARGV >= 1;
if ($opts{h} && $opts{H})
{
print STDERR "$0: mutually exclusive options -h and -H specified\n";
exit 1;
}
my $op = shift;
print "# regex = $op\n" if debug;
# print file names if -h omitted and more than one argument
$opts{F} = (defined $opts{H} || (!defined $opts{h} and scalar #ARGV > 1)) ? 1 : 0;
$opts{n} = 0 unless defined $opts{n};
my $before = (defined $opts{b}) ? $opts{b} + 0 : 3;
my $after = (defined $opts{f}) ? $opts{f} + 0 : 3;
print "# before = $before; after = $after\n" if debug;
my #lines = (); # Accumulated lines
my $tail = 0; # Line number of last line in list
my $tbp_1 = 0; # First line to be printed
my $tbp_2 = 0; # Last line to be printed
# Print lines from #lines in the range $tbp_1 .. $tbp_2,
# leaving $leave lines in the array for future use.
sub print_leaving
{
my ($leave) = #_;
while (scalar(#lines) > $leave)
{
my $line = shift #lines;
my $curr = $tail - scalar(#lines);
if ($tbp_1 <= $curr && $curr <= $tbp_2)
{
print "$ARGV:" if $opts{F};
print "$curr:" if $opts{n};
print $line;
}
}
}
# General logic:
# Accumulate each line at end of #lines.
# ** If current line matches, record range that needs printing
# ** When the line array contains enough lines, pop line off front and,
# if it needs printing, print it.
# At end of file, empty line array, printing requisite accumulated lines.
while (<>)
{
# Add this line to the accumulated lines
push #lines, $_;
$tail = $.;
printf "# array: N = %d, last = $tail: %s", scalar(#lines), $_ if debug > 1;
if (m/$op/o)
{
# This line matches - set range to be printed
my $lo = $. - $before;
$tbp_1 = $lo if ($lo > $tbp_2);
$tbp_2 = $. + $after;
print "# $. MATCH: print range $tbp_1 .. $tbp_2\n" if debug;
}
# Print out any accumulated lines that need printing
# Leave $before lines in array.
print_leaving($before);
}
continue
{
if (eof)
{
# Print out any accumulated lines that need printing
print_leaving(0);
# Reset for next file
close ARGV;
$tbp_1 = 0;
$tbp_2 = 0;
$tail = 0;
#lines = ();
}
}
Pre-Perl Unix solution (using plain ed, sed, and sort - though it uses getopt which was not necessarily available back then):
#!/bin/ksh
#
# #(#)$Id: old.sgrep.sh,v 1.5 2007/09/15 22:15:43 jleffler Exp $
#
# Special grep
# Finds a pattern and prints lines either side of the pattern
# Line numbers are always produced by ed (substitute for grep),
# which allows us to eliminate duplicate lines cleanly. If the
# user did not ask for numbers, these are then stripped out.
#
# BUG: if the pattern occurs in in the first line or two and
# the number of lines to go back is larger than the line number,
# it fails dismally.
set -- `getopt "f:b:hn" "$#"`
case $# in
0) echo "Usage: $0 [-hn] [-f x] [-b y] pattern [files]" >&2
exit 1;;
esac
# Tab required - at least with sed (perl would be different)
# But then the whole problem would be different if implemented in Perl.
number="'s/^\\([0-9][0-9]*\\) /\\1:/'"
filename="'s%^%%'" # No-op for sed
f=3
b=3
nflag=no
hflag=no
while [ $# -gt 0 ]
do
case $1 in
-f) f=$2; shift 2;;
-b) b=$2; shift 2;;
-n) nflag=yes; shift;;
-h) hflag=yes; shift;;
--) shift; break;;
*) echo "Unknown option $1" >&2
exit 1;;
esac
done
pattern="${1:?'No pattern'}"
shift
case $# in
0) tmp=${TMPDIR:-/tmp}/`basename $0`.$$
trap "rm -f $tmp ; exit 1" 0
cat - >$tmp
set -- $tmp
sort="sort -t: -u +0n -1"
;;
*) filename="'s%^%'\$file:%"
sort="sort -t: -u +1n -2"
;;
esac
case $nflag in
yes) num_remove='s/[0-9][0-9]*://';;
no) num_remove='s/^//';;
esac
case $hflag in
yes) fileremove='s%^$file:%%';;
no) fileremove='s/^//';;
esac
for file in $*
do
echo "g/$pattern/.-${b},.+${f}n" |
ed - $file |
eval sed -e "$number" -e "$filename" |
$sort |
eval sed -e "$fileremove" -e "$num_remove"
done
rm -f $tmp
trap 0
exit 0
The shell version of sgrep was written in February 1989, and bug fixed in May 1989. It then remained unchanged except for an administrative change (SCCS to RCS transition) in 1997 until 2007, when I added the -h option. I switched to the Perl version in 2007.
http://thedailywtf.com/Articles/The_Complicator_0x27_s_Gloves.aspx
You can use sed to print specific lines, lets say you want line 20
sed '20 p' -n FILE_YOU_WANT_THE_LINE_FROM
Done.
-n prevents echoing lines from the file. The part in quotes is a sed rule to apply, it specifies that you want the rule to apply to line 20, and you want to print.
With GNU grep on Windows:
$ grep --context 3 FAIL output.log
$ grep --help | grep context
-B, --before-context=NUM print NUM lines of leading context
-A, --after-context=NUM print NUM lines of trailing context
-C, --context=NUM print NUM lines of output context
-NUM same as --context=NUM