I have converted one function from C language to Unix ksh script and below is the code
function greg_julian
{
typeset -i long_date=$1;
print "$scriptname ld $long_date" 1>&2; # Prints 20160121
set days -9999 0 31 59 90 120 151 181 212 243 273 304 334;
if [[ $long_date < 101 ]] then
return 0;
fi
typeset -i yy=$long_date/10000;
typeset -i mm=$long_date/100 % 100;
ymod=$(( $yy % 4 ))
if [[ $ymod -eq 0 && $mm -gt 2 && $yy -ne 0 ]] then
dm=1;
else
dm=0;
fi
echo $yy*1000 + ${days[$mm]} + $long_date % 100 + $dm;
}
I get an error as below. Tried removing space around %. but same error. I am Unable to figure out the error please help.
greg_julian[87]: %: is not an identifier
You can use an intermediate variable:
#!/bin/ksh
long_date=20160121
typeset -i yy=$long_date/10000;
typeset -i yymm=$long_date/100;
typeset -i mm=$yymm%100;
printf "%04d\n" $yy
printf "%02d\n" $mm
One way to calculate mathematical expressions in shell script is to use expr.
Can you, please, try it and make a feedback please?
Related
I am following this thread: https://stackoverflow.com/a/19742842/5057251
for typeset (or declare) in ZSH, not BASH.
#Declare (or typeset) an array of integers
#declare -ai int_array
typeset -ai int_array
int_array=(1 2 3)
echo "${int_array[#]}"
Then
# Attempt to change 1st element to string. (expect to fail)
int_array[1]="Should fail" || echo "error: ${LINENO}"
echo "${int_array[#]}"
Bash finds the error, gracefully reports error and lineno, prints:
1 2 3
But Zsh accepts, prints:
Should fail 2 3
Not sure why different.
There are two problems here:
In bash, and zsh, assigning a string to an integer variable causes that string to be evaluated as an arithmetic expression. Thus, this is not an error:
$ typeset -i foo
$ foo="bar"
If bar was a variable previously set to an arithmetic expression, then bar's expansion would be evaluated as such:
$ bar=10+2
$ typeset -i foo
$ foo="bar"
$ echo "$foo"
12
The error in your assignment, of course, is that there's no way to expand Should fail like that. If it were, say, Should - fail (an arithmetic expression subtracting the value of the two variables Should and fail, for example, it would still work:
$ foo="Should - fail"
$ echo "$foo"
0
The second problem is that nothing in the zsh docs indicate that -i may be set for an entire array, and so the -a in -ai is ignored:
bash-5.0$ typeset -ai foo
bash-5.0$ declare -p foo
declare -ai foo=([0]="0") # the previous value was retained in the array
vs zsh:
% typeset -ai foo
% foo[1]=10
% foo[2]=20
% declare -p foo
typeset -i foo=20 # treated as a normal variable, not array
What you're seeing is essentially int_array being redeclared as an array (without any qualifiers) when you do int_array=(1 2 3):
% foo=(1 2 3)
% declare -p foo
typeset -a foo=( 1 2 3 )
Using zsh typeset can produce a few possible outcomes:
- no errors, works (yeah!).
- errors, script fails (fix!).
- no errors, but unexpected behavior. (scratch head)
As an example of last category, this produces no errors, but the typeset -p reveals -i is ignored.
{
unset int_array
typeset -ia int_array
int_array=(1 2 3)
echo $? "-Point A"
typeset -p int_array
} always {
echo $? "-Point B"
typeset -p int_array
(( TRY_BLOCK_ERROR=0 ))
}
echo $? "-Point C"
echo "survived"
produces
0 -Point A
typeset -a int_array=( 1 2 3 )
0 -Point B
typeset -a int_array=( 1 2 3 )
0 -Point C
survived
The first line unsets int_array. The typeset command declares
int_array to be both an array and int, which is not what zsh allows. The next
line assigns int_array to a value. There is no error as the $? tells us,
but close examination of final typeset -p int_array reveals what actually
happened.
With a small change, we can produce errors and use the always block and
typeset -p to find more details.
{
unset int_array
typeset -ia int_array=(1 2 3) # error
echo $? "-Point A"
typeset -p int_array
} always {
echo $? "-Point B"
typeset -p int_array
(( TRY_BLOCK_ERROR=0 ))
}
echo $? "-Point C"
echo "survived"
040_declare_version2.sh:typeset:135: int_array: inconsistent type for assignment
1 -Point B
040_declare_version2.sh:typeset:140: no such variable: int_array
1 -Point C
survived
The only difference is int_array was given a value in the faulty typeset -ia statement.
This produces errors, and the script jumps to the always block.
The (( TRY_BLOCK_ERROR=0)) allows the script to continue
and not terminate, but the error is still reported at "Point C".
To check shell version:
$SHELL --version
zsh 5.4.2 (x86_64-ubuntu-linux-gnu)
I am trying to check if each line has a same length(or number of fields) in a file.
I am doing the following but it seems not to work.
NR==1 {length=NF}
NR>1 && NF!=length {print}
Can this be done by a one-liner awk? or a program is fine.
A sample of input would be:
12 34 54 56
12 89 34 33
12
29 56 42 42
My expected output would be "yes" or "no" if they have the same number of fields or not.
You could try this command which checks the number of fields in each line and compares it to the number of fields of the first line:
awk 'NR==1{a=NF; b=0} (NR>1 && NF!=a){print "No"; b=1; exit 1}END{if (b==0) print "Yes"}' test.txt
Checking is aborted in the first line whose number of fields is distinct from the first line of input.
For input
12 43 43
12 32
you will get "No"
Try:
awk 'BEGIN{a="yes"} last!="" && NF!=last{a="no"; exit} {last=NF} END{print a}' file
How it works
BEGIN{a="yes"}
This initializes the variable a to yes. (We assume all lines have the same number fields until proven otherwise.)
last!="" && NF!=last{a="no"; exit}
If last has been assigned a value and the number of fields on the current line is not the same as last, then set a to no and exit.
{last=NF}
Update last to the number of fields on the current line.
END{print a}
Before exiting, print a.
Examples
$ cat file1
2 34 54 56
12 89 34 33
12
29 56 42 42
$ awk 'BEGIN{a="yes"} last!="" && NF!=last{a="no"; exit} {last=NF} END{print a}' file1
no
$ cat file2
2 34 54 56
12 89 34 33
29 56 42 42
$ awk 'BEGIN{a="yes"} last!="" && NF!=last{a="no"; exit} {last=NF} END{print a}' file2
yes
I am assuming that you want to check fields of all lines, if they are equal or not if this is case then try following.
awk '
FNR==1{
value=NF
count++
next
}
{
count=NF==value?++count:count
}
END{
if(count==FNR){
print "All lines are of same fields"
}
else{
print "All lines are NOT of same fields."
}
}
' Input_file
Additional stuff(only if require): In case you want to print contents of file whose all lines are having same fields along with yes or all are same fields in file message in output then try following.
awk '
{
val=val?val ORS $0:$0
}
FNR==1{
value=NF
count++
next
}
{
count=NF==value?++count:count
}
END{
if(count==FNR){
print "All lines are of same fields" ORS val
}
else{
print "All lines are NOT of same fields."
}
}
' Input_file
this should do
$ awk 'NR==1{p=NF} p!=NF{s=1; exit} END{print s?"No":"Yes"}' file
however, setting the exit status would be better if this will be part of a workflow.
Since equivalence has transitive property, there is no need to keep NF other than the first line; setting 0 as your success value doesn't require initialization to default value.
An efficient even fields shell function, using sed to construct a regex, (based on the first line of input), to feed to GNU grep, which looks for field length mismatches:
# Usage: ef filename
ef() { sed '1s/[^ ]*/[^ ]*/g;q' "$1" | grep -v -m 1 -q -f - "$1" \
&& echo no || echo yes ; }
For files with uneven fields grep -m 1 quits after the first non-uniform line -- so if the file is a million lines long, but the mismatch occurs on line #2, grep only needs to read two lines, not a million. On the other hand, if there's no mismatch grep would have to read a million lines.
I have several fixed files and need to quickly scan them for occurrence of letter L on field 159 and print.
Example:
123456ABCDEF23253657LA00000000MMMKSNS
123456ABCDEF23253657BA00000000MMMKSNS
123456ABCDEF23253657LA00000000MMMKSNS
123456ABCDEF23253657LA00000000MMMKSNS
12345 3253657LA00000000MMMKSNS
I tried grep -E '^.{21} L' but doesn't find for a 180 fixed length grep -E '^.{159} L'
With awk you can print the substring. See nim's answer for syntax.
while read line; \
do echo "$line" | awk '{print substr($0,21,1)}'| \
grep -q "L"; [ $? -eq 0 ] && \
echo "$line"; \
done < "${file}"
Output:
123456ABCDEF23253657LA00000000MMMKSNS
123456ABCDEF23253657LA00000000MMMKSNS
123456ABCDEF23253657LA00000000MMMKSNS
12345 3253657LA00000000MMMKSNS
You were very close,
echo "123456ABCDEF23253657LA00000000MMMKSNS
123456ABCDEF23253657BA00000000MMMKSNS
123456ABCDEF23253657LA00000000MMMKSNS
123456ABCDEF23253657LA00000000MMMKSNS
12345 3253657LA00000000MMMKSNS" | grep -E '^.{20}L'
output
123456ABCDEF23253657LA00000000MMMKSNS
123456ABCDEF23253657LA00000000MMMKSNS
123456ABCDEF23253657LA00000000MMMKSNS
12345 3253657LA00000000MMMKSNS
Note that having a space between {20} L throws off the position, AND that as your L is in Position 21, you skip 20 chars.
IHTH
for var in 1 2 3 4 5 6 7 8 9 10
do
echo $var
export lower_bound=$((($var - 1) * 10))
echo $lower_bound
export upper_bound=$(($var * 10))
echo $upper_bound
awk '$2 >= $(($lower_bound)) && $2 < $(($upper_bound))' shubham_test.txt > file_$var.txt
#awk -v var1="$lower_bound" -v var2="$upper_bound" '{$2 >= $var1 && $2 < $var2}' shubham_test.txt > file_$var.txt
done
I am trying to split a file based on the value of a field in a text file in unix. The awk works when I pass hardcoded values to caompare but does not work properly if I am passing varaibles for comparison (upper_bound and lower_bound).
I looked it up and even replaced the awk command with the following:
awk -v var1="$lower_bound" -v var2="$upper_bound" '{$2 >= $var1 && $2 < $var2}' shubham_test.txt > file_$var.txt
so that it takes arguments. But this is not working either. Can anyone help?
Your second format should work, but without the $ for the variable names - so :
awk -v var1="$lower_bound" -v var2="$upper_bound" '{$2 >= var1 && $2 < var2}' shubham_test.txt
Your first version fails because the single quotes mean the unix shell does not evaluate the $lower_bound variables, but instead is passing the entire literal string to awk. If you're writing shell scripts, you must know how to use single-, double- and back-quotes - take a look at http://www.tutorialspoint.com/unix/unix-quoting-mechanisms.htm for example.
I am working on a UNIX box, and trying to run an application, which gives some debug logs to the standard output. I have redirected this output to a log file, but now wish to get the lines where the error is being shown.
My problem here is that a simple
cat output.log | grep FAIL
does not help out. As this shows only the lines which have FAIL in them. I want some more information along with this. Like the 2-3 lines above this line with FAIL. Is there any way to do this via a simple shell command? I would like to have a single command line (can have pipes) to do the above.
grep -C 3 FAIL output.log
Note that this also gets rid of the useless use of cat (UUOC).
grep -A $NUM
This will print $NUM lines of trailing context after matches.
-B $NUM prints leading context.
man grep is your best friend.
So in your case:
cat log | grep -A 3 -B 3 FAIL
I have two implementations of what I call sgrep, one in Perl, one using just pre-Perl (pre-GNU) standard Unix commands. If you've got GNU grep, you've no particular need of these. It would be more complex to deal with forwards and backwards context searches, but that might be a useful exercise.
Perl solution:
#!/usr/perl/v5.8.8/bin/perl -w
#
# #(#)$Id: sgrep.pl,v 1.6 2007/09/18 22:55:20 jleffler Exp $
#
# Perl-based SGREP (special grep) command
#
# Print lines around the line that matches (by default, 3 before and 3 after).
# By default, include file names if more than one file to search.
#
# Options:
# -b n1 Print n1 lines before match
# -f n2 Print n2 lines following match
# -n Print line numbers
# -h Do not print file names
# -H Do print file names
use strict;
use constant debug => 0;
use Getopt::Std;
my(%opts);
sub usage
{
print STDERR "Usage: $0 [-hnH] [-b n1] [-f n2] pattern [file ...]\n";
exit 1;
}
usage unless getopts('hnf:b:H', \%opts);
usage unless #ARGV >= 1;
if ($opts{h} && $opts{H})
{
print STDERR "$0: mutually exclusive options -h and -H specified\n";
exit 1;
}
my $op = shift;
print "# regex = $op\n" if debug;
# print file names if -h omitted and more than one argument
$opts{F} = (defined $opts{H} || (!defined $opts{h} and scalar #ARGV > 1)) ? 1 : 0;
$opts{n} = 0 unless defined $opts{n};
my $before = (defined $opts{b}) ? $opts{b} + 0 : 3;
my $after = (defined $opts{f}) ? $opts{f} + 0 : 3;
print "# before = $before; after = $after\n" if debug;
my #lines = (); # Accumulated lines
my $tail = 0; # Line number of last line in list
my $tbp_1 = 0; # First line to be printed
my $tbp_2 = 0; # Last line to be printed
# Print lines from #lines in the range $tbp_1 .. $tbp_2,
# leaving $leave lines in the array for future use.
sub print_leaving
{
my ($leave) = #_;
while (scalar(#lines) > $leave)
{
my $line = shift #lines;
my $curr = $tail - scalar(#lines);
if ($tbp_1 <= $curr && $curr <= $tbp_2)
{
print "$ARGV:" if $opts{F};
print "$curr:" if $opts{n};
print $line;
}
}
}
# General logic:
# Accumulate each line at end of #lines.
# ** If current line matches, record range that needs printing
# ** When the line array contains enough lines, pop line off front and,
# if it needs printing, print it.
# At end of file, empty line array, printing requisite accumulated lines.
while (<>)
{
# Add this line to the accumulated lines
push #lines, $_;
$tail = $.;
printf "# array: N = %d, last = $tail: %s", scalar(#lines), $_ if debug > 1;
if (m/$op/o)
{
# This line matches - set range to be printed
my $lo = $. - $before;
$tbp_1 = $lo if ($lo > $tbp_2);
$tbp_2 = $. + $after;
print "# $. MATCH: print range $tbp_1 .. $tbp_2\n" if debug;
}
# Print out any accumulated lines that need printing
# Leave $before lines in array.
print_leaving($before);
}
continue
{
if (eof)
{
# Print out any accumulated lines that need printing
print_leaving(0);
# Reset for next file
close ARGV;
$tbp_1 = 0;
$tbp_2 = 0;
$tail = 0;
#lines = ();
}
}
Pre-Perl Unix solution (using plain ed, sed, and sort - though it uses getopt which was not necessarily available back then):
#!/bin/ksh
#
# #(#)$Id: old.sgrep.sh,v 1.5 2007/09/15 22:15:43 jleffler Exp $
#
# Special grep
# Finds a pattern and prints lines either side of the pattern
# Line numbers are always produced by ed (substitute for grep),
# which allows us to eliminate duplicate lines cleanly. If the
# user did not ask for numbers, these are then stripped out.
#
# BUG: if the pattern occurs in in the first line or two and
# the number of lines to go back is larger than the line number,
# it fails dismally.
set -- `getopt "f:b:hn" "$#"`
case $# in
0) echo "Usage: $0 [-hn] [-f x] [-b y] pattern [files]" >&2
exit 1;;
esac
# Tab required - at least with sed (perl would be different)
# But then the whole problem would be different if implemented in Perl.
number="'s/^\\([0-9][0-9]*\\) /\\1:/'"
filename="'s%^%%'" # No-op for sed
f=3
b=3
nflag=no
hflag=no
while [ $# -gt 0 ]
do
case $1 in
-f) f=$2; shift 2;;
-b) b=$2; shift 2;;
-n) nflag=yes; shift;;
-h) hflag=yes; shift;;
--) shift; break;;
*) echo "Unknown option $1" >&2
exit 1;;
esac
done
pattern="${1:?'No pattern'}"
shift
case $# in
0) tmp=${TMPDIR:-/tmp}/`basename $0`.$$
trap "rm -f $tmp ; exit 1" 0
cat - >$tmp
set -- $tmp
sort="sort -t: -u +0n -1"
;;
*) filename="'s%^%'\$file:%"
sort="sort -t: -u +1n -2"
;;
esac
case $nflag in
yes) num_remove='s/[0-9][0-9]*://';;
no) num_remove='s/^//';;
esac
case $hflag in
yes) fileremove='s%^$file:%%';;
no) fileremove='s/^//';;
esac
for file in $*
do
echo "g/$pattern/.-${b},.+${f}n" |
ed - $file |
eval sed -e "$number" -e "$filename" |
$sort |
eval sed -e "$fileremove" -e "$num_remove"
done
rm -f $tmp
trap 0
exit 0
The shell version of sgrep was written in February 1989, and bug fixed in May 1989. It then remained unchanged except for an administrative change (SCCS to RCS transition) in 1997 until 2007, when I added the -h option. I switched to the Perl version in 2007.
http://thedailywtf.com/Articles/The_Complicator_0x27_s_Gloves.aspx
You can use sed to print specific lines, lets say you want line 20
sed '20 p' -n FILE_YOU_WANT_THE_LINE_FROM
Done.
-n prevents echoing lines from the file. The part in quotes is a sed rule to apply, it specifies that you want the rule to apply to line 20, and you want to print.
With GNU grep on Windows:
$ grep --context 3 FAIL output.log
$ grep --help | grep context
-B, --before-context=NUM print NUM lines of leading context
-A, --after-context=NUM print NUM lines of trailing context
-C, --context=NUM print NUM lines of output context
-NUM same as --context=NUM