How to use a Runtime variable in grep command in Unix - unix
I have a Time.sh file which has the code to store the StartTime and EndTime, entered by the user in runtime. The code in Time.sh fie is
read -p "Enter Start Time : " StartTime
read -p "Enter End Time : " EndTime
echo "Start and End Time are:" $StartTime, $EndTime
The following is the code I used to get the StartTime and EndTime in Runtime.
$ chmod +x Time.sh
$ ./Time.sh
Now I have to use this values in the two variables and print the data in the file "Test.log". The data in Test.log file is as follows,
May 10 01:07:05 server1 user:err|error IIB[78658909]: error text
May 10 01:07:06 server1 user:notice superstp[78653958]: infotext notice text
May 10 01:07:07 server1 user:info syslog: infotext
May 10 01:07:08 server1 user:err|error IIB[78658909]: error text
May 10 01:07:09 server1 user:warn|warning IBM Java[78650709]: warning text
May 10 01:07:10 server1 user:info syslog: infotext
May 10 01:07:11 server1 user:err|error IIB[78658909]: error text
May 10 01:07:12 server1 user:warn|warning IBM Java[78650709]: warning text
May 10 01:07:13 server1 user:notice superstp[78653958]: infotext notice text
May 10 01:07:14 server1 user:info syslog: infotext
May 10 01:07:15 server1 user:warn|warning IBM Java[78650709]: warning text
I have used the following code,
grep < "$EndTime" Test.log
But it is not working. Can anyone help to use both the variables and get the data within the StartTime and EndTime. Thanks in Advance.
The following should help:
BEGIN { Found = 0; EndFound = 0;}
{
if($0 ~ Start)
{
Found = 1;
}
if($0 ~ End)
{
EndFound = 1;
}
else
{
if(EndFound == 1)
{
Found = 0;
}
}
if(Found)
{
print $0;
}
}
It should be run as awk -v Start="01:07:06" -v End="01:07:10" -f program data_file, where program contains the code above and "01:07:06" and "01:07:10" start and end times respectively.
This is an example only. Please change fit it for your exact requirements.
Related
How to exclude parent Unix processes from grepped output from ps
I have got a file of pids and am using ps -f to get information about the pids. Here is an example.. ps -eaf | grep -f myfilename myuser 14216 14215 0 10:00 ? 00:00:00 /usr/bin/ksh /home/myScript.ksh myuser 14286 14216 0 10:00 ? 00:00:00 /usr/bin/ksh /home/myScript.ksh where myfilename contains only 14216. I've got a tiny problem where the output is giving me parent process id's as well as the child. I want to exclude the line for the parent process id. Does anyone know how I could modify my command to exclude parent process keeping in mind that I could have many process id's in my input file?
Hard to do with just grep but easy to do with awk.
Invoke the awk script below from the following command:
ps -eaf | awk -f script.awk myfilename -
Here's the script:
# process the first file on the command line (aka myfilename)
# this is the list of pids
ARGIND == 1 {
pids[$0] = 1
}
# second and subsequent files ("-"/stdin in the example)
ARGIND > 1 {
# is column 2 of the ps -eaf output [i.e.] the pid in the list of desired
# pids? -- if so, print the entire line
if ($2 in pids)
printf("%s\n",$0)
}
UPDATE:
When using GNU awk (gawk), the following may be ignored. For other [obsolete] versions, insert the following code at the top:
# work around old, obsolete versions
ARGIND == 0 {
defective_awk_flag = 1
}
defective_awk_flag != 0 {
if (FILENAME != defective_awk_file) {
defective_awk_file = FILENAME
ARGIND += 1
}
}
UPDATE #2:
The above is all fine. Just for fun, here's an alternate way to do the same thing with perl. One of the advantages is that everything can be contained in the script and no pipeline is necessary.
Invoke the script via:
./script.pl myfilename
And, here's script.pl. Note: I don't write idiomatic perl. My style is more akin to what one would expect to see in other languages like C, javascript, etc.:
#!/usr/bin/perl
master(#ARGV);
exit(0);
# master -- master control
sub master
{
my(#argv) = #_;
my($xfsrc);
my($pidfile);
my($buf);
# NOTE: "chomp" is a perl function that strips newlines
# get filename with list of pids (e.g. myfilename)
$pidfile = shift(#argv);
open($xfsrc,"<$pidfile") ||
die("master: unable to open '$pidfile' -- $!\n");
# create an associative array (a 'hash" in perl parlance) of the desired
# pid numbers
while ($pid = <$xfsrc>) {
chomp($pid);
$pid_desired{$pid} = 1;
}
close($xfsrc);
# run the 'ps' command and capture its output into an array
#pslist = (`ps -eaf`);
# process the command output, line-by-line
foreach $buf (#pslist) {
chomp($buf);
# the pid number we want is in the second column
(undef,$pid) = split(" ",$buf);
# print the line if the pid is one of the ones we want
print($buf,"\n")
if ($pid_desired{$pid});
}
}
Use this command: ps -eaf | grep -f myfilename | grep -v grep | grep -f myfilename
Unix C-Shell: ?Need help for the this task
i have given a task to make a C-shell script. I have list of ip address and device name respectively. For example;
cal 1 : 100.21.25.10
cal 2 : 100.21.25.11
cal 3 : 100.21.25.12
cal 4 : 100.21.25.14
and so on...
Based on this ip and device name, i need to rsh the ip address and get the disk free of the device. The result of disk free will be save to a log. the details of the log will be have device name need to be housekeep. My idea is:
declared array :
set device =( cal1 cal2 cal3)
set ip = (100.21.25.10 100.21.25.11 100.21.25.12 100.21.25.14)
set highspace = 90
foreach data($ip)
set space = rsh $ip df -k
if (${space} >= ${highspace}) then
echo "Please Housekeep $device:" >> $device.log
endif
is this gonna work? Or do you guys have better idea? Thanks.
The C shell should never be used anymore. Neither should rsh; we have ssh now.
Your task in Bourne shell:
#! /bin/sh
highspace=90
fs_to_watch=/path/to/filesystem/that/fills/up
exec 0<"$1"
while read cal calno colon addr; do
space=$(ssh "$addr" df -k "$fs_to_watch" |
awk 'NR > 1 { sub(/%$/, "", $5); print $5 }')
if [ "$space" -gt "$highspace" ]; then
echo "Please Housekeep Cal-$calno"
fi
done
To improve Calculate Number of Days Command
Would like to generate report, which calculate the number of days, the material is in the warehouse.
The number of days is the difference between date ($3 field) the material comes in and
against (01 OCT 2014) manual feed date.
Input.csv
Des11,Material,DateIN,Des22,Des33,MRP,Des44,Des55,Des66,Location,Des77,Des88
aa,xxx,19-AUG-14.08:08:01,cc,dd,x20,ee,ff,gg,XX128,hh,jj
aa,xxx,19-AUG-14.08:08:01,cc,dd,x20,ee,ff,gg,XX128,hh,jj
aa,yyy,13-JUN-14.09:06:08,cc,dd,x20,ee,ff,gg,XX128,hh,jj
aa,yyy,13-JUN-14.09:06:08,cc,dd,x20,ee,ff,gg,XX128,hh,jj
aa,yyy,05-FEB-14.09:02:09,cc,dd,x20,ee,ff,gg,YY250,hh,jj
aa,yyy,05-FEB-14.09:02:09,cc,dd,y35,ee,ff,gg,YY250,hh,jj
aa,zzz,05-FEB-14.09:02:09,cc,dd,y35,ee,ff,gg,YY250,hh,jj
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj
Currently i am using below command to popualte Ageing - No of days at $13 field ( thanks to gboffi)
awk -F, 'NR>0 {date=$3;
gsub("[-.]"," ",date);
printf $0 ",";system("date --date=\"" date "\" +%s")}
' Input.csv | awk -F, -v OFS=, -v now=`date --date="01 OCT 2014 " +%s` '
NR>0 {$13=now-$13; $13=$13/24/3600;print $0}' >Op_Step11.csv
while using the above command in Cygwin (windows), it is taking 50 minutes for 1 Lac (1,00,000) rows of sample input.
Since my actual input file contains 25 million rows of lines , it seems that the script will take couple of days ,
Looking for your suggestions to improve the command and advice !!!
Expected Output:
Des11,Material,DateIN,Des22,Des33,MRP,Des44,Des55,Des66,Location,Des77,Des88,Ageing-NoOfDays
aa,xxx,19-AUG-14.08:08:01,cc,dd,x20,ee,ff,gg,XX128,hh,jj,42.6611
aa,xxx,19-AUG-14.08:08:01,cc,dd,x20,ee,ff,gg,XX128,hh,jj,42.6611
aa,yyy,13-JUN-14.09:06:08,cc,dd,x20,ee,ff,gg,XX128,hh,jj,109.621
aa,yyy,13-JUN-14.09:06:08,cc,dd,x20,ee,ff,gg,XX128,hh,jj,109.621
aa,yyy,05-FEB-14.09:02:09,cc,dd,x20,ee,ff,gg,YY250,hh,jj,237.624
aa,yyy,05-FEB-14.09:02:09,cc,dd,y35,ee,ff,gg,YY250,hh,jj,237.624
aa,zzz,05-FEB-14.09:02:09,cc,dd,y35,ee,ff,gg,YY250,hh,jj,237.624
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj,476.787
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj,476.787
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj,476.787
I don't have the access to change the input format and dont have perl & python access.
Update#3:
BEGIN{ FS=OFS=","}
{
t1=$3
t2="01-OCT-14.00:00:00"
print $0,(cvttime(t2) - cvttime(t1))/24/3600
}
function cvttime(t, a) {
split(t,a,"[-.:]")
match("JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC",a[2])
a[2] = sprintf("%02d",(RSTART+2)/3)
return( mktime("20"a[3]" "a[2]" "a[1]" "a[4]" "a[5]" "a[6]) )
}
Since you are on cygwin you are using GNU awk which has it's own built-in time functions and so you do not need to be trying to use the shell date command. Just tweak this old command I had lying around to suit your input and output format:
function cvttime(t, a) {
split(t,a,"[/:]")
match("JanFebMarAprMayJunJulAugSepOctNovDec",a[2])
a[2] = sprintf("%02d",(RSTART+2)/3)
return( mktime(a[3]" "a[2]" "a[1]" "a[4]" "a[5]" "a[6]) )
}
BEGIN{
t1="01/Dec/2005:00:04:42"
t2="01/Dec/2005:17:14:12"
print cvttime(t2) - cvttime(t1)
}
It uses GNU awk for time functions, see http://www.gnu.org/software/gawk/manual/gawk.html#Time-Functions
Here is an example in Perl:
use feature qw(say);
use strict;
use warnings;
use Text::CSV;
use Time::Piece;
my $csv = Text::CSV->new;
my $te = Time::Piece->strptime('01-OCT-14', '%d-%b-%y');
my $fn = 'Input.csv';
open (my $fh, '<', $fn) or die "Could not open file '$fn': $!\n";
chomp(my $head = <$fh>);
say "$head,Ageing-NoOfDays";
while (my $line = <$fh>) {
chomp $line;
if ($csv->parse($line)) {
my $t = ($csv->fields())[2];
my $tp = Time::Piece->strptime($t, '%d-%b-%y.%T');
my $s = $te - $tp;
say "$line," . $s->days;
} else {
warn "Line could not be parsed: $line\n";
}
}
close($fh);
Capture and check bteq return code in Unix
I'm developing a script which in turn invokes several other scripts (.ksh). And basically when one of them fail they shouldn't proceed to the next one. So I tried checking for the return code in one script that involves bteq (Basic Teradata Query) session. Please find below the scenario:
bteq <<EOF!
.run file ${TGTRUNFILEN} ;
.maxerror 1;
.set width 245;
...
...
sel * from table ;
.if ACTIVITYCOUNT <> 0 then .GOTO QUIT
.os mail command "error msg"
exit 1;
.LABEL QUIT
.quit;
EOF!
echo $rcode
rcode=$?
if [[ $rcode != 0 ]]
then
echo "$0: Insufficient Perm Space : username " >&2
exit 4
fi
Here,the script fails and I can see the log saying failed with return code 1, but why isn't the text "$0:Insufficient Perm Space : Username" displayed. I think it exits the entire script, but I need this fixed somehow.
Can someone kindly help me on this?
Hi Thanks a ton for responding.I found a way to overcome this.I just added 'set' command like this. set +e bteq <<EOF! ... ... EOF! rcode=$? set -e Works fine for me. Cheers
Time command equivalent in PowerShell
What is the flow of execution of the time command in detail? I have a user created function in PowerShell, which will compute the time for execution of the command in the following way. It will open the new PowerShell window. It will execute the command. It will close the PowerShell window. It will get the the different execution times using the GetProcessTimes function function. Is the "time command" in Unix also calculated in the same way?
The Measure-Command cmdlet is your friend.
PS> Measure-Command -Expression {dir}
You could also get execution time from the command history (last executed command in this example):
$h = Get-History -Count 1
$h.EndExecutionTime - $h.StartExecutionTime
I've been doing this:
Time {npm --version ; node --version}
With this function, which you can put in your $profile file:
function Time([scriptblock]$scriptblock, $name)
{
<#
.SYNOPSIS
Run the given scriptblock, and say how long it took at the end.
.DESCRIPTION
.PARAMETER scriptBlock
A single computer name or an array of computer names. You mayalso provide IP addresses.
.PARAMETER name
Use this for long scriptBlocks to avoid quoting the entire script block in the final output line
.EXAMPLE
time { ls -recurse}
.EXAMPLE
time { ls -recurse} "All the things"
#>
if (!$stopWatch)
{
$script:stopWatch = new-object System.Diagnostics.StopWatch
}
$stopWatch.Reset()
$stopWatch.Start()
. $scriptblock
$stopWatch.Stop()
if ($name -eq $null) {
$name = "$scriptblock"
}
"Execution time: $($stopWatch.ElapsedMilliseconds) ms for $name"
}
Measure-Command works, but it swallows the stdout of the command being run. (Also see Timing a command's execution in PowerShell)
If you need to measure the time taken by something, you can follow this blog entry. Basically, it suggest to use the .NET StopWatch class: $sw = [System.Diagnostics.StopWatch]::startNew() # The code you measure $sw.Stop() Write-Host $sw.Elapsed