how to display output in the following format in unix - unix
Hi my data looks in the below format in unix file
BENEFITS-T_FACT_DEPNDT_COVRG-230
BENEFITS-T_FACT_DEPNDT_COVRG-290
BENEFITS-T_FACT_ELECTN-0
BENEFITS-T_FACT_ELECTN-39092
HR-T_DIM_CLNT-0
HR-T_DIM_CLNT-98
HR-T_DIM_CMPNY-10
HR-T_DIM_CMPNY-45
I need to display the output like this:
domain -tablename- before load cnt: 230 after load cnt :290
ex : BENEFITS - T_FACT_DEPNDT_COVRG -before load cnt: 230 after load cnt :290
HR -T_DIM_CLNT -before load cnt: 0 after load cnt :98
Can someone please suggest an answer in unix ???
Is this enough?
awk -F "-" '{if(NR%2){printf $1 " " $2 " -before load cnt: " $3}else{printf " after load cnt: " $3 "\n"}}' filename
Related
How to use a Runtime variable in grep command in Unix
I have a Time.sh file which has the code to store the StartTime and EndTime, entered by the user in runtime. The code in Time.sh fie is read -p "Enter Start Time : " StartTime read -p "Enter End Time : " EndTime echo "Start and End Time are:" $StartTime, $EndTime The following is the code I used to get the StartTime and EndTime in Runtime. $ chmod +x Time.sh $ ./Time.sh Now I have to use this values in the two variables and print the data in the file "Test.log". The data in Test.log file is as follows, May 10 01:07:05 server1 user:err|error IIB[78658909]: error text May 10 01:07:06 server1 user:notice superstp[78653958]: infotext notice text May 10 01:07:07 server1 user:info syslog: infotext May 10 01:07:08 server1 user:err|error IIB[78658909]: error text May 10 01:07:09 server1 user:warn|warning IBM Java[78650709]: warning text May 10 01:07:10 server1 user:info syslog: infotext May 10 01:07:11 server1 user:err|error IIB[78658909]: error text May 10 01:07:12 server1 user:warn|warning IBM Java[78650709]: warning text May 10 01:07:13 server1 user:notice superstp[78653958]: infotext notice text May 10 01:07:14 server1 user:info syslog: infotext May 10 01:07:15 server1 user:warn|warning IBM Java[78650709]: warning text I have used the following code, grep < "$EndTime" Test.log But it is not working. Can anyone help to use both the variables and get the data within the StartTime and EndTime. Thanks in Advance.
The following should help:
BEGIN { Found = 0; EndFound = 0;}
{
if($0 ~ Start)
{
Found = 1;
}
if($0 ~ End)
{
EndFound = 1;
}
else
{
if(EndFound == 1)
{
Found = 0;
}
}
if(Found)
{
print $0;
}
}
It should be run as awk -v Start="01:07:06" -v End="01:07:10" -f program data_file, where program contains the code above and "01:07:06" and "01:07:10" start and end times respectively.
This is an example only. Please change fit it for your exact requirements.
How to remove newline or 'ENTER" in file by using Unix command
I am getting input data in file with "|" delimeter but some of records breaking and coming as two records . below is the example , for most of description fields data is coming like this only, actually these are entering through web when user press enter while adding comment it going to newline , can you please let me know how to handle this. I want this record in single line, "2016-03-03 22:26:20|0|I|NOT SET ||||||||||||||||2015-02-12-04.34.38.734657|2015-02-13| |0|METER FAILURE >30 DAYS" " |259000-056608 |TRBLRPRT|BDMTRRPL| || |0||| "
echo "$yourdata" | tr -d '\r\n' Or cat filename.txt | tr -d '\r\n' To update file: datastr=$(cat filename.txt | tr -d '\r\n') then echo -n "$datastr" > filename.txt Hope it helps
This substitutes a space for carriage returns or line feeds cat test.txt | tr ["\n","\r"] ' '
HP-Unix: C-shell:Disk space checking
I have 10 devices that using hp-ux and i want to check the disk space in each devices.
my requirement is if the space more than 90%, the info of device ans space will be save to a log.
this is list of device and ip address which i set as variable ipadd:
lo1 100.45.32.43
lot2 100.45.32.44
lot3 100.45.32.44
lot4 100.45.32.45
lot5 100.45.32.46
and so on..
This is my script so far :
#!/bin/csh -f
set ipaddress = (`awk '{print $2}' "ipadd"`)
set device = (`awk '{print $1}' "ipadd"`)
# j = 1
while ($j <= $#ipaddress)
echo $ipaddress
set i = 90 # Threshold set at 90%
set max = 100
while ($i <= $max)
rsh $ipaddress[$j] bdf | grep /dev/vg00 | grep $i% \
|awk '{ file=substr($6,index($6,"/") + 1,length($6)); print "WARNING: $device[$j]:/" file " has reached " $5 ". Perform HouseKeeping IMMEDIATELY..." >> "/scripts/space." file ".file"}'
# i++
end
# j++
end
The output after bdf:
/dev/vg00/lvol2 15300207 10924582 28566314 79% /
/dev/vg00/lvol4 42529 23786 25510 55% /stand
The output at terminal after exec the script:
100.45.32.43
100.45.32.44
The output at .file:
WARNING: $device[$j]:/ has reached 79%. Perform HouseKeeping IMMEDIATELY...
My question is, is it my looping have something wrong cause only iterates one time only because my .file output only show one device only?
And why $device[$j] not come out in .file output?
or awk have problem?
Thank you for the advice.
Your code tested for each possible percentage between 90 and 100.
Persumably, you'd be OK with code that checks once, and asks 'is device percent greater than 90%'?. So then you don't need the inner loop at all, and you make only 1 connection per machine, try
#!/bin/csh -f
set ipaddress = (`awk '{print $2}' "ipadd"`)
set device = (`awk '{print $1}' "ipadd"`)
# j = 1
set i = 90 # Threshold set at 90%
while ($j <= $#ipaddress)
echo $ipaddress
echo "#dbg: ipaddress[$j]=${ibpaddress[$j]}"
rsh $ipaddress[$j] bdf \
| awk -v thresh="$i" -v dev="$device[$j]" \
'/\/dev\/vg00/ { \
sub(/%/,"",$5) \
if ($5 > thresh) { \
file=substr($6,index($6,"/") + 1,length($6)) \
print "WARNING: " dev ":/" file " has reached " $5 ". Perform HouseKeeping IMMEDIATELY..." >> "/scripts/space." file ".file" \
}\
}'
# j++
end
Sorry, but I don't have a csh available to dbl-chk for syntax errors.
So here is a one liner that we determined worked in your environment.
rsh $ipaddress[$j] bdf | nawk -v thresh="$i" -v dev="$device[$j]" '/\/dev\/vg00/ { sub(/%/,"",$5) ; if ($5 > thresh) { file=substr($6,index($6,"/") + 1,length($6));print "#dbg:file="file; print "WARNING: " dev ":/" file " has reached " $5 ". Perform HouseKeeping IMMEDIATELY..." >> "/scripts/space.file.TMP" } }'
I don't have a system with bdf available. Change the two references to $5 in the sub() and if test to match the field-number of the output that has the percentage you want to test.
Note that -v var="value" is the standard way to pass a variable value from the shell to an awk script that is enclosed in single-quotes.
Be careful that any '\' chars at the end of a line are the last chars, no trailing space or tabs, or you'll get an indecipherable error msg. ;-)
IHTH
To improve Calculate Number of Days Command
Would like to generate report, which calculate the number of days, the material is in the warehouse.
The number of days is the difference between date ($3 field) the material comes in and
against (01 OCT 2014) manual feed date.
Input.csv
Des11,Material,DateIN,Des22,Des33,MRP,Des44,Des55,Des66,Location,Des77,Des88
aa,xxx,19-AUG-14.08:08:01,cc,dd,x20,ee,ff,gg,XX128,hh,jj
aa,xxx,19-AUG-14.08:08:01,cc,dd,x20,ee,ff,gg,XX128,hh,jj
aa,yyy,13-JUN-14.09:06:08,cc,dd,x20,ee,ff,gg,XX128,hh,jj
aa,yyy,13-JUN-14.09:06:08,cc,dd,x20,ee,ff,gg,XX128,hh,jj
aa,yyy,05-FEB-14.09:02:09,cc,dd,x20,ee,ff,gg,YY250,hh,jj
aa,yyy,05-FEB-14.09:02:09,cc,dd,y35,ee,ff,gg,YY250,hh,jj
aa,zzz,05-FEB-14.09:02:09,cc,dd,y35,ee,ff,gg,YY250,hh,jj
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj
Currently i am using below command to popualte Ageing - No of days at $13 field ( thanks to gboffi)
awk -F, 'NR>0 {date=$3;
gsub("[-.]"," ",date);
printf $0 ",";system("date --date=\"" date "\" +%s")}
' Input.csv | awk -F, -v OFS=, -v now=`date --date="01 OCT 2014 " +%s` '
NR>0 {$13=now-$13; $13=$13/24/3600;print $0}' >Op_Step11.csv
while using the above command in Cygwin (windows), it is taking 50 minutes for 1 Lac (1,00,000) rows of sample input.
Since my actual input file contains 25 million rows of lines , it seems that the script will take couple of days ,
Looking for your suggestions to improve the command and advice !!!
Expected Output:
Des11,Material,DateIN,Des22,Des33,MRP,Des44,Des55,Des66,Location,Des77,Des88,Ageing-NoOfDays
aa,xxx,19-AUG-14.08:08:01,cc,dd,x20,ee,ff,gg,XX128,hh,jj,42.6611
aa,xxx,19-AUG-14.08:08:01,cc,dd,x20,ee,ff,gg,XX128,hh,jj,42.6611
aa,yyy,13-JUN-14.09:06:08,cc,dd,x20,ee,ff,gg,XX128,hh,jj,109.621
aa,yyy,13-JUN-14.09:06:08,cc,dd,x20,ee,ff,gg,XX128,hh,jj,109.621
aa,yyy,05-FEB-14.09:02:09,cc,dd,x20,ee,ff,gg,YY250,hh,jj,237.624
aa,yyy,05-FEB-14.09:02:09,cc,dd,y35,ee,ff,gg,YY250,hh,jj,237.624
aa,zzz,05-FEB-14.09:02:09,cc,dd,y35,ee,ff,gg,YY250,hh,jj,237.624
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj,476.787
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj,476.787
aa,zzz,11-JUN-13.05:06:17,cc,dd,y35,ee,ff,gg,YY250,hh,jj,476.787
I don't have the access to change the input format and dont have perl & python access.
Update#3:
BEGIN{ FS=OFS=","}
{
t1=$3
t2="01-OCT-14.00:00:00"
print $0,(cvttime(t2) - cvttime(t1))/24/3600
}
function cvttime(t, a) {
split(t,a,"[-.:]")
match("JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC",a[2])
a[2] = sprintf("%02d",(RSTART+2)/3)
return( mktime("20"a[3]" "a[2]" "a[1]" "a[4]" "a[5]" "a[6]) )
}
Since you are on cygwin you are using GNU awk which has it's own built-in time functions and so you do not need to be trying to use the shell date command. Just tweak this old command I had lying around to suit your input and output format:
function cvttime(t, a) {
split(t,a,"[/:]")
match("JanFebMarAprMayJunJulAugSepOctNovDec",a[2])
a[2] = sprintf("%02d",(RSTART+2)/3)
return( mktime(a[3]" "a[2]" "a[1]" "a[4]" "a[5]" "a[6]) )
}
BEGIN{
t1="01/Dec/2005:00:04:42"
t2="01/Dec/2005:17:14:12"
print cvttime(t2) - cvttime(t1)
}
It uses GNU awk for time functions, see http://www.gnu.org/software/gawk/manual/gawk.html#Time-Functions
Here is an example in Perl:
use feature qw(say);
use strict;
use warnings;
use Text::CSV;
use Time::Piece;
my $csv = Text::CSV->new;
my $te = Time::Piece->strptime('01-OCT-14', '%d-%b-%y');
my $fn = 'Input.csv';
open (my $fh, '<', $fn) or die "Could not open file '$fn': $!\n";
chomp(my $head = <$fh>);
say "$head,Ageing-NoOfDays";
while (my $line = <$fh>) {
chomp $line;
if ($csv->parse($line)) {
my $t = ($csv->fields())[2];
my $tp = Time::Piece->strptime($t, '%d-%b-%y.%T');
my $s = $te - $tp;
say "$line," . $s->days;
} else {
warn "Line could not be parsed: $line\n";
}
}
close($fh);
Converting dates in AWK
I have a file containing many columns of text, including a timestamp along the lines of Fri Jan 02 18:23 and I need to convert that date into MM/DD/YYYY HH:MM format. I have been trying to use the standard `date' tool with awk getline to do the conversion, but I can't quite figure out how to pass the fields into the 'date' command in the format it expects (quoted with " or 's,) as getline needs the command string enclosed in quotes too. Something like "date -d '$1 $2 $3 $4' +'%D %H:%M'" | getline var Now that I think about it, I guess what I'm really asking is how to embed awk variables into a string.
If you're using gawk, you don't need the external date which can be expensive to call repeatedly:
awk '
BEGIN{
m=split("Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec",d,"|")
for(o=1;o<=m;o++){
months[d[o]]=sprintf("%02d",o)
}
format = "%m/%d/%Y %H:%M"
}
{
split($4,time,":")
date = (strftime("%Y") " " months[$2] " " $3 " " time[1] " " time[2] " 0")
print strftime(format, mktime(date))
}'
Thanks to ghostdog74 for the months array from this answer.
you can try this. Assuming just the date you specified is in the file
awk '
{
cmd ="date \"+%m/%d/%Y %H:%M\" -d \""$1" "$2" "$3" "$4"\""
cmd | getline var
print var
close(cmd)
}' file
output
$ ./shell.sh
01/02/2010 18:23
and if you are not using GNU tools, like if you are in Solaris for example, use nawk
nawk 'BEGIN{
m=split("Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec",d,"|")
for(o=1;o<=m;o++){
months[d[o]]=sprintf("%02d",o)
}
cmd="date +%Y"
cmd|getline yr
close(cmd)
}
{
day=$3
mth=months[$2]
print mth"/"day"/"yr" "$4
} ' file
I had a similar issue converting a date from RRDTool databases using rrdfetch but prefer one liners that I've been using since Apollo computer days.
Data looked like this:
localTemp rs1Temp rs2Temp thermostatMode
1547123400: 5.2788174937e+00 4.7788174937e+00 -8.7777777778e+00 2.0000000000e+00
1547123460: 5.1687014581e+00 4.7777777778e+00 -8.7777777778e+00 2.0000000000e+00
One liner:
rrdtool fetch -s -14400 thermostatDaily.rrd MAX | sed s/://g | awk '{print "echo ""\`date -r" $1,"\`" " " $2 }' | sh
Result:
Thu Jan 10 07:25:00 EST 2019 5.3373432378e+00
Thu Jan 10 07:26:00 EST 2019 5.2788174937e+00
On the face of it this doesn't look very efficient to me but this kind of methodology has always proven to be fairly low overhead under most circumstances even for very large files on very low power computer (like 25Mhz NeXT Machines). Yes Mhz.
Sed deletes the colon, awk is used to print the other various commands of interest including just echoing the awk variables and sh or bash executes the resulting string.
For methodology or large files or streams I just head the first few lines and gradually build up the one liner. Throw away code.