I wrote the following code:
success="Traffic Engineering Tunnel validated successfully"
text=$(tail -2 /apps/pofitb/logs/tunnels.log)
echo $executed_date
if [[ $text = $success ]]
then
text=$(tail -2 /apps/pofitb/logs/tunnels.log)
echo "Tunnel script execution is successful" | mailx -s "Tunnel script is executed succefsfully on $executed_date" abc#gmail.com
else
echo "Tunnel script lastly executed on $executed_date" | mailx -s "Tunnel script FAILED!!!" abc#gmail.com
fi
exit
Currently tunnel.log file has blank line while being updated. So, text=$(tail -2 /apps/pofitb/logs/tunnels.log) extracts the last non-blank line from the end of the file. This works if the number of blank line inserted at the end of the file is 1.
How can I modify the script such that the script searches for the last last non-blank line from the file tunnel.log, irrespective of the number of blank lines inserted ?
awk to the rescue!
tac log | awk 'NF{print;exit}'
if your log is too long, start with a generous tail first
tail -5 log | tac | awk 'NF{print;exit}'
will print the last non-empty line.
Related
I have 2 files: file.dat and mapping.dat.
file.dat contains entries (can contain duplicate), and mapping.dat is a static file which contains the entries and their corresponding session/job names separated by a comma.
I have developed a simple UNIX Shell script which runs in loop where for each entry in file.dat, it searches for the session/job name in mapping.dat and displays the output.
Content of file.dat
ekm
ckm
cnc
cnx
ekm
hnm
dam
Content of mapping.dat
#Entry,Job_Name#
ckm,BDSCKM
cnc,BDSCNC
cnx,BDSCNX
ekm,BDSEKM
azm,BDSAZM
bam,BDSBAM
cam,BDSCAM
oid,BDSOID
hnm,BDSHNM
dam,BDSDAM
Current Script:
#!/bin/ksh
for FILE in `cat file.dat`
do
SESSION=`grep $FILE mapping.dat | cut -d, -f2`
echo "For ${FILE}, Session launched is: ${SESSION} "
done
Current output
For ekm, Session launched is: BDSEKM
For ckm, Session launched is: BDSCKM
For cnc, Session launched is: BDSCNC
For cnx, Session launched is: BDSCNX
For ekm, Session launched is: BDSEKM
For hnm, Session launched is: BDSHNM
For dam, Session launched is: BDSDAM
My question is I want to introduce a wait/sleep time for every 2 occurrences of output i.e. it should first display
For ekm, Session launched is: BDSEKM
For ckm, Session launched is: BDSCKM
wait for 90 seconds, and then
For cnc, Session launched is: BDSCNC
For cnx, Session launched is: BDSCNX
..and so on
Try helping yourself using the modulo operator %.
#!/bin/ksh
count=1
for file in $( cat file.dat )
do
session=$( grep $file mapping.dat | cut -d, -f2 )
echo "For ${file}, session launched is ${session}."
if (( count % 2 == 0 ))
then
sleep 90
fi
(( count++ ))
done
Here's how I would do it, I added in a message for unrecognised items (you can safely remove that by simply deleting || session='UNRECOGNIZED' from the first line of the while read loop). I'm not overly familiar with ksh, but I believe read is the same as bash in this context (I'm very familiar with bash).
I tested with your example data, and it works on both ksh and bash.
#!/bin/ksh
# Print 2 mappings every 90 seconds
FILE="./file.dat"
MAP="./mapping.dat"
while IFS= read -r line; do
session=$(grep "$line" "$MAP") || session='UNRECOGNIZED'
echo "For $line, session launched is: ${session#*,}"
((count++ % 2)) && sleep 90
done < "$FILE"
I used non greedy suffix removal (#) to isolate the 'session'.
To print the first 2 consecutive lines, then wait 90 seconds, then print the next 2 lines, etc.
Use the % modulo operator as suggested by Tony Stark. There is no need to initialize the counter.
#!/bin/bash
for file in $( cat file.dat )
do
session=$( grep $file mapping.dat | cut -d, -f2 )
echo "For ${file}, session launched is ${session}."
(( count++ % 2 )) && sleep 90
done
Trying to write a simple script to find the latest modified file from a directory and then count the lines of that modified file. Below is part of my script.
Note: the $stg variable is created for another directory
echo "LATEST LOG = $(ls -ltr $stg/abc/foo***.txt | awk '{print $5, $6, $7, $8, $9}' | tail -n1)"
echo "COUNT = $(wc -l $stg/abc/foo***.txt | tail -n1)"
What happens on the "COUNT" part is that it does not match the count of the LATEST LOG because it seems to be counting a different log file.
Any suggestions? Thank you!
Suggestion: store the result of the latest log in a variable, and reuse it in the count. Like this:
#!/bin/bash
latestlogline=$(ls -ltr foo*.txt | awk '{print $5, $6, $7, $8, $9}' | tail -n1)
latestlogfilename=$(echo $latestlogline | awk 'NF>1{print $NF}')
echo "LATEST LOG = $(echo $latestlogline)"
echo "COUNT = $(wc -l $latestlogfilename)"
Details:
latestlogline: your code exactly, to extract the complete line of information
latestlogfilename: just the filename. wc -l expects a filename, so extract it from your first command.
Then just echo the variable values.
As commented before, *** is exactly the same thing as *.
I have a question on creating file name on basis of output we get on running a command. Below is the example
Have 2 records like below
cat test1.txt
Unable to find Solution
Can you please help
And I am running below command to get the last word from the first line and i want to have file name to be that name(Last word name)
cat test1.txt | head -1 | awk '{ print $NF }'
Solution
Can you please help me to get the file name as a last word name
When you want to redirect your output to a file and have the filename from the first line of your output, you can use
outfile=""
your_command | while read line; do
if [ -z "${outfile}" ]; then
outfile=$(echo "${line}"| awk '{ print $NF }')
fi
echo "${line}" >> ${outfile}
done
I need to find files where a specific string appears twice or more.
For example, for three files:
File 1:
Hello World!
File 2:
Hello World!
Hello !
File 3:
Hello World!
Hello
Hello Again.
--
I want to grep Hello and only get files 2 & 3.
What about this:
grep -o -c Hello * | awk -F: '{if ($2 > 1){print $1}}'
Since the question is tagged grep, here is a solution using only that utility and bash (no awk required):
#!/bin/bash
for file in *
do
if [ "$(grep -c "Hello" "${file}")" -gt 1 ]
then
echo "${file}"
fi
done
Can be a one-liner:
for file in *; do if [ "$(grep -c "Hello" "${file}")" -gt 1 ]; then echo "${file}"; fi; done
Explanation
You can modify the for file in * statement with whatever shell expansion you want to get all the data files.
grep -c returns the number of lines that match the pattern, with multiple matches on a line still counting for just one matched line.
if [ ... -gt 1 ] test that more than one line is matched in the file. If so:
echo ${file} print the file name.
This awk will print the file name of all files with 2 or more Hello
awk 'FNR==1 {if (a>1) print f;a=0} /Hello/ {a++} {f=FILENAME} END {if (a>1) print f}' *
file2
file3
What you need is a grep that can recognise patterns across line endings ("hello" followed by anything (possibly even line endings), followed by "hello")
As grep processes your files line by line, it is (by itself) not the right tool for the job - unless you manage to cram the whole file into one single line.
Now, that is easy, for example using the tr command, replacing line endings by spaces:
if cat $file | tr '\n' ' ' | grep -q 'hello.*hello'
then
echo "$file matches"
fi
This is quite efficient, even on large files with many (say 100000) lines, and can be made even more efficient by calling grep with --max-count=1 , making it stop the search after a match has been found. It doesn't matter whether the two hellos are on the same line or not.
After reading your question, I think you also want to find the case hello hello in one line. ( find files where a specific string appears twice or more.) so I come up with this one-liner:
awk -v p="hello" 'FNR==1{x=0}{x+=gsub(p,p);if(x>1){print FILENAME;nextfile}}' *
in the above line, p is the pattern you want to search
it will print the filename if the file contains the pattern two or more times. no matter they are in same or different lines
during the processing, after checking some line, if we had already found two or more pattern, print the filename and stop processing current file, take the next input file, if there still are. This is helpful if you have big files.
A little test:
kent$ head f*
==> f <==
hello hello world
==> f2 <==
hello
==> f3 <==
hello
hello
SK-Arch 22:27:00 /tmp/test
kent$ awk -v p="hello" 'FNR==1{x=0}{x+=gsub(p,p);if(x>1){print FILENAME;nextfile}}' f*
f
f3
Another way:
grep Hello * | cut -d: -f1 | uniq -d
Grep for lines containing 'Hello'; keep only the file names; print only the duplicates.
grep -c Hello * | egrep -v ':[01]$' | sed 's/:[0-9]*$//'
Piping to a scripting language might be overkill, but it's oftentimes much easier than just using awk
grep -rnc "Hello" . | ruby -ne 'file, count = $_.split(":"); puts "#{file}: #{count}" if count&.to_i >= 2'
So for your input, we get
$ grep -rnc "Hello" . | ruby -ne 'file, count = $_.split(":"); puts "#{file}: #{count}" if count&.to_i >= 2'
./2: 2
./3: 3
Or to omit the count
grep -rnc "Hello" . | ruby -ne 'file, _ = $_.split(":"); puts file if count&.to_i >= 2'
I am trying to get the line count to a variable. The source file filename.dat contains 2 lines of records as:
112233;778899
445566
Script 1
line_cnt=$(more /home/filename.dat | wc -l)
echo $line_cnt
When I run this script, I get the output of 2. Now, I have a modified version:
Script 2
filename=/home/filename.dat
line_cnt=$(more ${filename} | wc -l)
echo $line_cnt
The input file has the same records. But this is giving me an output of 5 even though it has only 2 records.
Can someone tell me what is wrong?
Edit - Corrected the file path in 2nd script
line_cnt=`cat ${filename} | wc -l`
The cat ${filename} | wc -l should be within back quotes.