I am trying to get the line count to a variable. The source file filename.dat contains 2 lines of records as:
112233;778899
445566
Script 1
line_cnt=$(more /home/filename.dat | wc -l)
echo $line_cnt
When I run this script, I get the output of 2. Now, I have a modified version:
Script 2
filename=/home/filename.dat
line_cnt=$(more ${filename} | wc -l)
echo $line_cnt
The input file has the same records. But this is giving me an output of 5 even though it has only 2 records.
Can someone tell me what is wrong?
Edit - Corrected the file path in 2nd script
line_cnt=`cat ${filename} | wc -l`
The cat ${filename} | wc -l should be within back quotes.
Related
I wrote the following code:
success="Traffic Engineering Tunnel validated successfully"
text=$(tail -2 /apps/pofitb/logs/tunnels.log)
echo $executed_date
if [[ $text = $success ]]
then
text=$(tail -2 /apps/pofitb/logs/tunnels.log)
echo "Tunnel script execution is successful" | mailx -s "Tunnel script is executed succefsfully on $executed_date" abc#gmail.com
else
echo "Tunnel script lastly executed on $executed_date" | mailx -s "Tunnel script FAILED!!!" abc#gmail.com
fi
exit
Currently tunnel.log file has blank line while being updated. So, text=$(tail -2 /apps/pofitb/logs/tunnels.log) extracts the last non-blank line from the end of the file. This works if the number of blank line inserted at the end of the file is 1.
How can I modify the script such that the script searches for the last last non-blank line from the file tunnel.log, irrespective of the number of blank lines inserted ?
awk to the rescue!
tac log | awk 'NF{print;exit}'
if your log is too long, start with a generous tail first
tail -5 log | tac | awk 'NF{print;exit}'
will print the last non-empty line.
I have a question on creating file name on basis of output we get on running a command. Below is the example
Have 2 records like below
cat test1.txt
Unable to find Solution
Can you please help
And I am running below command to get the last word from the first line and i want to have file name to be that name(Last word name)
cat test1.txt | head -1 | awk '{ print $NF }'
Solution
Can you please help me to get the file name as a last word name
When you want to redirect your output to a file and have the filename from the first line of your output, you can use
outfile=""
your_command | while read line; do
if [ -z "${outfile}" ]; then
outfile=$(echo "${line}"| awk '{ print $NF }')
fi
echo "${line}" >> ${outfile}
done
Hi I am new to UNIX and I have to get the count of lines from incoming csv files. I have used the following command to get the count.
wc -l filename.csv
Consider files coming with 1 record iam getting some files with * at the start and for those files if i issue the same command iam getting count as 0. Does * mean anything in here and if i get a file with ctrlm(CR) instead of NL how to get the count of lines in that file. gimme a command that solves the issue.
The following query helps you to get the count
cat FILE_NAME | wc -l
All of the answers are wrong. CSV files accept line breaks in between quotes which should still be considered part of the same line. If you have either Python or PHP on your machine, you could be doing something like this:
Python
//From stdin
cat *.csv | python -c "import csv; import sys; print(sum(1 for i in csv.reader(sys.stdin)))"
//From file name
python -c "import csv; print(sum(1 for i in csv.reader(open('csv.csv'))))"
PHP
//From stdin
cat *.csv | php -r 'for($i=0; fgetcsv(STDIN); $i++);echo "$i\n";'
//From file name
php -r 'for($i=0, $fp=fopen("csv.csv", "r"); fgetcsv($fp); $i++);echo "$i\n";'
I have also created a script to simulate the output of wc -l: https://github.com/dhulke/CSVCount
In case you have multiple .csv files in the same folder, use
cat *.csv | wc -l
to get the total number of lines in all csv files in the current directory. So,
-c counts characters and -m counts bytes (identical as long as you use ASCII). You can also use wc to count the number of files, e.g. by: ls -l | wc -l
wc -l mytextfile
Or to only output the number of lines:
wc -l < mytextfile
Usage: wc [OPTION]... [FILE]...
or: wc [OPTION]... --files0-from=F
Print newline, word, and byte counts for each FILE, and a total line if
more than one FILE is specified. With no FILE, or when FILE is -,
read standard input.
-c, --bytes print the byte counts
-m, --chars print the character counts
-l, --lines print the newline counts
--files0-from=F read input from the files specified by
NUL-terminated names in file F;
If F is - then read names from standard input
-L, --max-line-length print the length of the longest line
-w, --words print the word counts
--help display this help and exit
--version output version information and exit
You can also use xsv for that. It also supports many other subcommands that are useful for csv files.
xsv count file.csv
echo $(wc -l file_name.csv|awk '{print $1}')
I'm parsing a properties file to get a list of properties defiend. I want to check all the places these properties are used (target dir and subdirs), flagging up any that are defined in the properties file but not used anywhere in the targer dir. Thus far I have
FILE=$1
TARGETROOT=$2
for LINE in `grep '[A-Z]*=' $FILE | awk -F '=' '{print$1}'`;
do
done;
Inside this loop I want to find those $LINE vars which are not in $TARGETROOT or its subdirs
Example files
Properties File
a=1
b=2
c=3
...
Many files that contain references to properties via
FILE 1
PropAValue = a
check the return code of grep.
You can do this by inspecting the $? variable.
if it is 0 then the string was found, otherwise the string was not found. If not 0 then add that string to a 'not found' array and that should be your list of not found properties.
grep "string"
if [$? -ne 0]
then
string not found
fi
Using xyz | while read PROP instead of for PROP in ``xyz``; do for those cases when xyz can get arbitrarily large
Using grep -l ... >/dev/null || xyz to execute xyz if grep fails to match, and discard the grep output do /dev/null without executing xyz if one match is found (-l stops grep after the first match, if any, making it more efficient)
FILE=$1
TARGETROOT=$2
grep '^[A-Z]*=' "$FILE2" | awk -F= '{print$1}' | while read PROP ; do
find "$TARGETROOT" -type f | while read FILE2 ; do
grep -l "^${PROP}=" "$FILE2" >/dev/null || {
echo "Propery $PROP missing from $FILE2"
}
done
done
If dealing with a large number of properties and/or files under $TARGETROOT you can use the following, more efficient approach (which opens and scans each file only once instead of the previous solution's N times, where N was the number of properties in $FILE):
Using a temporary file with all sorted properties from $FILE to avoid duplicating work
Using awk ... | sort -u to isolate all sorted properties appearing in another file $FILE2
Using comm -23 "$PROPSFILE" - to isolate those lines (properties) which only appear in $PROPSFILE and not on the standard input (i.e. in $FILE2)
FILE=$1
TARGETROOT=$2
PROPSFILE="/tmp/~props.$$"
grep '^[A-Z]*=' "$FILE" | awk -F= '{print$1}' | sort -u >"$PROPSFILE"
find "$TARGETROOT" -type f | while read FILE2 ; do
grep '^[A-Z]*=' "$FILE2" | awk -F= '{print$1}' | sort -u |
comm -23 "$PROPSFILE" - | while read PROP ; do
echo "Propery $PROP missing from $FILE2"
done
done
rm -f "$PROPSFILE"
I want to delete one or more specific line numbers from a file. How would I do this using sed?
If you want to delete lines from 5 through 10 and line 12th:
sed -e '5,10d;12d' file
This will print the results to the screen. If you want to save the results to the same file:
sed -i.bak -e '5,10d;12d' file
This will store the unmodified file as file.bak, and delete the given lines.
Note: Line numbers start at 1. The first line of the file is 1, not 0.
You can delete a particular single line with its line number by
sed -i '33d' file
This will delete the line on 33 line number and save the updated file.
and awk as well
awk 'NR!~/^(5|10|25)$/' file
$ cat foo
1
2
3
4
5
$ sed -e '2d;4d' foo
1
3
5
$
This is very often a symptom of an antipattern. The tool which produced the line numbers may well be replaced with one which deletes the lines right away. For example;
grep -nh error logfile | cut -d: -f1 | deletelines logfile
(where deletelines is the utility you are imagining you need) is the same as
grep -v error logfile
Having said that, if you are in a situation where you genuinely need to perform this task, you can generate a simple sed script from the file of line numbers. Humorously (but perhaps slightly confusingly) you can do this with sed.
sed 's%$%d%' linenumbers
This accepts a file of line numbers, one per line, and produces, on standard output, the same line numbers with d appended after each. This is a valid sed script, which we can save to a file, or (on some platforms) pipe to another sed instance:
sed 's%$%d%' linenumbers | sed -f - logfile
On some platforms, sed -f does not understand the option argument - to mean standard input, so you have to redirect the script to a temporary file, and clean it up when you are done, or maybe replace the lone dash with /dev/stdin or /proc/$pid/fd/1 if your OS (or shell) has that.
As always, you can add -i before the -f option to have sed edit the target file in place, instead of producing the result on standard output. On *BSDish platforms (including OSX) you need to supply an explicit argument to -i as well; a common idiom is to supply an empty argument; -i ''.
The shortest, deleting the first line in sed
sed -i '1d' file
As Brian states here, <address><command> is used, <address> is <1> and <command> <d>.
I would like to propose a generalization with awk.
When the file is made by blocks of a fixed size
and the lines to delete are repeated for each block,
awk can work fine in such a way
awk '{nl=((NR-1)%2000)+1; if ( (nl<714) || ((nl>1025)&&(nl<1029)) ) print $0}'
OriginFile.dat > MyOutputCuttedFile.dat
In this example the size for the block is 2000 and I want to print the lines [1..713] and [1026..1029].
NR is the variable used by awk to store the current line number.
% gives the remainder (or modulus) of the division of two integers;
nl=((NR-1)%BLOCKSIZE)+1 Here we write in the variable nl the line number inside the current block. (see below)
|| and && are the logical operator OR and AND.
print $0 writes the full line
Why ((NR-1)%BLOCKSIZE)+1:
(NR-1) We need a shift of one because 1%3=1, 2%3=2, but 3%3=0.
+1 We add again 1 because we want to restore the desired order.
+-----+------+----------+------------+
| NR | NR%3 | (NR-1)%3 | (NR-1)%3+1 |
+-----+------+----------+------------+
| 1 | 1 | 0 | 1 |
| 2 | 2 | 1 | 2 |
| 3 | 0 | 2 | 3 |
| 4 | 1 | 0 | 1 |
+-----+------+----------+------------+
cat -b /etc/passwd | sed -E 's/^( )+(<line_number>)(\t)(.*)/--removed---/g;s/^( )+([0-9]+)(\t)//g'
cat -b -> print lines with numbers
s/^( )+(<line_number>)(\t)(.*)//g -> replace line number to null (remove line)
s/^( )+([0-9]+)(\t)//g #remove numbers the cat printed