I am trying to get line count from a file like below in a ksh script. But it returns nothing :
filerecordcount= $((`wc -l <../data/act.dat`))
I also tried these :
filerecordcount= `wc -l <../data/act.dat`
filerecordcount= $(wc -l <../data/act.dat)
When i print the variable its not printing the value in the variable.
print "Record Count in .dat file : $filerecordcount." 1>&2;
But when i try the same from command prompt it returns the count
wc -l<../data/act.dat
You don't have to give the space after the = in assigning part. Use the below one. It will work fine. But don't forget to print the variable filerecordcount.
UPDATE:
filerecordcount=$((`wc -l <../data/act.dat`))
Simplify simplify. Your backquotes are doing the command expansion, and their output is being processed by $((...)) as an arithmetic expression. It's a little redundant.
filerecordcount=$(wc -l < ../data/act.dat)
No space after the =, and just one level of command expansion.
Alternately you can use process substitution:
read filerecordcount < <(wc -l < ../data/act.dat)
Or you could even do this without a subshell, using a loop:
filerecordcount=0
while read junk; do ((filerecordcount++)); done < ../data/act.dat
Related
I will be getting one command line argument in the script I'm writing which will itself be a space delimited list of the actual command line arguments. I'd like to set the arguments of the current script with these arguments. How might I accomplish that?
I'd like to use set -- but I'm not sure how this would work.
E.g.
Given arguments to my script: -a -b -c
echo $1 # prints "-a -b -c"
You can do this with set -- "${(z)1}". This will split $1 into words, handling quoting the same way the shell itself does:
% cat script.zsh
#!/usr/bin/env zsh
set -- "${(z)1}"
for arg; do
echo "==$arg=="
done
% ./script.zsh "-a -b -c -d'has spaces'"
==-a==
==-b==
==-c==
==-d'has spaces'==
If you also want to remove a level of quotes, use "${(#Q)${(z)1}}" instead.
I have a file test.sh from which I am executing the following awk command.
awk -f x.awk < result/output.txt >>difference.txt
x.awk
while (getline < result/$bld/$DeviceType)
the variable DeviceType and bld are available in test.sh.
I have declared them as export type.
export DeviceType=$line
Even then while executing test.sh file, the script stops at following line
awk -f x.awk < result/output.txt >>difference.txt
and I am getting
awk: x.awk:4: (FILENAME=- FNR=116) fatal: division by zero attempted
error.
The awk script is read by awk, not touched by the shell. Inside an awk script, $bld means 'the field designated by the number in the variable bld' (that's the awk variable bld).
You can set awk variables on the command line (officially with the -v option):
awk -v bld="$bld" -v dev="$DeviceType" -f x.awk < result/output.txt >> difference.txt
Whether that does what you want is still debatable. Most likely you need x.awk to contain something like:
BEGIN { file = sprintf("result/%s/%s", bld, dev); }
{ while ((getline < file) > 0) print }
awk is not shell just like C is not shell. You should not expect to be able to access shell variables within an awk program any more than you can access shell variables within a C program.
To pass the VALUE of shell variables to an awk script, see http://cfajohnson.com/shell/cus-faq-2.html#Q24 for details but essentially:
awk -v awkvar="$shellvar" '{ ... use awkvar ...}'
is usually the right approach.
Having said that, whatever you're trying to do it looks like the wrong approach. If you are considering using getline, make sure to read http://awk.freeshell.org/AllAboutGetline first and understand all of the caveats but if you tell us what it is you're trying to do with sample input and expected output we can almost certainly help you come up with a better approach that has nothing to do with getline.
For grep there's a fixed string option, -F (fgrep) to turn off regex interpretation of the search string.
Is there a similar facility for sed? I couldn't find anything in the man. A recommendation of another gnu/linux tool would also be fine.
I'm using sed for the find and replace functionality: sed -i "s/abc/def/g"
Do you have to use sed? If you're writing a bash script, you can do
#!/bin/bash
pattern='abc'
replace='def'
file=/path/to/file
tmpfile="${TMPDIR:-/tmp}/$( basename "$file" ).$$"
while read -r line
do
echo "${line//$pattern/$replace}"
done < "$file" > "$tmpfile" && mv "$tmpfile" "$file"
With an older Bourne shell (such as ksh88 or POSIX sh), you may not have that cool ${var/pattern/replace} structure, but you do have ${var#pattern} and ${var%pattern}, which can be used to split the string up and then reassemble it. If you need to do that, you're in for a lot more code - but it's really not too bad.
If you're not in a shell script already, you could pretty easily make the pattern, replace, and filename parameters and just call this. :)
PS: The ${TMPDIR:-/tmp} structure uses $TMPDIR if that's set in your environment, or uses /tmp if the variable isn't set. I like to stick the PID of the current process on the end of the filename in the hopes that it'll be slightly more unique. You should probably use mktemp or similar in the "real world", but this is ok for a quick example, and the mktemp binary isn't always available.
Option 1) Escape regexp characters. E.g. sed 's/\$0\.0/0/g' will replace all occurrences of $0.0 with 0.
Option 2) Use perl -p -e in conjunction with quotemeta. E.g. perl -p -e 's/\\./,/gi' will replace all occurrences of . with ,.
You can use option 2 in scripts like this:
SEARCH="C++"
REPLACE="C#"
cat $FILELIST | perl -p -e "s/\\Q$SEARCH\\E/$REPLACE/g" > $NEWLIST
If you're not opposed to Ruby or long lines, you could use this:
alias replace='ruby -e "File.write(ARGV[0], File.read(ARGV[0]).gsub(ARGV[1]) { ARGV[2] })"'
replace test3.txt abc def
This loads the whole file into memory, performs the replacements and saves it back to disk. Should probably not be used for massive files.
If you don't want to escape your string, you can reach your goal in 2 steps:
fgrep the line (getting the line number) you want to replace, and
afterwards use sed for replacing this line.
E.g.
#/bin/sh
PATTERN='foo*[)*abc' # we need it literal
LINENUMBER="$( fgrep -n "$PATTERN" "$FILE" | cut -d':' -f1 )"
NEWSTRING='my new string'
sed -i "${LINENUMBER}s/.*/$NEWSTRING/" "$FILE"
You can do this in two lines of bash code if you're OK with reading the whole file into memory. This is quite flexible -- the pattern and replacement can contain newlines to match across lines if needed. It also preserves any trailing newline or lack thereof, which a simple loop with read does not.
mapfile -d '' < file
printf '%s' "${MAPFILE//"$pat"/"$rep"}" > file
For completeness, if the file can contain null bytes (\0), we need to extend the above, and it becomes
mapfile -d '' < <(cat file; printf '\0')
last=${MAPFILE[-1]}; unset "MAPFILE[-1]"
printf '%s\0' "${MAPFILE[#]//"$pat"/"$rep"}" > file
printf '%s' "${last//"$pat"/"$rep"}" >> file
perl -i.orig -pse 'while (($i = index($_,$s)) >= 0) { substr($_,$i,length($s), $r)}'--\
-s='$_REQUEST['\'old\'']' -r='$_REQUEST['\'new\'']' sample.txt
-i.orig in-place modification with backup.
-p print lines from the input file by default
-s enable rudimentary parsing of command line arguments
-e run this script
index($_,$s) search for the $s string
substr($_,$i,length($s), $r) replace the string
while (($i = index($_,$s)) >= 0) repeat until
-- end of perl parameters
-s='$_REQUEST['\'old\'']', -r='$_REQUEST['\'new\'']' - set $s,$r
You still need to "escape" ' chars but the rest should be straight forward.
Note: this started as an answer to How to pass special character string to sed hence the $_REQUEST['old'] strings, however this question is a bit more appropriately formulated.
You should be using replace instead of sed.
From the man page:
The replace utility program changes strings in place in files or on the
standard input.
Invoke replace in one of the following ways:
shell> replace from to [from to] ... -- file_name [file_name] ...
shell> replace from to [from to] ... < file_name
from represents a string to look for and to represents its replacement.
There can be one or more pairs of strings.
I found the following bash script in order to monitor cp progress.
#!/bin/sh
cp_p()
{
strace -q -ewrite cp -- "${1}" "${2}" 2>&1 \
| awk '{
count += $NF
if (count % 10 == 0) {
percent = count / total_size * 100
printf "%3d%% [", percent
for (i=0;i<=percent;i++)
printf "="
printf ">"
for (i=percent;i<100;i++)
printf " "
printf "]\r"
}
}
END { print "" }' total_size=$(stat -c '%s' "${1}") count=0
}
I don't understand the "-ewrite" option for the strace command. The closest thing I've found is the man page for strace which is
-e write=set Perform a full hexadecimal and ASCII dump of all the
data written to file descriptors
listed in the specified set. For
example, to see all output activity on
file descriptors 3 and 5 use -e
write=3,5. Note that this is
independent from the normal tracing of
the write(2) system call which is
controlled by the option -e
trace=write.
However I don't understand what the -ewrite option does.
-ewrite means that only the "write" system call will be traced.
-e expr A qualifying expression which modifies which events
to trace or how to trace them. The format of the
expression is:
[qualifier=][!]value1[,value2]...
where qualifier is one of trace, abbrev, verbose,
raw, signal, read, or write and value is a quali-
fier-dependent symbol or number. The default qual-
ifier is trace. Using an exclamation mark negates
the set of values. For example, -eopen means lit-
erally -e trace=open which in turn means trace only
the open system call. By contrast, -etrace=!open
means to trace every system call except open. In
addition, the special values all and none have the
obvious meanings.
Note that some shells use the exclamation point for
history expansion even inside quoted arguments. If
so, you must escape the exclamation point with a
backslash.
What is the simplest way to parse line continuation characters? This seems like such a basic action that I'm surprised there's no basic command for doing this. 'while read' and 'while read -r' loops don't do what I want, and the easiest solution I've found is the sed solution below. Is there a way to do this with something basic like tr?
$ cat input
Output should be \
one line with a '\' character.
$ while read l; do echo $l; done < input
Output should be one line with a '' character.
$ while read -r l; do echo $l; done < input
Output should be \
one line with a '\' character.
$ sed '/\\$/{N; s/\\\n//;}' input
Output should be one line with a '\' character.
$ perl -0777 -pe 's/\\\n//s' input
Output should be one line with a '\' character.
If by "simplest" you mean concise and legible, I'd suggest your perl-ism with one small modification:
$ perl -pe 's/\\\n//' /tmp/line-cont
No need for the possibly memory intensive ... -0777 ... (whole file slurp mode) switch.
If, however, by "simplest" you mean not the leaving shell, this will suffice:
$ { while read -r LINE; do
printf "%s" "${LINE%\\}"; # strip line-continuation, if any
test "${LINE##*\\}" && echo; # emit newline for non-continued lines
done; } < /tmp/input
(I prefer printf "%s" $USER_INPUT to echo $USER_INPUT because echo cannot portably be told to stop looking for switches, and printf is commonly a built-in anyway.)
Just tuck that in a user-defined function and never be revolted by it again. Caution: this latter approach will add a trailing newline to a file which lacks one.
The regex way looks like the way to go.
I would go with the Perl solution simply because it will likely be the most extensible if you want to add more functionality later.