A basic Unix question.
I have a script which counts the number of records in a delta file.
awk '{
n++
} END {
if(n >= 1000) print "${completeFile}"; else print "${deltaFile}";
}' <${deltaFile} >${fileToUse}
Then, depending on the IF condition, I want to process the appropriate file:
cut -c2-11 < ${fileToUse}
But how do I use the contents of the file as the filename itself?
And if there are any tweaks to be made, feel free.
Thanks in advance
Cheers
Simon
To use as a filename the contents of a file which is itself identified by a variable (as asked)
cut -c2-11 <"$( cat $filetouse )"
// or in zsh just
cut -c2-11 <"$( < $filetouse )"
unless the filename in the file ends with one or more newline character(s), which people rarely do because it's quite awkward and inconvenient, then something like:
read -rdX var <$filetouse; cut -c2-11 < "${var%?}"
// where X is a character that doesn't occur in the filename
// maybe something like $'\x1f'
Tweaks: your awk prints the variable reference ${completeFile} or ${deltaFile} (because they're within the single-quoted awk script) not the value of either variable. If you actually want the value, as I'd expect from your description, you should pass the shell vars to awk vars like this
awk -vf="$completeFile" -vd="$deltaFile" '{n++} END{if(n>=1000)print f; else print d}' <"$deltaFile"`
# the " around $var can be omitted if the value contains no whitespace and no glob chars
# people _often_ but not always choose filenames that satisfy this
# and they must not contain backslash in any case
or export the shell vars as env vars (if they aren't already) and access them like
awk '{n++} END{if(n>=1000) print ENVIRON["completeFile"]; else print ENVIRON["deltaFile"]}' <"$deltaFile"
Also you don't need your own counter, awk already counts input records
awk -vf=... -vd=... 'END{if(NR>=1000)print f;else print d}' <...
or more briefly
awk -vf=... -vd=... 'END{print (NR>=1000?f:d)}' <...
or using a file argument instead of redirection so the name is available to the script
awk -vf="$completeFile" 'END{print (NR>=1000?f:FILENAME)}' "$deltaFile" # no <
and barring trailing newlines as above you don't need an intermediate file at all, just
cut -c2-11 <"$( awk -vf="$completeFile" -'END{print (NR>=1000?f:FILENAME)}' "$deltaFile")"
Or you don't really need awk, wc can do the counting and any POSIX or classic shell can do the comparison
if [ $(wc -l <"$deltaFile") -ge 1000 ]; then c="$completeFile"; else c="$deltaFile"; fi
cut -c2-11 <"$c"
Present
1856292496,-1863203096,302,918468087151,808648712,405670043170066,919015026101,M,6,T,0,15,2c,Dear Customer, Your Request is under Process,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,03,,255,,333,ERecharge_RCOM,919015540301
Requirement
1856292496,-1863203096,302,918468087151,808648712,405670043170066,919015026101,M,6,T,0,15,2c,Dear Customer Your Request is under Process,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,03,,255,,333,ERecharge_RCOM,919015540301
Current
1856292499,-1863203087,301,918081224379,808648711,405540046666191,919026240102,M,6,T,0,15,8d,Dear Business Partner,your current Core balance is Rs.29.8,GSM balance is Rs.12892.14,MRCOM balance is Rs.1 and MRTL balance is Rs.1.Reliance,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,01,,255,,333,BalQuery_RCOM,919835853611
Requirement
1856292499,-1863203087,301,918081224379,808648711,405540046666191,919026240102,M,6,T,0,15,8d,Dear Business Partner your current Core balance is Rs.29.8 GSM balance is Rs.12892.14 MRCOM balance is Rs.1 and MRTL balance is Rs.1.Reliance,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,01,,255,,333,BalQuery_RCOM,919835853611
I need to replace all the commas between the 13th comma from the left to the 12th comma starting from the right with a space, on a Unix system.
Here's a moderately succinct but mostly inscrutable (if not incomprehensible) solution using Perl.
#!/usr/bin/perl -anlF,
use strict;
use warnings;
my $lhs = 13;
my $rhs = 13;
$, = ","; # Perl is obscure on occasion!
my($nflds) = scalar(#F);
print #F[0 .. $lhs-1], "#F[$lhs .. $nflds-$rhs-1]", #F[$nflds-$rhs .. $nflds-1]
if ($nflds > $lhs + $rhs);
The shebang line uses -l to make Perl handle newlines automatically. See perldoc perlrun.
It also uses -F, which, in Perl 5.20.0 and later, is explicitly documented to automatically put Perl into -a (awk) mode and -n (read loop but don't print) mode. The input lines are automatically split into the array F using , as the delimiter. Earlier versions of Perl do not infer -a and -n from the presence of -F, so the code (now) uses -an as well as -F,. The updated code has been shown to work with Perl 5.10.0 and with each major release up to 5.18, as well as 5.20 and later.
The use strict; and use warnings; lines set Perl to fussy. You should always use them.
The two assignments set up the values you specified in the question, except that it seems to be the 13th field from the right, rather than the 12th, that you want combined. They're easily fungible if you need to.
The $, = ","; lines sets the output field separator (OFS in Awk, and $OFS in Perl under use English qw( -no_match_vars );). See perldoc English and perldoc perlvars.
The my($nflds) = scalar(#F); line determines the number of fields.
The print line is conditional on there being enough fields.
It uses Perl's array slices to:
print fields 0..$lhs-1 as separate comma-separated fields
combine fields $lhs..$nflds-$rhs-1 as a single space-separated field (by virtue of the string around the slice)
print fields $nflds-$rhs..$nflds-1 as separate comma-separated fields
The output from that, given your input data, is:
1856292496,-1863203096,302,918468087151,808648712,405670043170066,919015026101,M,6,T,0,15,2c,Dear Customer Your Request is under Process,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,03,,255,,333,ERecharge_RCOM,919015540301
1856292499,-1863203087,301,918081224379,808648711,405540046666191,919026240102,M,6,T,0,15,8d,Dear Business Partner your current Core balance is Rs.29.8 GSM balance is Rs.12892.14 MRCOM balance is Rs.1 and MRTL balance is Rs.1.Reliance,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,01,,255,,333,BalQuery_RCOM,919835853611
Note that the leading space on one of the fields in the first line is preserved.
I didn't come up with that immediately. I generated a more verbose solution like this, first:
#!/usr/bin/env perl -l
use strict;
use warnings;
my $lhs = 13;
my $rhs = 13;
while (<>)
{
chomp;
my(#fields) = split /,/;
my($nflds) = scalar(#fields);
my(#output) = #fields;
if ($nflds > $lhs + $rhs)
{
my(#combine) = #fields[$lhs .. $nflds-$rhs-1];
my $composite = "#combine";
#output = (#fields[0 .. $lhs-1], $composite, #fields[$nflds-$rhs .. $nflds-1]);
}
local $, = ",";
print #output;
}
This produces the same output as the other script. I had the scripts called rs13.pl (verbose) and rs17.pl (compact) and checked them like this (data contains your two lines of input data):
diff <(perl rs13.pl data) <(perl rs17.pl data)
There was no difference.
There are ways to make the compact solution more compact, but I'm not sure they help much.
Here is another version that uses the splice and join functions instead of array slices. In some ways, it is tidier than the other two, but it doesn't have the same protection against lines with too few fields in them.
#!/usr/bin/perl -anlF,
use strict;
use warnings;
my $lhs = 13;
my $rhs = 13;
$, = ","; # Perl is obscure on occasion!
my($nflds) = scalar(#F);
splice(#F, $lhs, $nflds - $lhs - $rhs, join(' ', #F[$lhs .. $nflds-$rhs-1]));
print #F;
It produces the same result as the other two scripts.
Yes, you could write the code in Awk; it wouldn't be as compact as this.
i'm confused about the $symbol for unix.
according to the definition, it states that it is the value stored by the variable following it. i'm not following the definition - could you please give me an example of how it is being used?
thanks
You define a variable like this:
greeting=hello
export name=luc
and use like this:
echo $greeting $name
If you use export that means the variable will be visible to subshells.
EDIT: If you want to assign a string containing spaces, you have to quote it either using double quotes (") or single quotes ('). Variables inside double quotes will be expanded whereas in single quotes they won't:
axel#loro:~$ name=luc
axel#loro:~$ echo "hello $name"
hello luc
axel#loro:~$ echo 'hello $name'
hello $name
In case of shell sctipts. When you assign a value to a variable you does not need to use $ simbol. Only if you want to acces the value of that variable.
Examples:
VARIABLE=100000;
echo "$VARIABLE";
othervariable=$VARIABLE+10;
echo $othervariable;
The other thing: if you use assignment , does not leave spaces before and after the = simbol.
Here is a good bash tutorial:
http://linuxconfig.org/Bash_scripting_Tutorial
mynameis.sh:
#!/bin/sh
finger | grep "`whoami` " | tail -n 1 | awk '{FS="\t";print $2,$3;}'
finger: prints all logged in user example result:
login Name Tty Idle Login Time Office Office Phone
xuser Forname Nickname tty7 3:18 Mar 9 07:23 (:0)
...
grep: filter lines what containing the given string (in this example we need to filter xuser if our loginname is xuser)
http://www.gnu.org/software/grep/manual/grep.html
whoami: prints my loginname
http://linux.about.com/library/cmd/blcmdl1_whoami.htm
tail -n 1 : shows only the last line of results
http://unixhelp.ed.ac.uk/CGI/man-cgi?tail
the awk script: prints the second and third column of the result: Forname, Nickname
http://www.staff.science.uu.nl/~oostr102/docs/nawk/nawk_toc.html
I am not able to get the same output as the example given in the SED tutorial for branching below,
http://www.grymoire.com/Unix/Sed.html#uh-59
Quoting the code here:
#!/bin/sh
sed '
:again
s/([ ^I]*)//
t again
'
The spaces are still in the brackets after this filter.
[UPDATE]
Here is my output:
$echo "( ( test ) )" | sed '
> :again
> s/([ ]*)//
> t again
> '
( ( test ) )
$
Shouldn't that be ((test))?
How do I get the script to delete the blank spaces in the nested parenthesis as demonstrated by the author?
[/UPDATE]
[UPDATE2]
$echo " ( ( ) ) " | sed '
> :again
> s/\([ ]*\)//
> t again
> '
Prompt is not back.
[/UPDATE2]
Also how do I enter the "^I" character? I think it is the horizontal tab, but I am not able to key in like other control characters via puTTY(for eg, to get "Enter", I type "Ctrl-V" followed by the "Enter" key, but this isn't working for tab). I tried with spaces only(using regex [ ]* instead of [ ^I]*), but this also failed to work.
Bully for you to work thru some tutorials.
Assuming you're using vi or vim all you need to do to include a tab char inside the [ .. ] grouping, is to type the tab key. ( I use putty all the time, and if pressing tab char doesn't "insert" a tab char into document/command-line, then you have a putty configuration problem ).
The ^I is from the vi list mode. List mode is handy to see where are line-feed chars (\n) will show as the reg-exp char $ (which in reg-ex is an "end-of-line anchor", the other being ^ char (beginning of line)).
So turning on vi list mode, with :li and you'll see all tab chars expanded as ^I and all end of lines as $
As you say
How do I get the script to delete the blank spaces in the nested parenthesis as demonstrated
That is slightly ambiguous, as newer seds use plain parens as grouping chars to create replacement group like \1 for the replacement-side of the s/pat/repl/ substitute cmd.
Given that your example has no numbered-replacement value in the replacement-side, I'll assume that the purpose is the remove a literal () pair AND that it should work as indicated. Once you :set list, add a tab-char inside the [ ... ], it should work. If not, please edit your question with any error messages that might appear.
I hope this helps.
( test ) does not match the regex ([ ]*). ([ ]*) only matches strings that contain nothing but spaces inside parens. Perhaps you are looking for ([ ]* to remove leading spaces inside and [ ]*) to remove trailing spaces.
I am trying to see if a new line has been matched. The reason is that my script asks for the user's name. If they press enter, then a default name is used.
The problem is I cannot seem to check whether they have pressed enter, or used a name of their own. I have searched a lot on the net, and can't find a working answer. Here is what I have tried to implement:
if [ `expr match "$temp1" "\n"` != 0 ]
Very new to this. Thanks!
You don't need to check if they have pressed enter. Just check the length of the string. Example:
if [ -z $input ]
then
echo "No name was input. Setting default name"
name=JOE
fi
Alternatively, you can use the following bash syntax to set a variable to a default value:
name=${input:-JOE}
This means that if the variable input is not set, name would be set to the default of "JOE".
Try something like this:
#!/bin/sh
/bin/echo -n "Who? "
read name
if [ "x$name" != "x" ]
then
echo "Hi $name"
else
echo "Hi no-name"
fi
You can use the `read' bash builtin for this:
echo 'what is your name?'; read name; echo Hi ${name:='John'}
That will assign the name "John" as the default name if the user presses enter without entering any name.
I forgot to add that the feature that helps is is the default shell variable value assignment if it is not set ${name:='John'}.