A basic Unix question.
I have a script which counts the number of records in a delta file.
awk '{
n++
} END {
if(n >= 1000) print "${completeFile}"; else print "${deltaFile}";
}' <${deltaFile} >${fileToUse}
Then, depending on the IF condition, I want to process the appropriate file:
cut -c2-11 < ${fileToUse}
But how do I use the contents of the file as the filename itself?
And if there are any tweaks to be made, feel free.
Thanks in advance
Cheers
Simon
To use as a filename the contents of a file which is itself identified by a variable (as asked)
cut -c2-11 <"$( cat $filetouse )"
// or in zsh just
cut -c2-11 <"$( < $filetouse )"
unless the filename in the file ends with one or more newline character(s), which people rarely do because it's quite awkward and inconvenient, then something like:
read -rdX var <$filetouse; cut -c2-11 < "${var%?}"
// where X is a character that doesn't occur in the filename
// maybe something like $'\x1f'
Tweaks: your awk prints the variable reference ${completeFile} or ${deltaFile} (because they're within the single-quoted awk script) not the value of either variable. If you actually want the value, as I'd expect from your description, you should pass the shell vars to awk vars like this
awk -vf="$completeFile" -vd="$deltaFile" '{n++} END{if(n>=1000)print f; else print d}' <"$deltaFile"`
# the " around $var can be omitted if the value contains no whitespace and no glob chars
# people _often_ but not always choose filenames that satisfy this
# and they must not contain backslash in any case
or export the shell vars as env vars (if they aren't already) and access them like
awk '{n++} END{if(n>=1000) print ENVIRON["completeFile"]; else print ENVIRON["deltaFile"]}' <"$deltaFile"
Also you don't need your own counter, awk already counts input records
awk -vf=... -vd=... 'END{if(NR>=1000)print f;else print d}' <...
or more briefly
awk -vf=... -vd=... 'END{print (NR>=1000?f:d)}' <...
or using a file argument instead of redirection so the name is available to the script
awk -vf="$completeFile" 'END{print (NR>=1000?f:FILENAME)}' "$deltaFile" # no <
and barring trailing newlines as above you don't need an intermediate file at all, just
cut -c2-11 <"$( awk -vf="$completeFile" -'END{print (NR>=1000?f:FILENAME)}' "$deltaFile")"
Or you don't really need awk, wc can do the counting and any POSIX or classic shell can do the comparison
if [ $(wc -l <"$deltaFile") -ge 1000 ]; then c="$completeFile"; else c="$deltaFile"; fi
cut -c2-11 <"$c"
Present
1856292496,-1863203096,302,918468087151,808648712,405670043170066,919015026101,M,6,T,0,15,2c,Dear Customer, Your Request is under Process,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,03,,255,,333,ERecharge_RCOM,919015540301
Requirement
1856292496,-1863203096,302,918468087151,808648712,405670043170066,919015026101,M,6,T,0,15,2c,Dear Customer Your Request is under Process,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,03,,255,,333,ERecharge_RCOM,919015540301
Current
1856292499,-1863203087,301,918081224379,808648711,405540046666191,919026240102,M,6,T,0,15,8d,Dear Business Partner,your current Core balance is Rs.29.8,GSM balance is Rs.12892.14,MRCOM balance is Rs.1 and MRTL balance is Rs.1.Reliance,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,01,,255,,333,BalQuery_RCOM,919835853611
Requirement
1856292499,-1863203087,301,918081224379,808648711,405540046666191,919026240102,M,6,T,0,15,8d,Dear Business Partner your current Core balance is Rs.29.8 GSM balance is Rs.12892.14 MRCOM balance is Rs.1 and MRTL balance is Rs.1.Reliance,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,01,,255,,333,BalQuery_RCOM,919835853611
I need to replace all the commas between the 13th comma from the left to the 12th comma starting from the right with a space, on a Unix system.
Here's a moderately succinct but mostly inscrutable (if not incomprehensible) solution using Perl.
#!/usr/bin/perl -anlF,
use strict;
use warnings;
my $lhs = 13;
my $rhs = 13;
$, = ","; # Perl is obscure on occasion!
my($nflds) = scalar(#F);
print #F[0 .. $lhs-1], "#F[$lhs .. $nflds-$rhs-1]", #F[$nflds-$rhs .. $nflds-1]
if ($nflds > $lhs + $rhs);
The shebang line uses -l to make Perl handle newlines automatically. See perldoc perlrun.
It also uses -F, which, in Perl 5.20.0 and later, is explicitly documented to automatically put Perl into -a (awk) mode and -n (read loop but don't print) mode. The input lines are automatically split into the array F using , as the delimiter. Earlier versions of Perl do not infer -a and -n from the presence of -F, so the code (now) uses -an as well as -F,. The updated code has been shown to work with Perl 5.10.0 and with each major release up to 5.18, as well as 5.20 and later.
The use strict; and use warnings; lines set Perl to fussy. You should always use them.
The two assignments set up the values you specified in the question, except that it seems to be the 13th field from the right, rather than the 12th, that you want combined. They're easily fungible if you need to.
The $, = ","; lines sets the output field separator (OFS in Awk, and $OFS in Perl under use English qw( -no_match_vars );). See perldoc English and perldoc perlvars.
The my($nflds) = scalar(#F); line determines the number of fields.
The print line is conditional on there being enough fields.
It uses Perl's array slices to:
print fields 0..$lhs-1 as separate comma-separated fields
combine fields $lhs..$nflds-$rhs-1 as a single space-separated field (by virtue of the string around the slice)
print fields $nflds-$rhs..$nflds-1 as separate comma-separated fields
The output from that, given your input data, is:
1856292496,-1863203096,302,918468087151,808648712,405670043170066,919015026101,M,6,T,0,15,2c,Dear Customer Your Request is under Process,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,03,,255,,333,ERecharge_RCOM,919015540301
1856292499,-1863203087,301,918081224379,808648711,405540046666191,919026240102,M,6,T,0,15,8d,Dear Business Partner your current Core balance is Rs.29.8 GSM balance is Rs.12892.14 MRCOM balance is Rs.1 and MRTL balance is Rs.1.Reliance,03,11/05/2017 10:00:00,11/05/2017 10:00:00,11/,11/05/2017 10:00:00,0,01,,255,,333,BalQuery_RCOM,919835853611
Note that the leading space on one of the fields in the first line is preserved.
I didn't come up with that immediately. I generated a more verbose solution like this, first:
#!/usr/bin/env perl -l
use strict;
use warnings;
my $lhs = 13;
my $rhs = 13;
while (<>)
{
chomp;
my(#fields) = split /,/;
my($nflds) = scalar(#fields);
my(#output) = #fields;
if ($nflds > $lhs + $rhs)
{
my(#combine) = #fields[$lhs .. $nflds-$rhs-1];
my $composite = "#combine";
#output = (#fields[0 .. $lhs-1], $composite, #fields[$nflds-$rhs .. $nflds-1]);
}
local $, = ",";
print #output;
}
This produces the same output as the other script. I had the scripts called rs13.pl (verbose) and rs17.pl (compact) and checked them like this (data contains your two lines of input data):
diff <(perl rs13.pl data) <(perl rs17.pl data)
There was no difference.
There are ways to make the compact solution more compact, but I'm not sure they help much.
Here is another version that uses the splice and join functions instead of array slices. In some ways, it is tidier than the other two, but it doesn't have the same protection against lines with too few fields in them.
#!/usr/bin/perl -anlF,
use strict;
use warnings;
my $lhs = 13;
my $rhs = 13;
$, = ","; # Perl is obscure on occasion!
my($nflds) = scalar(#F);
splice(#F, $lhs, $nflds - $lhs - $rhs, join(' ', #F[$lhs .. $nflds-$rhs-1]));
print #F;
It produces the same result as the other two scripts.
Yes, you could write the code in Awk; it wouldn't be as compact as this.
i'm confused about the $symbol for unix.
according to the definition, it states that it is the value stored by the variable following it. i'm not following the definition - could you please give me an example of how it is being used?
thanks
You define a variable like this:
greeting=hello
export name=luc
and use like this:
echo $greeting $name
If you use export that means the variable will be visible to subshells.
EDIT: If you want to assign a string containing spaces, you have to quote it either using double quotes (") or single quotes ('). Variables inside double quotes will be expanded whereas in single quotes they won't:
axel#loro:~$ name=luc
axel#loro:~$ echo "hello $name"
hello luc
axel#loro:~$ echo 'hello $name'
hello $name
In case of shell sctipts. When you assign a value to a variable you does not need to use $ simbol. Only if you want to acces the value of that variable.
Examples:
VARIABLE=100000;
echo "$VARIABLE";
othervariable=$VARIABLE+10;
echo $othervariable;
The other thing: if you use assignment , does not leave spaces before and after the = simbol.
Here is a good bash tutorial:
http://linuxconfig.org/Bash_scripting_Tutorial
mynameis.sh:
#!/bin/sh
finger | grep "`whoami` " | tail -n 1 | awk '{FS="\t";print $2,$3;}'
finger: prints all logged in user example result:
login Name Tty Idle Login Time Office Office Phone
xuser Forname Nickname tty7 3:18 Mar 9 07:23 (:0)
...
grep: filter lines what containing the given string (in this example we need to filter xuser if our loginname is xuser)
http://www.gnu.org/software/grep/manual/grep.html
whoami: prints my loginname
http://linux.about.com/library/cmd/blcmdl1_whoami.htm
tail -n 1 : shows only the last line of results
http://unixhelp.ed.ac.uk/CGI/man-cgi?tail
the awk script: prints the second and third column of the result: Forname, Nickname
http://www.staff.science.uu.nl/~oostr102/docs/nawk/nawk_toc.html
I now have a full path for a file as a string like:
"/db/Liebherr/Content_Repository/Techpubs/Topics/HyraulicPowerDistribution/Released/TRN_282C_HYD_MOD_1_Drive_Shaft_Rev000.xml"
However, now I need to take out only the folder path, so it will be the above string without the last back slash content like:
"/db/Liebherr/Content_Repository/Techpubs/Topics/HyraulicPowerDistribution/Released/"
But it seems that the substring() function in xQuery only has substring(string,start,len) or substring(string,start), I am trying to figure out a way to specify the last occurence of the backslash, but no luck.
Could experts help? Thanks!
Try out the tokenize() function (for splitting a string into its component parts) and then re-assembling it, using everything but the last part.
let $full-path := "/db/Liebherr/Content_Repository/Techpubs/Topics/HyraulicPowerDistribution/Released/TRN_282C_HYD_MOD_1_Drive_Shaft_Rev000.xml",
$segments := tokenize($full-path,"/")[position() ne last()]
return
concat(string-join($segments,'/'),'/')
For more details on these functions, check out their reference pages:
fn:tokenize()
fn:string-join()
fn:replace can do the job with a regular expression:
replace("/db/Liebherr/Content_Repository/Techpubs/Topics/HyraulicPowerDistribution/Released/TRN_282C_HYD_MOD_1_Drive_Shaft_Rev000.xml",
"[^/]+$",
"")
This can be done even with a single XPath 2.0 (subset of XQuery) expression:
substring($fullPath,
1,
string-length($fullPath) - string-length(tokenize($fullPath, '/')[last()])
)
where $fullPath should be substituted with the actual string, such as:
"/db/Liebherr/Content_Repository/Techpubs/Topics/HyraulicPowerDistribution/Released/TRN_282C_HYD_MOD_1_Drive_Shaft_Rev000.xml"
The following code tokenizes, removes the last token, replaces it with an empty string, and joins back.
string-join(
(
tokenize(
"/db/Liebherr/Content_Repository/Techpubs/Topics/HyraulicPowerDistribution/Released/TRN_282C_HYD_MOD_1_Drive_Shaft_Rev000.xml",
"/"
)[position() ne last()],
""
),
"/"
)
It seems to return the desired result on try.zorba-xquery.com. Does this help?