I have created a script in GAWK, which produces a log file.
I have a number of conditions throughout the script for error handling, and these errors are output to the log file if triggered. I also have a counter which increments when ever an error is encountered. At the end of the script I have the following:
if (intCounter) {
print "the file contains " intCounter " errors, please see below for details" >> LOG_FILE
} else {
print "no errors, operation successful" >> LOG_FILE
}
This is being printed at the end of my log file, in GAWK is there any way of printing something as the first record in a file if that file already contains text?
Thanks,
Shaw
Here is an example:
cat file
data
some other data
here vi have out pattern
this should work
awk 'FNR==NR {if (/pattern/) f=1;next} f {print "This file has pattern";f=0} 1' file file
This file has pattern
data
some other data
here vi have out pattern
this should work
Related
Can any one please explian me the below bteq code.
Is this script valid?
exec 1> $CODE/edlr2/logs/AGP_MBR_BTEQ_CSA_MBR_STG_LOAD_$(date +"%Y%m%d_%H%M%S").log 2>&1`echo "script file =" $0 PARM_FILE=$1 echo "parm file= "$PARM_FILE.parm . $CODE/edlr2/scripts/$PARM_FILE.parm select name from customer;
Can anyone please explain this code
See: https://superuser.com/questions/436586/why-redirect-output-to-21-and-12
exec 1> $CODE/edlr2/logs/AGP_MBR_BTEQ_CSA_MBR_STG_LOAD_$(date +"%Y%m%d_%H%M%S").log
This writes to a log file
2>&1 `echo "script file =" $0 PARM_FILE=$1 echo "parm file= "$PARM_FILE.parm . $CODE/edlr2/scripts/$PARM_FILE.parm select name from customer;
2>&1 points the file descriptor #2 to where #1 (above) is already pointing (the .log file).
However it looks like you're missing an ending grave ` somewhere above since you start one before echo but never close it. So I don't think that script is valid. But I also know nothing about how your database is setup to evaluate if the rest is valid. Unless you can give specific errors and information about how your files are setup and doing, it's hard to help you.
Additional info: exec will run a script at a location, and so part:
echo "script file =" $0 PARM_FILE=$1 echo "parm file= "$PARM_FILE.parm . $CODE/edlr2/scripts/$PARM_FILE.parm select name from customer;
is essentially running a command script and logging it to a log file. It would output and run something like:
script file=/var/somefile
parm file=/var/someparms.parm
. /var/anotherparmfile.parm select name from customer;
What is exec
What is a dot command
As is, it is neither a unix script, nor some code, nor something bteq could use.
My guess would be, your 'script' looks like this (dismissed the lonely ` as typing error)
exec 1> $CODE/edlr2/logs/AGP_MBR_BTEQ_CSA_MBR_STG_LOAD_$(date +"%Y%m%d_%H%M%S").log 2>&1
echo "script file =" $0
PARM_FILE=$1
echo "parm file= "$PARM_FILE.parm
. $CODE/edlr2/scripts/$PARM_FILE.parm select name from customer;
As #Xander already guessed it would redirect output to a log-file and print info about script and logfile name and then execute the script $PARM_FILE.parm with some parameters.
Further guessing, because BTEQ is mentioned in the name for the log file, in that .parm script bteq may be used to execute a SQL-command which is passed to it as parameters.
bteq needs a logon command. If that is added in the .parm script, before the concatenated parameters, and that passed to bteq, you may get some meaningfull response.
Be aware, that the ; at the end would never be passed to the script. The shell would take it as end of command token. And the .parm script would have to add the ; too to construct a valied SQL-command.
Why a dot-command is used to execute a script, which is named .parm is beyond my imagination.
Hi I have a list here:
list_1.txt
Alpha
Bravo
Charlie
and files in a directory with the following filenames and contents:
Alpha_123.log
This is a sample line in the file
error_log "This is error1 in file"
This is another sample line in the file
This is another sample line in the file
This is another sample line in the file
error_log "This is error2 in file"
This is another sample line in the file
This is another sample line in the file
error_log "This is error3 in file"
This is another sample line in the file
This is another sample line in the file
Alpha_123.sh
This is a sample line in the file
This is another sample line in the file
This is another sample line in the file
error_log "This is errorA in file"
This is another sample line in the file
This is another sample line in the file
This is another sample line in the file
This is another sample line in the file
error_log "This is errorB in file"
This is another sample line in the file
This is another sample line in the file
error_log "This is errorC in file"
This is another sample line in the file
Bravo.log
Charlie.log
contents of Bravo.log and Charlie.log is similar to Alpha.log
I would like to have an output like this:
Alpha|"This is error1 in file"
Alpha|"This is error2 in file"
Alpha|"This is error3 in file"
Alpha|"This is errorA in file"
Alpha|"This is errorB in file"
Alpha|"This is errorC in file"
Any inputs is greatly appreciated. Thank you!
So basically, I would like to find first the files with names containing the string patterns in list_1.txt, then find the error messages and output with |
awk to the rescue!
awk '{gsub(/^error_log /,FILENAME"|")}1' $(awk '{print $0".log"}' list_1.txt)
UPDATE
Based on the updated info, I think that's what you're looking for.
awk '/^error_log/ {split(FILENAME,f,"_");
gsub(/^error_log /,f[1]"|")}' $(awk '{print $0"_*"}' list_1.txt)
If I understood correctly, this should do:
awk -vOFS=\| 'FNR==1{file=FILENAME;sub("[.]log$","",file)}{sub("^error_log ","");print file,$0}' *.log
explanation:
-vOFS=\| sets the output field separator to |. (The \ is needed to escape the | from the shell (which would treat it as pipe). You could use -vOFS='|' instead.)
FNR==1{...} makes sure this code is run only once per input file: FNR is number of records (i.e lines) read by awk so far from the current file. So this is only equal to 1 when processing the very first line of each file.
file=FILENAME just stores the filename of the currently processed input file in a variable for later editing.
sub("[.]log$","",file) removes .log (the [...] escape the dot (.) from being interpreted as any character in the regular expression. You could use \\. instead.) from the end (that's what the $ stands for) of the filename.
{...} runs the code for every record/line of each input file.
sub("^error_log ","") removes "error_log " (note the trailing space!) from the beginning (that's what the ^ stands for) of each line ("record") of the input.
print file,$0 prints the remainder of each record (i.e. line) prefixed by the corresponding filenames. Note that the comma (,) will be replaced by the output field separator specified earlier. You could use print file "|" $0 instead without specifying the OFS.
*.log will make every file ending in .log in the current directory an input file for the awk command. You could specify Alpha.log Bravo.log Charly.log explicitely instead.
Here is an alternative using your list.txt to construct the filenames:
awk -vOFS=\| '{file=$0;logfile=file ".log";while(getline < logfile){sub("^error_log ","");print file,$0}}' list.txt
explanation:
file=$0 saves the current line (record) from list.txt in a variable.
logfile=file ".log" appends .log to it to get the corresponding log filename.
while(getline < logfile){...} will run the code for each line/record in the current log file.
The rest should be clear from the above example.
I have many large text files, and I would like to add a line at the very beginning. I saw someone had asked this already here. However, this involves reading the entire text file and appending it to the single line. Is there a better (faster) way?
I tested this on windows 7 and it works. Essentially, you use the shell function and do everything on the windows cmd which is quite fast.
write_beginning <- function(text, file){
#write your text to a temp file i.e. temp.txt
write(text, file='temp.txt')
#print temp.txt to a new file
shell(paste('type temp.txt >' , 'new.txt'))
#append your file to new.txt
shell(paste('type', file, '>> new.txt'))
#remove temp.txt - I use capture output to get rid of the
#annoying TRUE printed by file.remove
dump <- capture.output(file.remove('temp.txt'))
#uncomment the last line below to rename new.txt with the name of your file
#and essentially modify your old file
#dump <- capture.output(file.rename('new.txt', file))
}
#assuming your file is test.txt and you want to add 'hello' at the beginning just do:
write_beginning('hello', 'test.txt')
On linux you just need to find the corresponding command in order to send a file to another one (I really think you need to replace type by cat on linux but I cannot test right now).
You'd use the system() function on a Linux distro:
system('cp file.txt temp.txt; echo " " > file.txt; cat temp.txt >> file.txt; rm temp.txt')
I am using the shell() command to generate pdf documents from .tex files within a function. This function sometimes gets ran multiple times with adjusted data and so will overwrite the documents. Of course, if the pdf file is open when the .tex file is ran, it generates an error saying it can't run the .tex file. So I want to know whether there are any R or Windows cmd commands which will check whether a file is open or not?
I'm not claiming this as a great solution: it is hacky but maybe it will do. You can make a copy of the file and try to overwrite your original file with it. If it fails, no harm is made. If it succeeds, you'll have modified the file's info (not the contents) but since your end goal is to overwrite it anyway I doubt it will be a huge problem. In either case, you'll be fixed about whether or not the file can be rewritten.
is.writeable <- function(f) {
tmp <- tempfile()
file.copy(f, tmp)
success <- file.copy(tmp, f)
return(success)
}
openfiles /query /v|(findstr /i /c:"C:\Users\David Candy\Documents\Super.xls"&&echo File is open||echo File isn't opened)
Output
592 David Candy 1756 EXCEL.EXE C:\Users\David Candy\Documents\Super.xls
File is open
Findstr returns 0 if found and 1+ if not found or error.
& seperates commands on a line.
&& executes this command only if previous command's errorlevel is 0.
|| (not used above) executes this command only if previous command's errorlevel is NOT 0
> output to a file
>> append output to a file
< input from a file
| output of one command into the input of another command
^ escapes any of the above, including itself, if needed to be passed to a program
" parameters with spaces must be enclosed in quotes
+ used with copy to concatinate files. E.G. copy file1+file2 newfile
, used with copy to indicate missing parameters. This updates the files modified date. E.G. copy /b file1,,
%variablename% a inbuilt or user set environmental variable
!variablename! a user set environmental variable expanded at execution time, turned with SelLocal EnableDelayedExpansion command
%<number> (%1) the nth command line parameter passed to a batch file. %0 is the batchfile's name.
%* (%*) the entire command line.
%<a letter> or %%<a letter> (%A or %%A) the variable in a for loop. Single % sign at command prompt and double % sign in a batch file.
.
--
I am just splitting a very large csv file in to parts. When ever i run the following command. the doesn't completely split rather returns me the following error. how can i avoid the split the whole file.
awk -F, '{print > $2}' test1.csv
awk: YY1 makes too many open files
input record number 31608, file test1.csv
source line number 1
Just close the files after writing:
awk -F, '{print > $2; close($2)}' test1.csv
You must have a lot of lines. Are you sure that the second row repeats enough to put those records into an individual file? Anyway, awk is holding the files open until the end. You'll need a process that can close the file handles when not in use.
Perl to the rescue. Again.
#!perl
while( <> ) {
#content = split /,/, $_;
open ( OUT, ">> $content[1]") or die "whoops: $!";
print OUT $_;
close OUT;
}
usage: script.pl your_monster_file.csv
outputs the entire line into a file named the same as the value of the second CSV column in the current directory, assuming no quoted fields etc.