Issue while renaming a file with file pattern in unix - unix

As part of our process, we get an input file in the .gz format. We need to unzip this file and add some suffix at the end of the file. The input file has timestamp so I am trying to use filter while unzipping and renaming this file.
Input file name :
Mem_Enrollment_20200515130341.dat.gz
Step 1:
Unzipping this file : (working as expected)
gzip -d Mem_Enrollment_*.dat.gz
output :
Mem_Enrollment_20200515130341.dat
Step 2: Renaming this file : (issues while renaming)
Again, I am going with the pattern but I know this won't work in this case. So, what should I do rename this file?
mv Mem_Enrollment_*.dat Mem_Enrollment_*.dat_D11
output :
Mem_Enrollment_*.dat_D11
expected output :
Mem_Enrollment_20200515130341.dat_D11

try
for fn in Mem_Enrollment_*.dat
do
mv ${fn} ${fn}_D11;
done

With just datastage you could loop over ls output from an execute command stage via "ls Mem_Enrollment_*.dat.gz" and then use an #FM as a delimiter when looping the output list. You could then breakout the gzip and rename into two separate commands, which helps with readability in your job.
Only caveat here is that the Start Loop stage doesn't accept the #FM in the delimiter due to some internal funkyness inside Datastage. So you need to set a user variable equal to it and pass that to the mark.

Related

zsh: create named file in place of argument?

realpath <<<'foo' fails "realpath: missing operand". I don't know what that means.
realpath <(<<<'foo') returns /proc/3443695/fd/pipe:[26244650] which I guess means it's creating a temporary pipe which will contain the string "foo".
Or maybe printf is more clear:
❯ printf "%q" <<<'foo' # no output
❯ printf "%q" <(<<<'foo')
/proc/self/fd/11%
The actual program I'm trying to call doesn't like either of those. I think I need an actual file.
I can do that in multiple commands by creating a file with mktemp and then writing to it, and then sending that off as the arg, but does zsh have any convenient syntax for doing this in-place? A 1-liner?
It looks like the =(list) process substitution should do what you want.
From the zshexpn man page:
If =(...) is used instead of <(...), then the file passed as an
argument will be the name of a temporary file containing the output
of the list process. This may be used instead of the < form for a
program that expects to lseek on the input file.
...
The temporary file created by the process substitution will be deleted when the function exits.
On my system, realpath =(<<<'foo') returns something like /private/tmp/zsh3YAdDx, i.e. the name of a temporary file that does indeed appear to be deleted after executing the command.
As a bonus, the documentation notes that in some cases the =(<<<...) form is optimized to execute completely in the current shell.

UNIX split command splitting this file, but what names are resulting?

We receive a big csv file from a client (500k lines, est) that we split into smaller chunks using the split command.
You can see how we're using the command below, but my bash knowledge is a bit rusty, could someone refresh me on the ${processFile}_ bit below, and how the files are being named in the end? Not recalling what the underscore does...
split -l 50000 $PROCESSING_CURRENT_DIR/$processFile ${processFile}_
This isn't anything to do with bash but how split(1) command processes its arguments to split the input.
Syntax is:
split [OPTION]... [FILE [PREFIX]]
DESCRIPTION
Output pieces of FILE to PREFIXaa, PREFIXab, ...; default size is 1000 lines, and default PREFIX is 'x'.
With no FILE, or when FILE is -, read standard input.
So it uses the given prefix and makes output files.

How to handle a file having header in between the records after removing duplicates from the file

We have a file which has been processed by unix command for removing duplicates. After the de-duplication new file has the header in-between the records. Please help to solve this and thanks in advance for inputs.
Unix Command : Sort -u >
I would do something like this:
grep "headers" >output.txt
grep -v "headers" >>output.txt
The idea is the following: first take the headers and put them into output.txt, and afterwards take everything which is not a header and put it into that output file.
First you need to put the information in the output file (which means you need to create the output file, hence the single > character), secondly you need to append the information to the already existing output file (hence the double >> character).

Check if a file is open using Windows command line within R

I am using the shell() command to generate pdf documents from .tex files within a function. This function sometimes gets ran multiple times with adjusted data and so will overwrite the documents. Of course, if the pdf file is open when the .tex file is ran, it generates an error saying it can't run the .tex file. So I want to know whether there are any R or Windows cmd commands which will check whether a file is open or not?
I'm not claiming this as a great solution: it is hacky but maybe it will do. You can make a copy of the file and try to overwrite your original file with it. If it fails, no harm is made. If it succeeds, you'll have modified the file's info (not the contents) but since your end goal is to overwrite it anyway I doubt it will be a huge problem. In either case, you'll be fixed about whether or not the file can be rewritten.
is.writeable <- function(f) {
tmp <- tempfile()
file.copy(f, tmp)
success <- file.copy(tmp, f)
return(success)
}
openfiles /query /v|(findstr /i /c:"C:\Users\David Candy\Documents\Super.xls"&&echo File is open||echo File isn't opened)
Output
592 David Candy 1756 EXCEL.EXE C:\Users\David Candy\Documents\Super.xls
File is open
Findstr returns 0 if found and 1+ if not found or error.
& seperates commands on a line.
&& executes this command only if previous command's errorlevel is 0.
|| (not used above) executes this command only if previous command's errorlevel is NOT 0
> output to a file
>> append output to a file
< input from a file
| output of one command into the input of another command
^ escapes any of the above, including itself, if needed to be passed to a program
" parameters with spaces must be enclosed in quotes
+ used with copy to concatinate files. E.G. copy file1+file2 newfile
, used with copy to indicate missing parameters. This updates the files modified date. E.G. copy /b file1,,
%variablename% a inbuilt or user set environmental variable
!variablename! a user set environmental variable expanded at execution time, turned with SelLocal EnableDelayedExpansion command
%<number> (%1) the nth command line parameter passed to a batch file. %0 is the batchfile's name.
%* (%*) the entire command line.
%<a letter> or %%<a letter> (%A or %%A) the variable in a for loop. Single % sign at command prompt and double % sign in a batch file.
.
--

What is the usage of braces in unix commands?

I have been looking all over for how to use the {} feature in unix filenames. For example, I have a directory that has subdirectories named as an ID#. Each subdirectory has a recorded audio file named as a specific test question ID#. The following command:
for file in */1-01*; do echo "$file"; done
will list the names of the audio files within each subdirectory of the current directory. Doing so gives me a list:
809043250/1-01-20131.wav
813777079/1-01-20131.wav
817786199/1-01-20131.wav
827832538/1-01-20131.wav
834820477/1-01-20131.wav
I want to rename each of the above .wav files as a different ID#, so it should end up like this:
809043250/5001.wav
813777079/5001.wav
817786199/5001.wav
827832538/5001.wav
834820477/5001.wav
So how do I use the ${file/firstOccurance/replaceWith} notation if I want to use the mv command to keep the person's ID# (809043250, 813777079, etc) and the first / but strip off the 1-01-20131.wav and tack on 5001.wav ?
I don't know how to search for this on google. Is it called regular expressions (i don't think so...)? Brace notation? filename patterns? Does anyone know of a good explanation of these?
Thanks!
In bash, one variation on the notation is called Brace Expansion. This generates file names.
The other variation on the notation is called Shell Parameter Expansion. Here, bash provides a regex-ish substitute notation ${var/match/replace}.
For your purposes, you need the paremeter expansion:
$ x="809043250/1-01-20131.wav
> 813777079/1-01-20131.wav
> 817786199/1-01-20131.wav
> 827832538/1-01-20131.wav
> 834820477/1-01-20131.wav"
$ for file in $x; do echo $file ${file/1-01-20131/5001}; done
809043250/1-01-20131.wav 809043250/5001.wav
813777079/1-01-20131.wav 813777079/5001.wav
817786199/1-01-20131.wav 817786199/5001.wav
827832538/1-01-20131.wav 827832538/5001.wav
834820477/1-01-20131.wav 834820477/5001.wav
$

Resources