I have a list of 87 text files populated daily in UNIX.
Eg: FP*.txt --- file names start with FP & are txt files
Daily new content gets appended to the end. I want to take a back up of all the text files using shell script and create blank files of the same name. I tried the following.
echo "" > FP*.txt
and > FP*.txt
Both gave the same error.
FP*.txt: ambiguous redirect
however when used on single file both work fine. So how to resolve it. And also creating a list of hard-coded names is not an option, as 87 are too many and they may increase in future.
Also some file names contain ( & , in their names. When tried the following codes,
echo "" > FP---Sample.txt
This worked fine. But when tried the following gave error.
echo "" > FP(1)---Sample.txt
This gave an error.
syntax error near unexpected token `('
So how to accomplish it??
Wrap the filename in quotes:
echo "" > "FP(1)---Sample.txt"
You get a
FP*.txt: ambiguous redirect
because you can just redirect to one file, while FP*.txt expands to more than one. That's why it works when the expansion just returns one name.
To do what you need, you'd better loop through the results of FP*.txt. For example, you can use:
bk_date=$(date "+%Y%m%d_%H%M%S") # 20140210_150526 format of date
for file in FP*.txt
do
cp -p "$file" "${file}.${bk_date}" # make a backup of the file
> "$file" # blank it. This is same as echo "" > "$file"
done
Regarding the error:
syntax error near unexpected token `('
Martin Dinov correctly explained that you need to quote the name to make it be part of the file name.
Related
As part of our process, we get an input file in the .gz format. We need to unzip this file and add some suffix at the end of the file. The input file has timestamp so I am trying to use filter while unzipping and renaming this file.
Input file name :
Mem_Enrollment_20200515130341.dat.gz
Step 1:
Unzipping this file : (working as expected)
gzip -d Mem_Enrollment_*.dat.gz
output :
Mem_Enrollment_20200515130341.dat
Step 2: Renaming this file : (issues while renaming)
Again, I am going with the pattern but I know this won't work in this case. So, what should I do rename this file?
mv Mem_Enrollment_*.dat Mem_Enrollment_*.dat_D11
output :
Mem_Enrollment_*.dat_D11
expected output :
Mem_Enrollment_20200515130341.dat_D11
try
for fn in Mem_Enrollment_*.dat
do
mv ${fn} ${fn}_D11;
done
With just datastage you could loop over ls output from an execute command stage via "ls Mem_Enrollment_*.dat.gz" and then use an #FM as a delimiter when looping the output list. You could then breakout the gzip and rename into two separate commands, which helps with readability in your job.
Only caveat here is that the Start Loop stage doesn't accept the #FM in the delimiter due to some internal funkyness inside Datastage. So you need to set a user variable equal to it and pass that to the mark.
We have a file which has been processed by unix command for removing duplicates. After the de-duplication new file has the header in-between the records. Please help to solve this and thanks in advance for inputs.
Unix Command : Sort -u >
I would do something like this:
grep "headers" >output.txt
grep -v "headers" >>output.txt
The idea is the following: first take the headers and put them into output.txt, and afterwards take everything which is not a header and put it into that output file.
First you need to put the information in the output file (which means you need to create the output file, hence the single > character), secondly you need to append the information to the already existing output file (hence the double >> character).
I'm writing my first script that takes a file and moves it to another folder, except that I want to change the filename of the file to filename_inode instead of just filename incase there are any files with the same name
I've figured out how to show this by creating the following 4 variables
inode=$(ls -i $1 | cut -c1-7) #lists the file the user types, cuts the inode from it
space="_" #used to put inbetween the filename and bname
bname=$(basename $1) #gets the basename of the file without the directory etc
bnamespaceinode=$bname$space$inode #combines the 3 values into one variable
echo "$bnamespaceinode #prints filename_inode to the window
So the bottom echo shows filename_inode which is what I want, except now when I try to move this using mv or cp i'm getting the following errors
I dont think it's anything wrong with the syntax i'm using for the mv and cv commands, and so I'm thinking I need to concatenate the 3 variables into a new file or use the result of the first and then append the other 2 to that file?
I've tried both of the above but still not having any luck, any ideas?
Thanks
Without clearer examples, I guess this could work:
$TARGETDIR=/my/target/directory
mv $1 $TARGETDIR/$(basename "$1" | sed 's/_.*/_inode/')
I want to move a file called dog to $HOME/deleted2.
The unix command I use is:
mv dog $HOME/deleted2
However I want to move it to the exact same destination but this time $HOME/deleted2 is stored in a hidden file called .rm.cfg
I want to extract the location from .rm.cfg, this file contains one line which says $HOME/deleted2.
Here is what I did:
pathname=$(cat $HOME/.rm.cfg),
mv dog $pathname.
However this time I get an error saying $HOME/deleted2 does not exist. Why is this?
Sorry for not putting it in code format, I tried to indent by fours spaces but it did not work.
cat $HOME/.rm.cfg will only "outputs" the raw file, but it does not evaluate variables.
To put the full interpreted string in your pathname variable, you need to evaluate it:
pathname=$(eval echo $(cat $HOME/.rm.cfg))
I do not want:
$ cat file > dummy; $ cat header dummy > file
I want similar to the command below but to the beginning, not to the end:
$ cat header >> file
You can't append to the beginning of a file without rewriting the file. The first way you gave is the correct way to do this.
This is easy to do in sed if you can embed the header string directly in the command:
$ sed -i "1iheader1,header2,header3"
Or if you really want to read it from a file, you can do so with bash's help:
$ sed -i "1i$(<header)" file
BEWARE that "-i" overwrites the input file with the results. If you want sed to make a backup, change it to "-i.bak" or similar, and of course always test first with sample data in a temp directory to be sure you understand what's going to happen before you apply to your real data.
The whole dummy file thing is pretty annoying. Here's a 1-liner solution that I just tried out which seems to work.
echo "`cat header file`" > file
The ticks make the part inside quotes execute first so that it doesn't complain about the output file being an input file. It seems related to hhh's solution but a bit shorter. I suppose if the files are really large this might cause problems though because it seems like I've seen the shell complain about the ticks making commands too long before. Somewhere the part that is executed first must be stored in a buffer so that the original can be overwritten, but I'm not enough of an expert to know what/where that buffer would be or how large it could be.
You can't prepend to a file without reading all the contents of the file and writing a new file with your prepended text + contents of the file. Think of a file in Unix as a stream of bytes - it's easy to append to an end of a stream, but there is no easy operation to "rewind" the stream and write to it. Even a seek operation to the beginning of the file will overwrite the beginning of with any data you write.
One possibility is to use a here-document:
cat > "prependedfile" << ENDENDEND
prepended line(s)
`cat "file"`
ENDENDEND
There may be a memory limitation to this trick.
Thanks to right searchterm!
echo "include .headers.java\n$(cat fileObject.java )" > fileObject.java
Then with a file:
echo "$(cat .headers.java)\n\n$(cat fileObject.java )" > fileObject.java
if you want to pre-pend "header" to "file" why not append "file" to "Header"
cat file >> header
Below is a simple c-shell attempt to solve this problem. This "prepend.sh" script takes two parameters:
$1 - The file containing the pre-appending wording.
$2 - The original/target file to be modified.
#!/bin/csh
if (if ./tmp.txt) then
rm ./tmp.txt
endif
cat $1 > ./tmp.txt
cat $2 >> ./tmp.txt
mv $2 $2.bak
mv ./tmp.txt $2