I am newbie to unix command usage.
I would like to ask if there is a way to copy multiple files from multiple directories into 1 new directory?
example:
in /tmp/dirA --> it contains file A.run.log and A.skip.log
in /tmp/dirB--> it contains file B.run.log and B.skip.log
in /tmp/dirC --> it contains file C.run.log and C.skip.log
and i would like to have all
A.run.log
A.skip.log
B.run.log
B.skip.log
C.run.log
C.skip.log
into a new folder called /tmp/dirNew
Is there a unix command that able to do it? Really appreciate it. Thank you.
JS
use this :
cp /tmp/dir?/* /tmp/dirNew
find /tmp -path "/tmp/dirNew" -prune -o -name '*.run.log' -exec cp {} /tmp/dirNew/ \;
find /tmp -path "/tmp/dirNew" -prune -o -name '*.skip.log' -exec cp {} /tmp/dirNew/ \;
Related
My use case is I want to search a collection of JARs for a specific class file. More specifically, I want to search recursively within a directory for all *.jar files, then list their contents, looking for a specific class file.
So this is what I have so far:
find . -name *.jar -type f -exec echo {} \; -exec jar tf {} \;
This will list the contents of all JAR files found recursively. I want to put a grep within the seconed exec because I want the second exec to only print the contents of the JAR that grep matches.
If I just put a pipe and pipe it all to grep afterward, like:
find . -name *.jar -type f -exec echo {} \; -exec jar tf {} \; | grep $CLASSNAME
Then I lose the output of the first exec, which tells me where the class file is (the name of JAR file is likely to not match the class file name).
So if there was a way for the exec to run two commands, like:
-exec "jar tf {} | grep $CLASSNAME" \;
Then this would work. Using a grep $(...) in the exec command wouldn't work because I need the {} from the find to take the place of the file that was found.
Is this possible?
(Also I am open to other ways of doing this, but the command line is preferred.)
i find it difficult to execute multiple commands within find-exec, so i usually only grab the results with find and loop around the results.
maybe something like this might help?
find . -type f -name *.jar | while read jarfile; do echo $jarfile; jar tf $jarfile; done
I figured it out - still using "one" command. What I was looking for was actually answered in the question How to use pipe within -exec in find. What I have to do is use a shell command with my exec. This ends up making the command look like:
find . -name *.jar -type f -exec echo {} \; -exec sh -c "jar tf {} | grep --color $CLASSNAME" \;
The --color will help the final result to stick out while the command is recursively listing all JAR files.
A couple points:
This assumes I have a $CLASSNAME set. The class name has to appear as it would in a JAR, not within a Java package. So com.ibm.json.java.JSONObject would become com/ibm/json/java/JSONObject.class.
This requires a JDK - that is where we get the jar command. The JDK must be accessible on the system path. If you have a JDK that is not on the system path, you can set an environment variable, such as JAR to point to the jar executable. I am running this from cygwin, so it turns out my jar installation is within the "Program Files" directory. The presence of a space breaks this, so I have to add these two commands:
export JAR=/cygdrive/c/Program\ Files/Java/jdk1.8.0_65/bin/jar
find . -name *.jar -type f -exec echo {} \; -exec sh -c "\"$JAR\" tf {} | grep --color $CLASSNAME" \;
The $JAR in the shell command must be escaped otherwise the terminal will not know what to do with the space in "Program Files".
I had a directory with many files and sub-directories. To move only the sub-directories, I just learned you can use:
ls -d BASEDIR/*/ | xargs -n1 -I% mv % TARGETDIR/
I use the following:
$ mv ./*/ DirToMoveTo
For example:
Say I wanted to move all directories with "Old" in the name to a folder called "Old_Dirs" on /data.
The command would look like this:
mv ./*Old*/ /data/
Why not use find?
find . -maxdepth 1 -type d -exec mv '{}' /tmp \;
-maxdepth 1 makes sure find won't go deeper than current directory
-type d tells find to only find directories
-exec execute a command with the result of the find referenced by {}
In my opinion a cleaner solution and it also works better then using xargs when you have files with white space or tabs in them.
With a file structure like this:
/dir2move2
/dir
/subdir1
/subdir2
index.js
To move only the sub directories and not the files you could just do:
mv ./dir/*/ ./dir2move2
Possible solution:
find BASEDIR/ -maxdepth 1 -mindepth 1 -type d -exec mv '{}' TARGETDIR \;
you can simply use the same command for moving a file but put a slash after the name of the subdirectory
sudo mv */ path/to/destination
sudo mv subdir/ path/to/subdirectory
I have a folder named "in" that contains several folders "a" "b" "c" and I want to move all files to thhe folder "proc" and compress them. The tricky part is the files in "in/a" have to be moved to "proc/a", "in/b" have to be moved to "proc/b" and so on
I managed to find all files and zip them whit this command
find . -type f ! \( -name "*gz" -o -name "*tmp" -o -name "*xftp" \) -exec gzip -n '{}' \;
But I'm not finding a generic command to move the files that works whiteout me telling the name of the folders. Can anyone give me a hand?
Well I ended up finding out I had a couple more problems for example the target folder not existing so I ended up using this code
find . -type f ! \( -name "*gz" -o -name "*tmp" -o -name "*xftp" \) -exec gzip -n '{}' \;
find . -name "*.gz" | cpio -p -dumv $1
if [ "$?" = "0" ]; then
find . -name "*.gz" -exec rm -rf {} \;
else
echo "cpio Failed!" 1>&2
exit 1
fi
the 1st line finds all files to be processed and zips them.
the second line finds all files and copies to the target dir, in my case it was $1 (argument 1), creating as many folders as necessary to ensure the same structure.
The third line checks the status of the last command if it worked it finds and removes all gz files from the source folder whiteout deleting any folder. If it didn't deletes nothing so I can analyse what happened (maybe run out of space)
I bet there's a faster way of doing this whiteout having to use so much disk space but since that was not a problem for me it looks acceptable.
I am using AIX.
When I try to copy all the file in a folder to another folder with the following command:
cp ./00012524/*.PDF ./dummy01
The shell complains:
ksh: /usr/bin/cp: 0403-027 The parameter list is too long.
How to deal with it? My folder contain 8xxxx files, how can I copy them very fast? each file have size of 4x kb to 1xx kb.
Use find command in *nix:
find ./00012524 -type f -name "*.PDF" -exec cp {} ./dummy01/ \; -print
The cp command has a limitation of files which you can copy simultaneous.
One possibility you can copy them using multiple times the cp command bases on your file patterns, like:
cp ./00012524/A*.PDF ./dummy01
cp ./00012524/B*.PDF ./dummy01
cp ./00012524/C*.PDF ./dummy01
...
cp ./00012524/*.PDF ./dummy01
You can also copy trough find command:
find ./00012524 -name "*.PDF" -exec cp {} ./dummy01/ \;
$ ( cd 00012524; ls | grep '\.PDF$' | xargs -I{} cp {} ../dummy01/ )
The -t flag to cp is useful here:
find ./00012524 -name \*.PDF -print | xargs cp -t ./dummy01
The best command to copy large number of files from one directory to another.
find /path/to/source/ -name "*" -exec cp -ruf "{}" /path/to/destination/ \;
This helped me a lot.
You should be able use a for loop, e.g.
for f in $(ls ./00012524/*.pdf)
do
cp $f ./dummy01
done
I have no way of testing this, but it should work in theory.
you can do something like this and grab each line of the directory
# you can use the -rv to check the status of the command verbose
for i in /from_dir/*; do cp -rv "$i" /to_dir/; done
The unzip command doesn't have an option for recursively unzipping archives.
If I have the following directory structure and archives:
/Mother/Loving.zip
/Scurvy/Sea Dogs.zip
/Scurvy/Cures/Limes.zip
And I want to unzip all of the archives into directories with the same name as each archive:
/Mother/Loving/1.txt
/Mother/Loving.zip
/Scurvy/Sea Dogs/2.txt
/Scurvy/Sea Dogs.zip
/Scurvy/Cures/Limes/3.txt
/Scurvy/Cures/Limes.zip
What command or commands would I issue?
It's important that this doesn't choke on filenames that have spaces in them.
If you want to extract the files to the respective folder you can try this
find . -name "*.zip" | while read filename; do unzip -o -d "`dirname "$filename"`" "$filename"; done;
A multi-processed version for systems that can handle high I/O:
find . -name "*.zip" | xargs -P 5 -I fileName sh -c 'unzip -o -d "$(dirname "fileName")/$(basename -s .zip "fileName")" "fileName"'
A solution that correctly handles all file names (including newlines) and extracts into a directory that is at the same location as the file, just with the extension removed:
find . -iname '*.zip' -exec sh -c 'unzip -o -d "${0%.*}" "$0"' '{}' ';'
Note that you can easily make it handle more file types (such as .jar) by adding them using -o, e.g.:
find . '(' -iname '*.zip' -o -iname '*.jar' ')' -exec ...
Here's one solution that extracts all zip files to the working directory and involves the find command and a while loop:
find . -name "*.zip" | while read filename; do unzip -o -d "`basename -s .zip "$filename"`" "$filename"; done;
You could use find along with the -exec flag in a single command line to do the job
find . -name "*.zip" -exec unzip {} \;
This works perfectly as we want:
Unzip files:
find . -name "*.zip" | xargs -P 5 -I FILENAME sh -c 'unzip -o -d "$(dirname "FILENAME")" "FILENAME"'
Above command does not create duplicate directories.
Remove all zip files:
find . -depth -name '*.zip' -exec rm {} \;
Something like gunzip using the -r flag?....
Travel the directory structure recursively. If any of the file names specified on the command line are directories, gzip will descend into the directory and compress all the files it finds there (or decompress them in the case of gunzip ).
http://www.computerhope.com/unix/gzip.htm
If you're using cygwin, the syntax is slightly different for the basename command.
find . -name "*.zip" | while read filename; do unzip -o -d "`basename "$filename" .zip`" "$filename"; done;
I realise this is very old, but it was among the first hits on Google when I was looking for a solution to something similar, so I'll post what I did here. My scenario is slightly different as I basically just wanted to fully explode a jar, along with all jars contained within it, so I wrote the following bash functions:
function explode {
local target="$1"
echo "Exploding $target."
if [ -f "$target" ] ; then
explodeFile "$target"
elif [ -d "$target" ] ; then
while [ "$(find "$target" -type f -regextype posix-egrep -iregex ".*\.(zip|jar|ear|war|sar)")" != "" ] ; do
find "$target" -type f -regextype posix-egrep -iregex ".*\.(zip|jar|ear|war|sar)" -exec bash -c 'source "<file-where-this-function-is-stored>" ; explode "{}"' \;
done
else
echo "Could not find $target."
fi
}
function explodeFile {
local target="$1"
echo "Exploding file $target."
mv "$target" "$target.tmp"
unzip -q "$target.tmp" -d "$target"
rm "$target.tmp"
}
Note the <file-where-this-function-is-stored> which is needed if you're storing this in a file that is not read for a non-interactive shell as I happened to be. If you're storing the functions in a file loaded on non-interactive shells (e.g., .bashrc I believe) you can drop the whole source statement. Hopefully this will help someone.
A little warning - explodeFile also deletes the ziped file, you can of course change that by commenting out the last line.
Another interesting solution would be:
DESTINY=[Give the output that you intend]
# Don't forget to change from .ZIP to .zip.
# In my case the files were in .ZIP.
# The echo were for debug purpose.
find . -name "*.ZIP" | while read filename; do
ADDRESS=$filename
#echo "Address: $ADDRESS"
BASENAME=`basename $filename .ZIP`
#echo "Basename: $BASENAME"
unzip -d "$DESTINY$BASENAME" "$ADDRESS";
done;
You can also loop through each zip file creating each folder and unzip the zip file.
for zipfile in *.zip; do
mkdir "${zipfile%.*}"
unzip "$zipfile" -d "${zipfile%.*}"
done
this works for me
def unzip(zip_file, path_to_extract):
"""
Decompress zip archives recursively
Args:
zip_file: name of zip archive
path_to_extract: folder where the files will be extracted
"""
try:
if is_zipfile(zip_file):
parent_file = ZipFile(zip_file)
parent_file.extractall(path_to_extract)
for file_inside in parent_file.namelist():
if is_zipfile(os.path.join(os.getcwd(),file_inside)):
unzip(file_inside,path_to_extract)
os.remove(f"{zip_file}")
except Exception as e:
print(e)