I want to search number of abc.xml files inside my main directory . those xml files are inside jars.I' m using ubuntu 12.04 LTS . how to search those xml files inside jars using GREP
.I want to know the amount of the total xml files inside that directory
From what I can gather, you want to know the number of XML files contained within a .jar file. If that is right, then the following would achieve the desired:
jar tvf <jarfile> | grep "\.xml$" | wc -l
Assuming that you want to search within all the jar files in the current directory (and subdirectories), the following should do the trick:
total=0;
for i in `find . -type f -name "*.jar"`; do
count=$(jar tvf $i | grep "abc.xml" | wc -l);
let total+=count;
done
echo ${total}
For simply listing the matching jar files:
for i in `find . -type f -name "*.jar"`; do
jar tvf $i | grep "abc.xml";
if [ $? -eq 0 ]; then echo $i; fi;
done
Related
I have a directory named .poco that has subdirectories at different levels.
Some have *.css files in them. Some don't. The following script fails on
line 4 (the second for loop) if the current directory has no .css files
in it. How do I keep the script running if the current directory doesn't happen to have a match to *.css?
#!/bin/zsh
for dir in ~/pococms/poco/.poco/*; do
if [ -d "$dir" ]; then
for file in $dir/*.css # Fails if directory has no .CSS files
do
if [ -f $file ]; then
v "${file}"
fi
done
fi
done
That happens because of "shell globbing". Your shell tries to replace patterns like *.css with the list of files. When no files match the pattern, you get the "error" that you get.
You might want to use find:
find ~/pocoms/poco/.poco -mindepth 2 -maxdepth 2 -type f -name '*.css'
and then xargs to your program (in that case - echo) like:
find ~/pocoms/poco/.poco\
-mindepth 2\
-maxdepth 2\
-type f\
-name '*.css'\
-print0\
| xargs -0 -n1 -I{} echo "{}"
-n1 to pass the files one by one, remove it if you want your program to accept the full list of files as args.
I have a directory structure as below
output/a/1/multipleFiles
output/a/2/multipleFiles
output/a/3/multipleFiles
output/b/1/multipleFiles
output/b/2/multipleFiles
output/b/3/multipleFiles
I want to know number of lines each directory has. So basically, number of lines at each inner most directory level instead of file level. The innermost directories 1, 2, 3 are different kinds of output we generate for our analytics which contains multiple hadoop part-xxxx files.
I moved to output directory and tried the below command.
find . -maxdepth 2 -type d -name '*' | awk -F "/" 'NF==3' | awk '{print $0"/*"}' | xargs wc -l
But I am getting an error as
wc: ./a/1/*: No such file or directory
wc: ./a/2/*: No such file or directory
wc: ./a/3/*: No such file or directory
but if I try
wc -l ./a/1/*
I am getting correct output for that specific folder.
What am I missing here.
EDIT:
I updated my command as below to remove unnecessary awk commands.
find . -mindepth 2 -maxdepth 2 -type d -name '*' | xargs wc -l
This again results in error as
wc: ./a/1: Is a directory
wc: ./a/2: Is a directory
wc: ./a/2: Is a directory
Give a try to execdir, for example:
find . -maxdepth 2 -type f -execdir wc -l {} \;
This will run the command wc -l {} only within the directory that the file has been found, from the man:
-execdir The -execdir primary is identical to the -exec primary with
the exception that utility will be executed from the
directory that holds the current file.
My use case is I want to search a collection of JARs for a specific class file. More specifically, I want to search recursively within a directory for all *.jar files, then list their contents, looking for a specific class file.
So this is what I have so far:
find . -name *.jar -type f -exec echo {} \; -exec jar tf {} \;
This will list the contents of all JAR files found recursively. I want to put a grep within the seconed exec because I want the second exec to only print the contents of the JAR that grep matches.
If I just put a pipe and pipe it all to grep afterward, like:
find . -name *.jar -type f -exec echo {} \; -exec jar tf {} \; | grep $CLASSNAME
Then I lose the output of the first exec, which tells me where the class file is (the name of JAR file is likely to not match the class file name).
So if there was a way for the exec to run two commands, like:
-exec "jar tf {} | grep $CLASSNAME" \;
Then this would work. Using a grep $(...) in the exec command wouldn't work because I need the {} from the find to take the place of the file that was found.
Is this possible?
(Also I am open to other ways of doing this, but the command line is preferred.)
i find it difficult to execute multiple commands within find-exec, so i usually only grab the results with find and loop around the results.
maybe something like this might help?
find . -type f -name *.jar | while read jarfile; do echo $jarfile; jar tf $jarfile; done
I figured it out - still using "one" command. What I was looking for was actually answered in the question How to use pipe within -exec in find. What I have to do is use a shell command with my exec. This ends up making the command look like:
find . -name *.jar -type f -exec echo {} \; -exec sh -c "jar tf {} | grep --color $CLASSNAME" \;
The --color will help the final result to stick out while the command is recursively listing all JAR files.
A couple points:
This assumes I have a $CLASSNAME set. The class name has to appear as it would in a JAR, not within a Java package. So com.ibm.json.java.JSONObject would become com/ibm/json/java/JSONObject.class.
This requires a JDK - that is where we get the jar command. The JDK must be accessible on the system path. If you have a JDK that is not on the system path, you can set an environment variable, such as JAR to point to the jar executable. I am running this from cygwin, so it turns out my jar installation is within the "Program Files" directory. The presence of a space breaks this, so I have to add these two commands:
export JAR=/cygdrive/c/Program\ Files/Java/jdk1.8.0_65/bin/jar
find . -name *.jar -type f -exec echo {} \; -exec sh -c "\"$JAR\" tf {} | grep --color $CLASSNAME" \;
The $JAR in the shell command must be escaped otherwise the terminal will not know what to do with the space in "Program Files".
When we develop locally, we append ".dev" or ".prod" to files that should be made available only to the development/production server respectively.
What I would like to do is; after deploying the site to the server, recursively find all files with the ".dev" suffix (for example) and remove it (renaming the file). How would I go about doing this, preferably entirely in the shell (without scripts) so I can add it to our deployment script?
Our servers run Ubuntu 10.04.
Try this (not entirely shell-only, requires the find and mv utilities):
find . '(' -name '*.dev' -o -name '*.prod' ')' -type f -execdir sh -c 'mv -- "$0" "${0%.*}"' '{}' ';'
If you have the rename and xargs utilities, you can speed this up a lot:
find . '(' -name '*.dev' -o -name '*.prod' ')' -type f -print0 | xargs -0 rename 's/\.(dev|prod)$//'
Both versions should work with any file name, including file names containing newlines.
It's totally untested, but this should work in the POSIX-like shell of your choice:
remove-suffix () {
local filename
while read filename; do
mv "$filename" "$(printf %s "$filename" | sed "s/\\.$1\$//")"
done
}
find -name '*.dev' | remove-suffix .dev
Note: In the very unusual case that one or more of your filenames contains a newline character, this won't work.
for file in `ls *.dev`; do echo "Old Name $file"; new_name=`echo $file | sed -e 's/dev//'` ; echo "New Name $new_name"; mv $file $new_name; done
In an example of something I used recently this code looks for any file that ends with new.xml changes a date in the filename (filenames were of the form xmlEventLog_2010-03-23T11:16:16_PFM_1_1.xml), removes the _new from the name and renames the filename to the new name :
for file in `ls *new.xml`; do echo "Old Name $file"; new_name=`echo $file | sed -e 's/[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}/2010-03-23/g' | sed 's/_new//g'` ; echo "New Name $new_name"; mv $file $new_name; done
Is this the type of thing you wanted?
find /fullpath -type f -name "*.dev"|sed 's|\(.*\)\(\.pdf\)|mv & \1.sometag|' | sh
The unzip command doesn't have an option for recursively unzipping archives.
If I have the following directory structure and archives:
/Mother/Loving.zip
/Scurvy/Sea Dogs.zip
/Scurvy/Cures/Limes.zip
And I want to unzip all of the archives into directories with the same name as each archive:
/Mother/Loving/1.txt
/Mother/Loving.zip
/Scurvy/Sea Dogs/2.txt
/Scurvy/Sea Dogs.zip
/Scurvy/Cures/Limes/3.txt
/Scurvy/Cures/Limes.zip
What command or commands would I issue?
It's important that this doesn't choke on filenames that have spaces in them.
If you want to extract the files to the respective folder you can try this
find . -name "*.zip" | while read filename; do unzip -o -d "`dirname "$filename"`" "$filename"; done;
A multi-processed version for systems that can handle high I/O:
find . -name "*.zip" | xargs -P 5 -I fileName sh -c 'unzip -o -d "$(dirname "fileName")/$(basename -s .zip "fileName")" "fileName"'
A solution that correctly handles all file names (including newlines) and extracts into a directory that is at the same location as the file, just with the extension removed:
find . -iname '*.zip' -exec sh -c 'unzip -o -d "${0%.*}" "$0"' '{}' ';'
Note that you can easily make it handle more file types (such as .jar) by adding them using -o, e.g.:
find . '(' -iname '*.zip' -o -iname '*.jar' ')' -exec ...
Here's one solution that extracts all zip files to the working directory and involves the find command and a while loop:
find . -name "*.zip" | while read filename; do unzip -o -d "`basename -s .zip "$filename"`" "$filename"; done;
You could use find along with the -exec flag in a single command line to do the job
find . -name "*.zip" -exec unzip {} \;
This works perfectly as we want:
Unzip files:
find . -name "*.zip" | xargs -P 5 -I FILENAME sh -c 'unzip -o -d "$(dirname "FILENAME")" "FILENAME"'
Above command does not create duplicate directories.
Remove all zip files:
find . -depth -name '*.zip' -exec rm {} \;
Something like gunzip using the -r flag?....
Travel the directory structure recursively. If any of the file names specified on the command line are directories, gzip will descend into the directory and compress all the files it finds there (or decompress them in the case of gunzip ).
http://www.computerhope.com/unix/gzip.htm
If you're using cygwin, the syntax is slightly different for the basename command.
find . -name "*.zip" | while read filename; do unzip -o -d "`basename "$filename" .zip`" "$filename"; done;
I realise this is very old, but it was among the first hits on Google when I was looking for a solution to something similar, so I'll post what I did here. My scenario is slightly different as I basically just wanted to fully explode a jar, along with all jars contained within it, so I wrote the following bash functions:
function explode {
local target="$1"
echo "Exploding $target."
if [ -f "$target" ] ; then
explodeFile "$target"
elif [ -d "$target" ] ; then
while [ "$(find "$target" -type f -regextype posix-egrep -iregex ".*\.(zip|jar|ear|war|sar)")" != "" ] ; do
find "$target" -type f -regextype posix-egrep -iregex ".*\.(zip|jar|ear|war|sar)" -exec bash -c 'source "<file-where-this-function-is-stored>" ; explode "{}"' \;
done
else
echo "Could not find $target."
fi
}
function explodeFile {
local target="$1"
echo "Exploding file $target."
mv "$target" "$target.tmp"
unzip -q "$target.tmp" -d "$target"
rm "$target.tmp"
}
Note the <file-where-this-function-is-stored> which is needed if you're storing this in a file that is not read for a non-interactive shell as I happened to be. If you're storing the functions in a file loaded on non-interactive shells (e.g., .bashrc I believe) you can drop the whole source statement. Hopefully this will help someone.
A little warning - explodeFile also deletes the ziped file, you can of course change that by commenting out the last line.
Another interesting solution would be:
DESTINY=[Give the output that you intend]
# Don't forget to change from .ZIP to .zip.
# In my case the files were in .ZIP.
# The echo were for debug purpose.
find . -name "*.ZIP" | while read filename; do
ADDRESS=$filename
#echo "Address: $ADDRESS"
BASENAME=`basename $filename .ZIP`
#echo "Basename: $BASENAME"
unzip -d "$DESTINY$BASENAME" "$ADDRESS";
done;
You can also loop through each zip file creating each folder and unzip the zip file.
for zipfile in *.zip; do
mkdir "${zipfile%.*}"
unzip "$zipfile" -d "${zipfile%.*}"
done
this works for me
def unzip(zip_file, path_to_extract):
"""
Decompress zip archives recursively
Args:
zip_file: name of zip archive
path_to_extract: folder where the files will be extracted
"""
try:
if is_zipfile(zip_file):
parent_file = ZipFile(zip_file)
parent_file.extractall(path_to_extract)
for file_inside in parent_file.namelist():
if is_zipfile(os.path.join(os.getcwd(),file_inside)):
unzip(file_inside,path_to_extract)
os.remove(f"{zip_file}")
except Exception as e:
print(e)