I'm using pdftotext and find to find the contents of a PDF file and move it. I can find all of the files, but trying to add the mv command to the end with xargs returns "No such file or directory" for each file.
My code is as follows:
find ./ -name '*.pdf' -exec sh -c 'pdftotext "{}" - | grep -l -Z -i --with-filename --label="{}" --color "Thank you" | xargs -0 -I{} mv {} Found/' \;
Is xargs not getting the correct file path? I'm not sure what is happening.
you can get this, when the names contain spaces. Could you try this.
find ./ -name '*.pdf' -not -path "./Found/*" -exec sh -c 'pdftotext "{}" - | grep -l -Z -i --with-filename --label="{}" --color "Thank you"' \; | xargs -0 -I{} mv {} ./Found/
Related
I'd like to rename all files in several folders with filename containing '*file*' by '*doc*'. I've tried
find . -name "*file*" -exec mv {} `echo {} | sed "s/file/doc/"` \;
but got an error (see below).
~$ ls
my_file_1.txt my_file_2.txt my_file_3.txt
~$ find . -name "*file*"
./my_file_1.txt
./my_file_3.txt
./my_file_2.txt
~$ echo my_file_1.txt | sed "s/file/doc/"
my_doc_1.txt
~$ find . -name "*file*" -exec echo {} \;
./my_file_1.txt
./my_file_3.txt
./my_file_2.txt
~$ find . -name "*file*" -exec mv {} `echo {} | sed "s/file/doc/"` \;
mv: './my_file_1.txt' and './my_file_1.txt' are the same file
mv: './my_file_3.txt' and './my_file_3.txt' are the same file
mv: './my_file_2.txt' and './my_file_2.txt' are the same file
Many thanks for your help!
There are a thousand ways to do it, I'd do it with Perl, something like this will work:
find files -type f -name "file*" | perl -ne 'chomp; $f=$_; $f=~s/\/file/\/doc/; `mv $_ $f`;'
-ne process as inline script for each line input
chomp clean a newline
$f is new filename, same as old filename
s/\/file/\/doc/ replace "/file" with "/doc" in the new filename
mv $_ $f rename the file by running an OS command with back ticks
The problem with your solution is that the echo {} | sed "s/file/doc/" is executed before the rest of the find command. I tried to make a command demonstrating this:
find . -name "." -exec date \; -exec echo `date; sleep 5` \;
When the date commands aare executed from left to right, the dates would be equal. However the second date and the sleep are executed before find starts the first date.
Result:
Wed Aug 25 22:33:43 XXX 2021
Wed Aug 25 22:33:38 XXX 2021
The following solution is using print0 and xargs -0 for filenames with newlines. xargs will echo the mv command with two additional slashes.
The slashes will be found by the sed command, changing the target filename.
The result of sed is parsed by a new bash shell.
find . -name "*file1*" -print0 2>/dev/null |
xargs -0 -I {} echo mv '"{}"' //'"{}"' |
sed -r 's#//(.*)file(.*)#\1doc\2#' |
bash
See if you have rename command. If it is perl based:
# -n is for testing, remove it for actual renaming
find -name '*file*' -exec rename -n 's/file/doc/' {} +
If it is not perl based, see if this works:
# remove --no-act --verbose for actual renaming
find -name '*file*' -exec rename --no-act --verbose 'file' 'doc' {} +
I am currently running the below jpegoptim command via cron to have jpegoptim look for jpg files to compress.
find /home/public_html/public/uploads -name '*.jpg' -type f -print0 | xargs -0 jpegoptim -o -p --size=300k --strip-all
This works, but is there a way to include .jpeg, .JPG, and .JPEG without having to run the same command multiple times? Also, is it okay to run every hour, or would this keep compressing the same files eventually reducing the quality?
You can match all of *.jpg, *.JPG, *.jpeg, *.JPEG by using two case-insensitive name matches.
find /home/public_html/public/uploads \
'(' -iname '*.jpg' -o -iname '*.jpeg' ')' -type f -print0 \
| xargs -0 jpegoptim -o -p --size=300k --strip-all
You can keep a separate timestamp to avoid reprocessing the same files, for example
find /home/public_html/public/uploads \
-newer .STAMP '(' -iname '*.jpg' -o -iname '*.jpeg' ')' -type f -print0 \
| xargs -0 jpegoptim -o -p --size=300k --strip-all
touch .STAMP
I have directory: D:/Temp, where there are a lot of subfolders with text files. Each folder has "file.txt". In some file.txt files is a word - "pattern". I would like check how many pattern words there are, and also get the filepath to that file.txt:
find D:/Temp -type f -name "file.txt" -exec basename {} cat {} \; | sed -n '/pattern/p' | wc -l
Output should be:
4
D:/Temp/abc1/file.txt
D:/Temp/abc2/file.txt
D:/Temp/abc3/file.txt
D:/Temp/abc4/file.txt
Or similar.
You could use GNU grep :
grep -lr --include file.txt "pattern" "D:/Temp/"
This will return the file paths.
grep -cr --include file.txt "pattern" "D:/Temp/"
This will return the count (counting the pattern occurences rather than the number of files)
Explanation of the flags :
-r makes grep recursively browse its target, that can then be a directory
--include <glob> makes grep restrict its recursive browsing to files matching the <glob>.
-l makes grep only return the files path. Additionnaly, it will stop parsing a file as soon as it has encountered the pattern.
-c makes grep only return the number of matches
If your file names don't contain spaces then all you need is:
awk '/pattern/{print FILENAME; cnt++; nextfile} END{print cnt+0}' $(find D:/Temp -type f -name "file.txt")
The above used GNU awk for nextfile.
I'd propose you to use two commands : one for find all the files:
find ./ -name "file.txt" -exec fgrep -l "-pattern" {} \;
Another for counting them:
find ./ -name "file.txt" -exec fgrep -l "-pattern" {} \; | wc -l
Previously I've used:
grep -Hc "pattern" $(find D:/temp -type f -name "file.txt")
This will only work if file.txt is found. Otherwise you could use the following which will account for when both files are found or not found:
searchFiles=$(find D:/temp -type f -name "file.txt"); [[ ! -z "$searchFiles" ]] && grep -Hc "pattern" $searchFiles
The output for this would look more like:
D:/Temp/abc1/file.txt 2
D:/Temp/abc2/file.txt 1
D:/Temp/abc3/file.txt 1
D:/Temp/abc4/file.txt 1
I would use
find D:/Temp -type f -name "file.txt" -exec dirname {} \; > tmpfile
wc -l tmpfile
cat tmpfile
rm tmpfile
Give a try to this safe and standard version:
find D:/Temp -type f -name file.txt -printf "%p\0" | xargs -0 bash -c 'printf "%s" "${#}"; grep -c "pattern" "${#}"' | grep ":[1-9][0-9]*$"
For each file.txt file found in D:/Temp directory and sub-directories, the xargs command prints the filename and the number of lines which contain pattern (grep -c).
A final grep ":[1-9][0-9]*$" selects only filenames with a count greater than 0.
The way I'm reading your question, I'm going to answer as if:
some but not all file.txt files contain pattern,
you want a list of the paths leading to file.txt with pattern, and
you want a count of pattern in each of those files.
There are a few options. (Always multiple ways to do anything.)
If your bash is version 4 or higher, you can use globstar to recurse through directories:
shopt -s globstar
for file in **/file.txt; do
if count=$(grep -c 'pattern' "$file"); then
printf "%d %s\n" "$count" "${file%/*}"
fi
done
This works because the if evaluation considers a failed grep (i.e. zero occurrences) to be FALSE, and thus does not print results.
Note that this may be high impact because it launches a separate grep on each file that is found. A lighter weight alternative might be to run a single grep on the fileglob, and parse the results:
shopt -s globstar
grep -c 'pattern' **/file.txt | grep -v ':0$'
This also depends on bash 4, and of course if you have millions of files you may overwhelm bash's command line maximum length. The output of this will be obvious, but you'll need to parse it with care if your filenames contain colons. I.e. cut -d: -f2 may not cut it.
One more option that leverages grep instead of bash might be:
grep -r --include 'file.txt' -c 'pattern' ./ | grep -v ':0$'
This uses GNU grep's --include option which modified the behaviour of -r (recursive). It should work in Linux, FreeBSD, NetBSD, OSX, but not with the default grep on OpenBSD or most SVR4 (Solaris, HP/UX, etc).
Note that I have tested none of these. No liability assumed. May contain nuts.
This should do it:
find . -name "file.txt" -type f -printf '%p\n' | awk '{print} END { print NR }'
I'm trying to purge all thumbnails created by Wordpress because of a CMS switchover that I'm planning.
find -name \*-*x*.* | xargs rm -f
But I dont know bash or regex well enough to figure out how to add a bit more specifity such as only the following will be removed
All generated files have the syntax of
<img-name>-<width:integer>x<height:integer>.<file-ext> syntax
You didn't quote or escape all your wildcards, so the shell will try to expand them before find executes.
Quoting it should work
find -name '*-*x*.*'| xargs echo rm -f
Remove the echo when you're satisfied it works. You could also check that two of the fields are numbers by switching to -regex, but not sure if you need/want that here.
regex soultion
find -regex '^.*/[A-Za-z]+-[0-9]+x[0-9]+\.[A-Za-z]+$' | xargs echo rm -f
Note: I'm assuming img-name and file-ext can only contain letters
You can try this:
find -type f | grep -P '\w+-\d+x\d+\.\w+$' | xargs rm
If you have spaces in the path:
find -type f | grep -P '\w+-\d+x\d+\.\w+$' | sed -re 's/(\s)/\\\1/g' | xargs rm
Example:
find -type f | grep -P '\w+-\d+x\d+\.\w+$' | sed -re 's/(\s)/\\\1/g' | xargs ls -l
-rw-rw-r-- 1 tiago tiago 0 Jun 22 15:14 ./image-800x600.png
-rw-rw-r-- 1 tiago tiago 0 Jun 22 15:17 ./test 2/test 3/image-800x600.png
The below GNU find command will remove all the files which contain this <img-name>-<width:integer>x<height:integer>.<file-ext> syntax string. And also i assumed that the corresponding files has . in their file-names.
find . -name "*.*" -type f -exec grep -l '<img-name>-<width:integer>x<height:integer>.<file-ext> syntax' {} \; | xargs rm -f
Explanation:
. Directory in which find operation is going to takeplace.(. represnts your current directory)
-name "*.*" File must have dot in their file-names.
-type f Only files.
-exec grep -l '<img-name>-<width:integer>x<height:integer>.<file-ext> syntax' {} print the file names which contain the above mentioned pattern.
xargs rm -f For each founded files, the filename was fed into xargs and it got removed.
I need to find a text string within some files. This will give me a list:
find . -type f -print0 | xargs -0 grep -il "google"
then I need to copy those files to a folder and rename them. So I think I need to pipe them again to something like this
| xargs -0 -n1 -I '{}' cp '{}' ../testTarget/{}_RECOVERED
Alas:
find . -type f -print0 | xargs -0 grep -il "google" |
xargs -0 -n1 -I '{}' cp '{}' ../testTarget/{}_RECOVERED
Results: cp: {}: No such file or directory
Please advise
the filenames have space and commas in them (Dovecot)
find . -type f -print0 | xargs -0 grep -lh "google" | xargs -I % cp % ../testTarget/%_RECOVERED
or
find SOURCE/ -type f -print0 | xargs -0 grep -lh "SEARCH_STRING" | xargs -I % cp % TARGET_DIR/%_RECOVERED
for i in `ls -1tr`
{
SEARCH_STRING_LINE_NO=`grep -n SEARCH_STRING i | cut -d: -f1`;
if [ SEARCH_STRING_LINE_NO > 0 ] then
cp i folder_path_where_u_want_to_copy
fi
}
Replace SEARCH_STRING with the text string you want to find.
Replace folder_path_where_u_want_to_copy with the folder path where u want to move the files containing SEARCH_STRING
This should copy the files containing SEARCH_STRING to the folder folder_path_where_u_want_to_copy
I have not tested this code as I dont have a unix box with me right now, but atleast it should give u an idea to approach
The following command will grep for pattern and copy the files to your desired location.
find /path/to/search -type f -exec grep -q "pattern" '{}' ';' -exec cp '{}' /path/to/copy ';'
Once the files are copied over you can use rename command to rename them accordingly.