Question: How do I recursively process, using Imagemagik (convert), a nested directory of photos?
I have the following directory structure:
/
..2008/
....a.jpg
....b.jpg
..2009/
.....c.jpg
And I want to run the following ImageMagik command on each file, to clean/resize up the images, and then save the resulting image out as the exact same filename as the original file. Basically, I want to replace the original file with the generated resized file created.
// from unix command line
convert FILENAME.jpg -resize 100x100 -sharpen 1.5 -strip -profile "*" -sampling-factor 4x1 -quality 80 FILENAME.jpg;
Try using find -exec. For instance:
find dirname -type f -iname "*.jpg" -exec convert \{\} -resize 100x100 -sharpen 1.5 -strip -profile "*" -sampling-factor 4x1 -quality 80 \{\} \;
By the way, I don't recommend in-place editing. It's generally a bad idea, especially with storage so cheap. Why not be safe?
Related
I am in a directory that has let's say 100 directories (and nothing else) and each of them has another 50 directories (and nothing else) and each of the directory(of the 50) has some hidden files. All the 50 dirs have the same name for the hidden file.
How can I grep something in the hidden file?
Example:
grep "Killed" .log
(the .log file is inside each of the 50 dirs; but I am in the root of the 100 dirs)
Using GNU grep:
grep -r --include=.log 'Killed'
This starts a recursive grep in your current directory including only files matching the name .log.
The question is a bit ambiguous. Do you have multiple "hidden" files, and you only want to search for a string in files with a particular name, or do you want to search for the string in all of the files? Either way, it's pretty trivial:
find /root/dir -type f -exec grep pattern {} \; # Search all files
find /root/dir -type f -name '*.log' -exec grep pattern {} \; # Search only in files with names matching '*.log'
You'll often want to add a -H (or specify /dev/null as a second argument) to the invocation of grep to see filenames.
I want to convert a group of files, but not overwrite the existing file. How can I use mogrify to specificy the final file format? For example, firstpic.png -> firstpic-thumbnail.png, secondpic.png -> secondpic-thumbnail.png, etc.
gm mogrify -size 150x150 *.png -resize 150x150 +profile "*" "%f-thumbnail.png"
Is there any way to do this?
I don't know if there's a way to specify output file format from mogrify but I would use convert with simple bash loop instead:
for f in *.jpg;
do
convert "$f" -resize 150x150 +profile "*" "${f%.jpg}-thumbnail.jpg";
done;
Or if you really want to use mogrify you can use -output-directory (more on this option here) to put new files into separate directory and then take care of renaming:
mkdir tmp;
gm mogrify -output-directory tmp -size 150x150 -resize 150x150 +profile "" "*.jpg";
for f in tmp/*.jpg;
do
mv "$f" "${f%.jpg}-thumbnail.jpg";
done;
mv tmp/* .;
rm -rf tmp;
I like the first method better ;)
If the output format (extension) is different from the input format, the files won't get overwritten. If they are the same, you can use this trick, that makes them appear to be "different" for this purpose but are really the same but differ in the case of the extension:
gm mogrify -resize 150x150 -format PNG +profile "*" *.png
EDIT:
I don't know of a facility within "mogrify" to rename the output files other than specifying a different directory or a different extension. So fragphace's answer is correct; you will need to use a script to rename them. In combination with my answer:
gm mogrify -resize 150x150 -format PNG +profile "*" *.png
for file in *.PNG
do
basename=`echo $file | sed -e "s/.PNG//"`
mv $basename.PNG $basename-thumbnail.png
done
Using cat command as follows we can display content of multiple files on screen
cat file1 file2 file3
But in a directory if there are more than 20 files and I want content of all those files to be displayed on the screen without using the cat command as above by mentioning the names of all files.
How can I do this?
You can use the * character to match all the files in your current directory.
cat * will display the content of all the files.
If you want to display only files with .txt extension, you can use cat *.txt, or if you want to display all the files whose filenames start with "file" like your example, you can use cat file*
If it's just one level of subdirectory, use cat * */*
Otherwise,
find . -type f -exec cat {} \;
which means run the find command, to search the current directory (.) for all ordinary files (-type f). For each file found, run the application (-exec) cat, with the current file name as a parameter (the {} is a placeholder for the filename). The escaped semicolon is required to terminate the -exec clause.
I also found it useful to print filename before printing content of each file:
find ./ -type f | xargs tail -n +1
It will go through all subdirectories as well.
Have you tried this command?
grep . *
It's not suitable for large files but works for /sys or /proc, if this is what you meant to see.
You could use awk too. Lets consider we need to print the content of a all text files in a directory some-directory
awk '{print}' some-directory/*.txt
If you want to do more then just one command called for every file, you will be more flexible with for loop. For example if you would like to print filename and it contents
for file in parent_dir/*.file_extension; do echo $file; cat $file; echo; done
What is the most efficient way to get a "ls"-like output of the most recently created files in a very large unix file system (100 thousand files +)?
Have tried ls -a and some other varients.
You can also use less to search and scroll it easily.
ls -la | less
If I'm understanding your question correctly try
ls -a | tail
More information here
If the files are in a single directory, then you can use:
ls -lt | less
the -t option to ls will sort the files by modification time and less will let you scroll through them
If the want recent files across an entire file system --- i.e., in different directories, then you can use the find command:
find dir -mtime 1 -print | xargs ls -ld
Substitute the directory where you want to start the search for "dir". The find command will print the names of all of the files that have been modified in the last day (-mtime 1 means modified in the last one day) and the xargs command will take that list of files and feed it to ls, giving you the ls-like output you want
I have a directory with a ton of files I want to find the average file size of these files so something like ls somethinghere whats the average file size of everything meets that?
I found something here:
http://vivekjain10.blogspot.com/2008/02/average-file-size-within-directory.html
To calculate the average file size within a directory on a Linux system, following command can be used:
ls -l | gawk '{sum += $5; n++;} END {print sum/n;}'
A short, general and recursion-friendly variation of Ernstsson's answer:
find ./ -ls | awk '{sum += $7; n++;} END {print sum/n;}'
Or, for example, if you want to impede files above 100 KB from stewing the average:
find ./ -size -100000c -ls | awk '{sum += $7; n++;} END {print sum/n;}'
Use wc -c * to get the size of all the files and ls | wc -l to get the number of files. Then just divide one by the other.
This works portably, even on AIX.
Outputs average number of bytes for plain files in the specified directory (${directory} in the example below):
find "${directory}" '!' -path "${directory}" -prune -type f -ls | awk '{s+=$7} END {printf "%.0f\n", s/NR}'
No need in counting the number of files yourself. NR is an awk builtin for number of rows.
The '!' -path ${directory} -prune part is a portable way to achieve the equivalent of GNU find -maxdepth 1 by pruning any path that is not the same as the one we start at, thereby ignoring any subdirectories.
Adjust with restrictions on what files to count. For instance, to average all files except *.sh in the current directory, you could add '!' -name '*.sh':
find . '!' -path . -prune -type f '!' -name '*.sh' -ls | awk '{s+=$7} END {printf "%.0f\n", s/NR}'
or to count only *.mp3 and include all subdirectories (remove '!' -path . -prune):
find . -type f -name '*.mp3' -ls | awk '{s+=$7} END {printf "%.0f\n", s/NR}'
du -sh . # gives the total space used by the directory
find . -type f | wc -l # count the number of files
devide the first by the second.
If you want a one liner, here it is:
echo $(( `du -sb | tr '.' ' '` / `find . -type f | wc -l` ))
They are finding the size of a directory and finding the amount of free disk space that exists on your machine. The command you would use to find the directory size is ' du '. And to find the free disk space you could use ' df '.
All the information present in this article is available in the man pages for du and df. In case you get bored reading the man pages and you want to get your work done quickly, then this article is for you.
-
'du' - Finding the size of a directory
$ du
Typing the above at the prompt gives you a list of directories that exist in the current directory along with their sizes. The last line of the output gives you the total size of the current directory including its subdirectories. The size given includes the sizes of the files and the directories that exist in the current directory as well as all of its subdirectories. Note that by default the sizes given are in kilobytes.
**$ du /home/david**
The above command would give you the directory size of the directory /home/david
**$ du -h**
This command gives you a better output than the default one. The option '-h' stands for human readable format. So the sizes of the files / directories are this time suffixed with a 'k' if its kilobytes and 'M' if its Megabytes and 'G' if its Gigabytes.
**$ du -ah**
This command would display in its output, not only the directories but also all the files that are present in the current directory. Note that 'du' always counts all files and directories while giving the final size in the last line. But the '-a' displays the filenames along with the directory names in the output. '-h' is once again human readable format.
**$ du -c**
This gives you a grand total as the last line of the output. So if your directory occupies 30MB the last 2 lines of the output would be
30M .
30M total
The first line would be the default last line of the 'du' output indicating the total size of the directory and another line displaying the same size, followed by the string 'total'. This is helpful in case you this command along with the grep command to only display the final total size of a directory as shown below.
**$ du -ch | grep total**
This would have only one line in its output that displays the total size of the current directory including all the subdirectories.
Note : In case you are not familiar with pipes (which makes the above command possible) refer to Article No. 24 . Also grep is one of the most important commands in Unix. Refer to Article No. 25 to know more about grep.
**$ du -s**
This displays a summary of the directory size. It is the simplest way to know the total size of the current directory.
**$ du -S**
This would display the size of the current directory excluding the size of the subdirectories that exist within that directory. So it basically shows you the total size of all the files that exist in the current directory.
**$ du --exculde=mp3**
The above command would display the size of the current directory along with all its subdirectories, but it would exclude all the files having the given pattern present in their filenames. Thus in the above case if there happens to be any mp3 files within the current directory or any of its subdirectories, their size would not be included while calculating the total directory size.
'df' - finding the disk free space / disk usage
$ df
Typing the above, outputs a table consisting of 6 columns. All the columns are very easy to understand. Remember that the 'Size', 'Used' and 'Avail' columns use kilobytes as the unit. The 'Use%' column shows the usage as a percentage which is also very useful.
**$ df -h**
Displays the same output as the previous command but the '-h' indicates human readable format. Hence instead of kilobytes as the unit the output would have 'M' for Megabytes and 'G' for Gigabytes.
Most of the users don't use the other parameters that can be passed to 'df'. So I shall not be discussing them.
I shall in turn show you an example that I use on my machine. I have actually stored this as a script named 'usage' since I use it often.
Example :
I have my Linux installed on /dev/hda1 and I have mounted my Windows partitions as well (by default every time Linux boots). So 'df' by default shows me the disk usage of my Linux as well as Windows partitions. And I am only interested in the disk usage of the Linux partitions. This is what I use :
**$ df -h | grep /dev/hda1 | cut -c 41-43**
This command displays the following on my machine
45%
Basically this command makes 'df' display the disk usages of all the partitions and then extracts the lines with /dev/hda1 since I am only interested in that. Then it cuts the characters from the 41st to the 43rd column since they are the columns that display the usage in % , which is what I want.
There are a few more options that can be used with 'du' and 'df' . You could find them in the man pages.
In addition to #cnst,
if you need to exlcude folders from the calculation, use
find ./ -size +4096c -ls | awk '{sum += $7; n++;} END {print sum/n;}'
Use du to estimate file space usage for a given directory.
du -sh /Your/Path # Average file size in human readable format
-s (--summarize) display only a total for each argument.
-h (--human-readable) print sizes in human readable format (e.g. 1K, 234M, 2G).
Note that not using -h would give the default block size (512-byte blocks).
If you wish to specify the block size you can use -k (Kilobytes), -m (Megabytes), or -g (Gigabytes).
du -sk /Your/Path # Average file size in Kilobytes.
Footnote: Using a file path would give the specified files's size.