Well,I m trying to list the number of files created today and the count the number of lines in those files.I have to do it in unix.Please suggest how to write script for this.
To find the number of lines:
find / -type f -ctime 0 -mtime 0 -print0 | xargs -0 wc -l
This is almost what you want. There is no file created time in Unix, this is approximation with both file status changed time and file modified time.
If you would like to search only in certain directory, replace / with /path/to/your/dir.
To find the number of files:
find / -type f -ctime 0 -mtime 0 | wc -l
This will find files (-type f) in /path modified in the last 24 hours (-mtime -1 means modified in the last 1 day) and run wc -l to count the number of lines. {} is a placeholder for the file names and + means pass all the file names to a single invocation of wc.
find /path -mtime -1 -type f -exec wc -l {} +
Note that -ctime as suggested in other answers is change time, not creation time. It is the last time a file's owner, group, link count, mode, etc., was changed. Unix does not track the creation time of a file.
find . -maxdepth 1 -daystart -ctime 0 -type f | xargs wc -l
You'll need to change the maxdepth argument value if you need to look deeper.
To count the number of files changed today:
find . -daystart -type f -ctime -1 | wc -l
find finds all the files (-type f) in the current directory (.) created* (-ctime) more recently (-) than one (1) day since the start of this day (-daystart). wc counts the number of lines (-l) in find's output.
To count the lines in those files:
find -daystart -type f -ctime -1 -print0 | wc -l --files0-from=-
The first part is the same, except that find separates the filenames using nulls (-print0). wc counts the lines (-l) in the null-separated files (--files0-from=) on its standard input (-).
* ctime is not actually the creation time, but the time when the file's status was last changed. I don't think the filesystem holds on to the actual creation time.
Determining when a file was created reliably is hard. The mtime is when it was modified, the ctime is when the inode data was changed (change of permissions, for example), the atime is when the file data was last accessed. Usually, the mtime is surrogate for the create time; when a file is created, it records the creation time (as does the ctime and atime), but if the file is subsequently modified, the mtime records the time when the contents of the file was last modified.
find . -mtime -1 -print0 | xargs -0 wc -l
Find all the files under the current directory with a modification time less than 24 hours old and send the names to 'wc -l' - allowing for spaces and other odd characters in the file names.
Related
I am struggling with the following issue. I have a Unix directory containing ~ 60K files and I would like to be able to count all the words in each file and output a list with each count along with its corresponding filename.
This is a job for find and wc:
find . -maxdepth 1 -type f -exec wc -w {} \;
This will find all files (-type f) in the current working directory (.), without recursing into subdirectories (-maxdepth 1) and for each result will execute (-exec [...] \;) wc -w passing that filename ({}) as an argument.
wc prints the number of newlines, words, and bytes in files by default, -w specifies it should just print the word-count.
Few more handy commands to get result from current folder
To get word in all files with file type java
find . -name '*.java' | xargs grep 'word'
To get word in all files
find . -type f | xargs grep 'word'
I have a directory with a ton of files I want to find the average file size of these files so something like ls somethinghere whats the average file size of everything meets that?
I found something here:
http://vivekjain10.blogspot.com/2008/02/average-file-size-within-directory.html
To calculate the average file size within a directory on a Linux system, following command can be used:
ls -l | gawk '{sum += $5; n++;} END {print sum/n;}'
A short, general and recursion-friendly variation of Ernstsson's answer:
find ./ -ls | awk '{sum += $7; n++;} END {print sum/n;}'
Or, for example, if you want to impede files above 100 KB from stewing the average:
find ./ -size -100000c -ls | awk '{sum += $7; n++;} END {print sum/n;}'
Use wc -c * to get the size of all the files and ls | wc -l to get the number of files. Then just divide one by the other.
This works portably, even on AIX.
Outputs average number of bytes for plain files in the specified directory (${directory} in the example below):
find "${directory}" '!' -path "${directory}" -prune -type f -ls | awk '{s+=$7} END {printf "%.0f\n", s/NR}'
No need in counting the number of files yourself. NR is an awk builtin for number of rows.
The '!' -path ${directory} -prune part is a portable way to achieve the equivalent of GNU find -maxdepth 1 by pruning any path that is not the same as the one we start at, thereby ignoring any subdirectories.
Adjust with restrictions on what files to count. For instance, to average all files except *.sh in the current directory, you could add '!' -name '*.sh':
find . '!' -path . -prune -type f '!' -name '*.sh' -ls | awk '{s+=$7} END {printf "%.0f\n", s/NR}'
or to count only *.mp3 and include all subdirectories (remove '!' -path . -prune):
find . -type f -name '*.mp3' -ls | awk '{s+=$7} END {printf "%.0f\n", s/NR}'
du -sh . # gives the total space used by the directory
find . -type f | wc -l # count the number of files
devide the first by the second.
If you want a one liner, here it is:
echo $(( `du -sb | tr '.' ' '` / `find . -type f | wc -l` ))
They are finding the size of a directory and finding the amount of free disk space that exists on your machine. The command you would use to find the directory size is ' du '. And to find the free disk space you could use ' df '.
All the information present in this article is available in the man pages for du and df. In case you get bored reading the man pages and you want to get your work done quickly, then this article is for you.
-
'du' - Finding the size of a directory
$ du
Typing the above at the prompt gives you a list of directories that exist in the current directory along with their sizes. The last line of the output gives you the total size of the current directory including its subdirectories. The size given includes the sizes of the files and the directories that exist in the current directory as well as all of its subdirectories. Note that by default the sizes given are in kilobytes.
**$ du /home/david**
The above command would give you the directory size of the directory /home/david
**$ du -h**
This command gives you a better output than the default one. The option '-h' stands for human readable format. So the sizes of the files / directories are this time suffixed with a 'k' if its kilobytes and 'M' if its Megabytes and 'G' if its Gigabytes.
**$ du -ah**
This command would display in its output, not only the directories but also all the files that are present in the current directory. Note that 'du' always counts all files and directories while giving the final size in the last line. But the '-a' displays the filenames along with the directory names in the output. '-h' is once again human readable format.
**$ du -c**
This gives you a grand total as the last line of the output. So if your directory occupies 30MB the last 2 lines of the output would be
30M .
30M total
The first line would be the default last line of the 'du' output indicating the total size of the directory and another line displaying the same size, followed by the string 'total'. This is helpful in case you this command along with the grep command to only display the final total size of a directory as shown below.
**$ du -ch | grep total**
This would have only one line in its output that displays the total size of the current directory including all the subdirectories.
Note : In case you are not familiar with pipes (which makes the above command possible) refer to Article No. 24 . Also grep is one of the most important commands in Unix. Refer to Article No. 25 to know more about grep.
**$ du -s**
This displays a summary of the directory size. It is the simplest way to know the total size of the current directory.
**$ du -S**
This would display the size of the current directory excluding the size of the subdirectories that exist within that directory. So it basically shows you the total size of all the files that exist in the current directory.
**$ du --exculde=mp3**
The above command would display the size of the current directory along with all its subdirectories, but it would exclude all the files having the given pattern present in their filenames. Thus in the above case if there happens to be any mp3 files within the current directory or any of its subdirectories, their size would not be included while calculating the total directory size.
'df' - finding the disk free space / disk usage
$ df
Typing the above, outputs a table consisting of 6 columns. All the columns are very easy to understand. Remember that the 'Size', 'Used' and 'Avail' columns use kilobytes as the unit. The 'Use%' column shows the usage as a percentage which is also very useful.
**$ df -h**
Displays the same output as the previous command but the '-h' indicates human readable format. Hence instead of kilobytes as the unit the output would have 'M' for Megabytes and 'G' for Gigabytes.
Most of the users don't use the other parameters that can be passed to 'df'. So I shall not be discussing them.
I shall in turn show you an example that I use on my machine. I have actually stored this as a script named 'usage' since I use it often.
Example :
I have my Linux installed on /dev/hda1 and I have mounted my Windows partitions as well (by default every time Linux boots). So 'df' by default shows me the disk usage of my Linux as well as Windows partitions. And I am only interested in the disk usage of the Linux partitions. This is what I use :
**$ df -h | grep /dev/hda1 | cut -c 41-43**
This command displays the following on my machine
45%
Basically this command makes 'df' display the disk usages of all the partitions and then extracts the lines with /dev/hda1 since I am only interested in that. Then it cuts the characters from the 41st to the 43rd column since they are the columns that display the usage in % , which is what I want.
There are a few more options that can be used with 'du' and 'df' . You could find them in the man pages.
In addition to #cnst,
if you need to exlcude folders from the calculation, use
find ./ -size +4096c -ls | awk '{sum += $7; n++;} END {print sum/n;}'
Use du to estimate file space usage for a given directory.
du -sh /Your/Path # Average file size in human readable format
-s (--summarize) display only a total for each argument.
-h (--human-readable) print sizes in human readable format (e.g. 1K, 234M, 2G).
Note that not using -h would give the default block size (512-byte blocks).
If you wish to specify the block size you can use -k (Kilobytes), -m (Megabytes), or -g (Gigabytes).
du -sk /Your/Path # Average file size in Kilobytes.
Footnote: Using a file path would give the specified files's size.
How to find the files that are created in the last hour in unix
If the dir to search is srch_dir then either
$ find srch_dir -cmin -60 # change time
or
$ find srch_dir -mmin -60 # modification time
or
$ find srch_dir -amin -60 # access time
shows files whose metadata has been changed (ctime), the file contents itself have been modified (mtime), or accessed (atime) in the last hour, respectively.
ctime is non-unintuitive and warrants further explanation:
ctime:
Unlike mtime, which is only related to the contents inside a file, changed timestamp indicates the last time some metadata of a file was changed. ctime refers to the last time when a file’s metadata.
For example, if permission settings of a file were modified, ctime will indicate it.
UNIX filesystems (generally) don't store creation times. Instead, there are only access time, (data) modification time, and (inode) change time.
That being said, find has -atime -mtime -ctime predicates:
$ man 1 find
...
-ctime n
The primary shall evaluate as true if the time of last change of
file status information subtracted from the initialization time,
divided by 86400 (with any remainder discarded), is n.
...
Thus find -ctime 0 finds everything for which the inode has changed (e.g. includes file creation, but also counts link count and permissions and filesize change) less than an hour ago.
check out this link and then help yourself out.
the basic code is
#create a temp. file
echo "hi " > t.tmp
# set the file time to 2 hours ago
touch -t 200405121120 t.tmp
# then check for files
find /admin//dump -type f -newer t.tmp -print -exec ls -lt {} \; | pg
find ./ -cTime -1 -type f
OR
find ./ -cmin -60 -type f
sudo find / -Bmin 60
From the man page:
-Bmin n
True if the difference between the time of a file's inode creation and
the time find was started, rounded up to the next full minute, is n
minutes.
Obviously, you may want to set up a bit differently, but this primary seems the best solution for searching for any file created in the last N minutes.
Check out this link for more details.
To find files which are created in last one hour in current directory, you can use -amin
find . -amin -60 -type f
This will find files which are created with in last 1 hour.
I have a directory filled with files with random names. I'd like to be able to rename them 'file 1' 'file 2' etc based on chronological order, ie file creation date. I could be writing a short Python script but then I wouldn't learn anything. I was wondering if there's a clever 1 line command that can solve this. If anyone could point me in the right direction.
I'm using zsh.
Thanks!
For zsh:
saveIFS="$IFS"; IFS=$'\0'; while read -A line; do mv "${line[2]}" "${line[1]%.*}.${line[2]}"; done < <(find -maxdepth 1 -type f -printf "%T+ %f\n"); IFS="$saveIFS"
For Bash (note the differences in the option to read and zero-based indexing instead of one-based):
saveIFS="$IFS"; IFS=$'\0'; while read -a line; do mv "${line[1]}" "${line[0]%.*}.${line[1]}"; done < <(find -maxdepth 1 -type f -printf "%T+\0%f\n"); IFS="$saveIFS"
These rename files by adding the modification date to the beginning of the original filename, which is retained to prevent name collisions.
A filename resulting from this might look like:
2009-12-15+11:08:52.original.txt
Because a null is used as the internal field separator (IFS), filenames with spaces should be preserved.
I'd rather not do this in PHP so I'm hoping a someone decent at shell scripting can help.
I need a script that runs through directory recursively and finds all files with last modified date is greater than some date. Then, it will tar and zip the file(s) keeping the path information.
as simple as:
find . -mtime -1 | xargs tar --no-recursion -czf myfile.tgz
where find . -mtime -1 will select all the files in (recursively) current directory modified day before. you can use fractions, for example:
find . -mtime -1.5 | xargs tar --no-recursion -czf myfile.tgz
If you have GNU find, then there are a legion of relevant options. The only snag is that the interface to them is less than stellar:
-mmin n (modification time in minutes)
-mtime n (modification time in days)
-newer file (modification time newer than modification time of file)
-daystart (adjust start time from current time to start of day)
Plus alternatives for access time and 'change' or 'create' time.
The hard part is determining the number of minutes since a time.
One option worth considering: use touch to create a file with the required modification time stamp; then use find with -newer.
touch -t 200901031231.43 /tmp/wotsit
find . -newer /tmp/wotsit -print
rm -f /tmp/wotsit
This looks for files newer than 2009-01-03T12:31:43. Clearly, in a script, /tmp/wotsit would be a name with the PID or other value to make it unique; and there'd be a trap to ensure it gets removed even if the user interrupts, and so on and so forth.
You can do this directly with tar and even better:
tar -N '2014-02-01 18:00:00' -jcvf archive.tar.bz2 files
This instructs tar to compress files newer than 1st of January 2014, 18:00:00.
This will work for some number of files. You want to include "-print0" and "xargs -0" in case any of the paths have spaces in them. This example looks for files modified in the last 7 days. To find those modified before the last 7 days, use "+7".
find . -mtime -7 -print0 | xargs -0 tar -cjf /foo/archive.tar.bz2
As this page warns, xargs can cause the tar command to be executed multiple times if there are a lot of arguments, and the "-c" flag could cause problems. In that case, you would want this:
find . -mtime -7 -print0 | xargs -0 tar -rf /foo/archive.tar
You can't update a zipped tar archive with tar, so you would have to bzip2 or gzip it in a second step.
This should show all files modified within the last 7 days.
find . -type f -mtime -7 -print
Pipe that into tar/zip, and you should be good.
I would simply do the following to backup all new files from 7 days ago
tar --newer $(date -d'7 days ago' +"%d-%b") -zcf thisweek.tgz .
note you can also replace '7 days ago' with anything that suits your need
Can be : date -d'yesterday' +"%d-%b"
Or even : date -d'first Sunday last month' +"%d-%b"
well under linux try reading man page of the find command
man find
something like this should
find . -type f -mtime -7 -print -exec cat {} \; | tar cf - | gzip -9
and you have it
You can get a list of files last modified later than x days ago with:
find . -mtime -x
Then you just have to tar and zip files in the resulting list, e.g.:
tar czvf mytarfile.tgz `find . -mtime -30`
for all files modified during last month.
This script will find files having a modification date of two minutes before and after the given date (and you can change the values in the conditions as per your requirement)
PATH_SRC="/home/celvas/Documents/Imp_Task/"
PATH_DST="/home/celvas/Downloads/zeeshan/"
cd $PATH_SRC
TODAY=$(date -d "$(date +%F)" +%s)
TODAY_TIME=$(date -d "$(date +%T)" +%s)
for f in `ls`;
do
# echo "File -> $f"
MOD_DATE=$(stat -c %y "$f")
MOD_DATE=${MOD_DATE% *}
# echo MOD_DATE: $MOD_DATE
MOD_DATE1=$(date -d "$MOD_DATE" +%s)
# echo MOD_DATE: $MOD_DATE
DIFF_IN_DATE=$[ $MOD_DATE1 - $TODAY ]
DIFF_IN_DATE1=$[ $MOD_DATE1 - $TODAY_TIME ]
#echo DIFF: $DIFF_IN_DATE
#echo DIFF1: $DIFF_IN_DATE1
if [[ ($DIFF_IN_DATE -ge -120) && ($DIFF_IN_DATE1 -le 120) && (DIFF_IN_DATE1 -ge -120) ]]
then
echo File lies in Next Hour = $f
echo MOD_DATE: $MOD_DATE
#mv $PATH_SRC/$f $PATH_DST/$f
fi
done
For example you want files having modification date before the given date only, you may change 120 to 0 in $DIFF_IN_DATE parameter discarding the conditions of $DIFF_IN_DATE1 parameter.
Similarly if you want files having modification date 1 hour before and after given date,
just replace 120 by 3600 in if CONDITION.