For future reference, I want to compile a list of compression tools that are able to recurse into directories.
For example, gzip -r <directory> recurses into directories, and compresses every file it finds. On the other hand, zip -r foo.zip foo does not do what I mean - it recurses into the foo directory, and appends all zipped files to a single archive, foo.zip.
These tools don't have a built-in method for recursing into directories:
bzip2
lzma
lrzip
xz
zip
These tools can recurse into directories
gzip (grzip -r)
lrzip (version 0.630)
Are there any tools I missed?
The latest version of lrzip (0.630) does do recursion.
Anything can be made to do that quite easily:
find directory_to_compress -type f -exec compressor -o {}.compressed {} \;
Related
I am on Mac OS.
I have a directory with round about 3000 files and several subdirectories (wordpress installation)
Now I have to find all the files in a similar directory (have to separate master and child installation) that are additional files and have to copy them away into another directory.
I use this command:
$ diff -rq dt-the7 dt-the7-master-from-Yana|grep 'Only in dt-the7'|awk {' print $3 $4 '}|sed 's/:/\//g'|xargs -J {} rsync -av {} neu/
but somehow a certain file 3d.png and a list of other that should be in a subdir of the destination dir are copied into the root dir of the destination.
Any idea why that might be?
It makes no difference whether I use cp, rsync or ditto
You need the -R relative option on your rsync command.
Without this rsync just copies the item referenced rather than the path referenced, so items at the root level are copied as you expected but items in sub-directories are also copied to the root, which is not what you wanted.
With the option rsync takes account of the relative path and recreates it at the destination.
An example with another command might help, consider:
cp A/B.txt C/
that will copy B.txt into C, it does not create a folder A in C which in turn contains the file B.txt. rsync without -R behaves like that cp command, with -R it creates the A directory in C.
HTH
I want to copy a complete directory content from /home/private_html/userx/ into the /home/private_html/usery/, the problem is that the directory userx contains few symlinks, and when using the cp it just skip them (skip occurs, if symlinks directs into a file, in case if it points into the directory, it just copy WHOLE directory instead...).
The command I was using looks following:
# cp -iprv /home/private_html/userx/ /home/private_html/usery/
Has anyone a solution to copy the directory "just as it is" into other place?
On FreeBSD, cp doesn't have an -r option. It does have -R, which should do what you want:
-R If source_file designates a directory, cp copies the directory and
the entire subtree connected at that point. If the source_file
ends in a /, the contents of the directory are copied rather than
the directory itself. This option also causes symbolic links to be
copied, rather than indirected through, and for cp to create spe‐
cial files rather than copying them as normal files. Created
directories have the same mode as the corresponding source direc‐
tory, unmodified by the process' umask.
Roland is right about the -R flag. You could also use a pair of tar-processes, which would make your command a little bit more system-independent:
tar -C /home/private_html/userx/ -cpf - . | tar -C /home/private_html/usery/ -epf -
Figured maybe someone here might know whats going on, but essentially what I have to do is take a directory, and make a tar file omitting a subdir two levels down (root/1/2). Given it needs to work on a bunch of platforms, the easiest way I thought was to do a find and egrep that directory out, which works well giving me the list of files.
But then I pipe that file list into a xargs tar rvf command and the resulting file comes out something like 33gb. I've tried to output the find to a file, and use tar -T with that file as input, its still comes out to about 33gb, when if I did a straight tar of the whole directory (not omitting anything) it comes in where I'd expect it at 6-ish gb.
Any thoughts on what is going on? Or how to remedy this? I really need to get this figured out, I'm guessing it has to do with feeding it a list of files vs. having it just tar a directory, but not sure how to fix that.
Your find command will return directories as well as files
Consider using find to look for directories and to exclude some
tar cvf /path/to/archive.tar $(find suite -type d ! -name 'suite/tmp/Shared/*')
When you specify a directory in the file list, tar packages the directory and all the files in it. If you then list the files in the directory separately, it packages the files (again). If you list the sub-directories, it packages the contents of each subdirectory again. And so on.
If you're going to do a files list, make sure it truly is a list of files and that no directories are included.
find . -type f ...
The ellipsis might be find options to eliminate the files in the sub-directory, or it might be a grep -v that eliminates them. Note that -name normally only matches the last component of the name. GNU find has ! -path '*/subdir/*' or variants that will allow you to eliminate the file based on path, rather than just name:
find . -type f ! -path './root/1/2/*' -print
I have a directory images/ that I want to copy to build/images/ from within a Makefile. The directory might contain multiple levels of subdirectories. What would be the most elegant way to do that? I want:
avoid a full directory copy on each make run (i.e. no cp -r)
guaranteed consistency (i.e. if a file changed in images/ it should be automatically updated in build/images/)
avoid to specify a rule for each image and each subdirectory in the Makefile
solve the issue within make, so no rsync or cp -u if possible
I am using GNU make, so GNU specific stuff is allowed.
Well, I'd just use rsync. Any make script you will create with these constraints will just replicate its functionality, and most probably will be slower and may contain bugs. An example rule might look:
build/images:
rsync -rupE images build/
.PHONY: build/images
(.PHONY to trigger the rule every time).
Maybe symlinks or hardlinks can be used instead?
build/images:
ln -s ../images build/images
If you really want to avoid rsync and links, this piece re-implements them somehow (not tested, needs find, mkdir, and plain cp):
image_files:=$(shell find images -type f)
build/images/%: images/%
mkdir -p $(#D)
cp $< $#
build: $(patsubst %,build/%,$(image_files))
I've been stuck on a little unix command line problem.
I have a website folder (4gb) I need to grab a copy of, but just the .php, .html, .js and .css files (which is only a couple hundred kb).
I'm thinking ideally, there is a way to zip or tar a whole folder but only grabbing certain file extensions, while retaining subfolder structures. Is this possible and if so, how?
I did try doing a whole zip, then going through and excluding certain files but it seemed a bit excessive.
I'm kinda new to unix.
Any ideas would be greatly appreciated.
Switch into the website folder, then run
zip -R foo '*.php' '*.html' '*.js' '*.css'
You can also run this from outside the website folder:
zip -r foo website_folder -i '*.php' '*.html' '*.js' '*.css'
You can use find and grep to generate the file list, then pipe that into zip
e.g.
find . | egrep "\.(html|css|js|php)$" | zip -# test.zip
(-# tells zip to read a file list from stdin)
This is how I managed to do it, but I also like ghostdog74's version.
tar -czvf archive.tgz `find test/ | egrep ".*\.html|.*\.php"`
You can add extra extensions by adding them to the regex.
I liked Nick's answer, but, since this is a programming site, why not use Ant to do this. :)
Then you can put in a parameter so that different types of files can be zipped up.
http://ant.apache.org/manual/Tasks/zip.html
you may want to use find(GNU) to find all your php,html etc files.then tar them up
find /path -type f \( -iname "*.php" -o -iname "*.css" -o -iname "*.js" -o -iname "*.ext" \) -exec tar -r --file=test.tar "{}" +;
after that you can zip it up
You could write a shell script to copy files based on a pattern/expression into a new folder, zip the contents and then delete the folder. Now, as for the actual syntax of it, ill leave that to you :D.