How can I rsync mirror only *.php files? This gives me a bunch of empty dirs too and I don't want those.
rsync -v -aze 'ssh ' \
--numeric-ids \
--delete \
--include '*/' \
--exclude '*' \
--include '*.php' \
user#site.com:/home/www/domain.com \
/Volumes/Servers/
The culprit here is the
--include '*/'
When including a wildcard followed by the trailing forward-slash you're telling rsync to transfer all files ending with a '/' (that is, all directories).
Thus,
rsync -v -aze 'ssh ' \
--numeric-ids \
--delete \
--exclude '*' \
--include '*.php' \
user#site.com:/home/www/domain.com \
/Volumes/Servers/
If you were using that because you intend to recursively find all .php files, you’d have to use the ** wildcard.
That is,
--include '**/*.php'
Another way ( http://www.commandlinefu.com/commands/view/1481/rsync-find ) is pre-finding the target files and then using rsync,
find source -name "*.php" -print0 | rsync -av --files-from=- --from0 ./ ./destination/
Related
I want to exclude one special hidden file in just one special folder.
The command I used is:
rsync -a --delete \
--exclude='/absolute/path/to/webpage/folder1' \
--exclude='/absolute/path/to/webpage/backups' \
--exclude='/absolute/path/to/webpage/.htaccess' \
/absolute/path/to/webpage/ \
/absolute/path/to/copy_of_webpage &>/dev/null
rsync always overwrites my .htaccess.
Also I want to keep my .htpasswd and I thought about using wildcards like:
rsync -a --delete \
--exclude='/absolute/path/to/webpage/folder1' \
--exclude='/absolute/path/to/webpage/backups' \
--exclude='/absolute/path/to/webpage/.ht*' \
/absolute/path/to/webpage/ \
/absolute/path/to/copy_of_webpage &>/dev/null
But that doesn't work either.
You could exclude all .htaccess with --exclude '.htaccess'
Exclude the path relative to the source folder, not the absolute path.
If your root (as above) is:
/absolute/path/to/webpage/
and you wish to exclude:
/absolute/path/to/webpage/.htaccess
/absolute/path/to/webpage/backups
then you'll need to say:
--exclude='/.htaccess' --exclude='/backups'
Per the docs:
"/foo" would match a file called "foo" at... the "root of the transfer"
I'm trying to rsync my project folders to a server, and that includes various source code etc.
rsync "$GOPATH/src/" -rm \
-f '+ */' \
-f '+ *.go' \
-f '+ *.s' \
-f '+ *.html' \
-f '+ *.js' \
-f '+ *.css' \
-f '+ fonts/*' \
-f 'P */' \
-f '- *' \
myserver:its/gopath/src/
I want to remove superfluous code, and avoid creating directories on the receiver where possible. However by providing the -m, --prune-empty-directories option, I get warnings that the directories are not empty on the receiver.
*deleting github.com/anacrolix/jitter/.git/hooks/
cannot delete non-empty directory: github.com/anacrolix/jitter/.git
cannot delete non-empty directory: bitbucket.org/anacrolix/go-utils/bin
*deleting bitbucket.org/anacrolix/go-utils/bin/
How can I avoid attempting to delete those directories, or squash that warning via an rsync flag?
The solution is to combine -m, and -f 'P */' before -f '+ */'. This protects directories from being deleted on the receiver-side, and doesn't create unnecessary ones to house the sent files.
Would I be able to use rsync as such:
rsync -e ssh root#remote.com:/path/to/file:/path/to/second/file/ /local/directory/
or would i have to do something else?
Directly from the rsync man page:
The syntax for requesting multiple files from a remote host is done
by specifying additional remote-host args in the same style as the
first, or with the hostname omitted. For instance, all these work:
rsync -av host:file1 :file2 host:file{3,4} /dest/
rsync -av host::modname/file{1,2} host::modname/file3 /dest/
rsync -av host::modname/file1 ::modname/file{3,4}
This means your example should have a space added before the second path:
rsync -e ssh root#remote.com:/path/to/file :/path/to/second/file/ /local/directory/
I'd suggest you first try it with the -n or --dry-run option, so you see what will be done, before the copy (and possible deletions) are actually performed.
just an actual example of #tonin. Download specific directories from live server
rsync -av root#123.124.137.147:/var/www/html/cls \
:/var/www/html/index.php \
:/var/www/html/header.inc \
:/var/www/html/version.inc.php \
:/var/www/html/style.css \
:/var/www/html/accounts \
:/var/www/html/admin \
:/var/www/html/api \
:/var/www/html/config \
:/var/www/html/main \
:/var/www/html/reports .
I am using following script to rsync back files. If I amy execute those one by one on shell it work. But when I use to execute theem in script it is giving error
"rsync: link_stat "/home/tan/testnfs#015" failed: No such file or directory (2)"
015 is no where in script, I have edited the script and verify that no blank space or character left. But have same problem.
#!/bin/bash
#========================================
#Environment varibale settings
#========================================
username=test
codedir=/home/tan/testnfs
nfs=10.100.200.4::test
adminemail=backup#tan.com
errorlog=/home/tan/backuperror_log.txt
dat=$(date)
rm -fr $errorlog
echo $dat 2>&1>> $errorlog
echo $nfsserver
echo ========== Before rsync =================
rsync --stats -vr --exclude "*.png" --exclude "*.jpg" --exclude "*.jpeg" --exclude "*.zip" --exclude "*.pdf" --exclude "*.doc" --exclude "*.csv" --exclude "*.swf" $codedir $nfs
if [ $? = 0 ] then
mail -s "$username sync--complete" $adminemail < $errorlog
else
mail -s "$username sync--Incomplete" $adminemail < $errorlog
fi
I had figure that out. I was editing script on windows and it was adding its line terminator. I have saved it as linux file with notepad++ and it worked
I have a makefile:
#Nice, wonderful makefile written by Jason
CC=g++
CFLAGS=-c -Wall
BASE_DIR:=.
SOURCE_DIR:=$(BASE_DIR)/source
BUILD_DIR:=$(BASE_DIR)/build
TEST_DIR:=$(BASE_DIR)/build/tests
MAKEFILE_DIR:=$(BASE_DIR)/makefiles
DATA_DIR:=$(BASE_DIR)/data
DATA_DIR_TESTS:=$(DATA_DIR)/tests
MOLECULE_UT_SOURCES := $(SOURCE_DIR)/molecule_test/main.cc \
$(SOURCE_DIR)/molecule_manager.h \
$(SOURCE_DIR)/molecule_manager.cpp \
$(SOURCE_DIR)/molecule_manager_main.h \
$(SOURCE_DIR)/molecule_manager_main.cpp \
$(SOURCE_DIR)/molecule_reader.h \
$(SOURCE_DIR)/molecule_reader.cpp \
$(SOURCE_DIR)/molecule_reader_psf_pdb.h \
$(SOURCE_DIR)/molecule_reader_psf_pdb.cpp \
$(SOURCE_DIR)/parameter_manager_lj_molecule.h \
$(SOURCE_DIR)/parameter_manager_lj_molecule.cpp \
$(SOURCE_DIR)/parameter_manager.h \
$(SOURCE_DIR)/parameter_manager.cpp \
$(SOURCE_DIR)/parser.h \
$(SOURCE_DIR)/parser.cpp \
$(SOURCE_DIR)/common.h
MOLECULE_UT_DATA := \
$(DATA_DIR_TESTS)/molecule_test/par_oxalate_and_friends.inp \
$(DATA_DIR_TESTS)/molecule_test/dicarboxy-octane_4.pdb \
$(DATA_DIR_TESTS)/molecule_test/dicarboxy-octane_4.psf
PARAM_UT_SOURCES := $(SOURCE_DIR)/parameter_test/main.cc \
$(SOURCE_DIR)/parameter_manager_lj_molecule.h \
$(SOURCE_DIR)/parameter_manager_lj_molecule.cpp \
$(SOURCE_DIR)/parameter_manager.h \
$(SOURCE_DIR)/parameter_manager.cpp \
$(SOURCE_DIR)/parser.h \
$(SOURCE_DIR)/parser.cpp \
$(SOURCE_DIR)/common.h
PARAM_UT_DATA := $(DATA_DIR_TESTS)/molecule_test/par_oxalate_and_friends.inp
molecule_test : molecule_test_prepare_sources molecule_test_prepare_makefiles \
molecule_test_prepare_data_files
#$(shell cd $(TEST_DIR)/molecule_unit_test/; \
make ./bin/molecule_test)
molecule_test_prepare_sources: molecule_test_dir
#echo Copying sources...
#cp --preserve $(MOLECULE_UT_SOURCES) \
$(TEST_DIR)/molecule_unit_test/source
molecule_test_prepare_makefiles: $(MAKEFILE_DIR)/Makefile.molecule_test
#cp --preserve $(MAKEFILE_DIR)/Makefile.molecule_test \
$(TEST_DIR)/molecule_unit_test/Makefile
molecule_test_prepare_data_files:
cp --preserve $(MOLECULE_UT_DATA) $(TEST_DIR)/molecule_unit_test/bin/
molecule_test_dir:
#if test -d $(BUILD_DIR); then \
echo Build exists...; \
else \
echo Build directory does not exist, making build dir...; \
mkdir $(BUILD_DIR); \
fi
#if test -d $(TEST_DIR); then \
echo Tests exists...; \
else \
echo Tests directory does not exist, making tests dir...; \
mkdir $(TEST_DIR); \
fi
#if test -d $(TEST_DIR)/molecule_unit_test; then \
echo Molecule unit test directory exists...; \
else \
echo Molecule unit test directory does \
not exist, making build dir...; \
mkdir $(TEST_DIR)/molecule_unit_test; \
fi
#if test -d $(TEST_DIR)/molecule_unit_test/source; then \
echo Molecule unit test source directory exists...; \
else \
echo Molecule unit test source directory does \
not exist, making build dir...; \
mkdir $(TEST_DIR)/molecule_unit_test/source; \
fi
#if test -d $(TEST_DIR)/molecule_unit_test/obj; then \
echo Molecule unit test object directory exists...; \
else \
echo Molecule unit test object directory does \
not exist, making object dir...; \
mkdir $(TEST_DIR)/molecule_unit_test/obj; \
fi
#if test -d $(TEST_DIR)/molecule_unit_test/bin; then \
echo Molecule unit test executable directory exists...; \
else \
echo Molecule unit test executable directory does \
not exist, making executable dir...; \
mkdir $(TEST_DIR)/molecule_unit_test/bin; \
fi
param_test : param_test_prepare_sources param_test_prepare_makefiles \
param_test_prepare_data_files
#$(shell cd $(TEST_DIR)/param_unit_test/; \
make ./bin/param_test)
param_test_prepare_sources: param_test_dir
#echo Copying sources...
#cp --preserve $(PARAM_UT_SOURCES) $(TEST_DIR)/param_unit_test/source
param_test_prepare_makefiles: $(MAKEFILE_DIR)/Makefile.param_test
#cp --preserve $(MAKEFILE_DIR)/Makefile.param_test \
$(TEST_DIR)/param_unit_test/Makefile
param_test_prepare_data_files:
cp --preserve $(PARAM_UT_DATA) $(TEST_DIR)/param_unit_test/bin/
param_test_dir:
#if test -d $(BUILD_DIR); then \
echo Build exists...; \
else \
echo Build directory does not exist, making build dir...; \
mkdir $(BUILD_DIR); \
fi
#if test -d $(TEST_DIR); then \
echo Tests exists...; \
else \
echo Tests directory does not exist, making tests dir...; \
mkdir $(TEST_DIR); \
fi
#if test -d $(TEST_DIR)/param_unit_test; then \
echo Param unit test directory exists...; \
else \
echo Param unit test directory does \
not exist, making build dir...; \
mkdir $(TEST_DIR)/param_unit_test; \
fi
#if test -d $(TEST_DIR)/param_unit_test/source; then \
echo Param unit test source directory exists...; \
else \
echo Param unit test source directory does \
not exist, making build dir...; \
mkdir $(TEST_DIR)/param_unit_test/source; \
fi
#if test -d $(TEST_DIR)/param_unit_test/obj; then \
echo Param unit test object directory exists...; \
else \
echo Param unit test object directory does \
not exist, making object dir...; \
mkdir $(TEST_DIR)/param_unit_test/obj; \
fi
#if test -d $(TEST_DIR)/param_unit_test/bin; then \
echo Param unit test executable directory exists...; \
else \
echo Param unit test executable directory does \
not exist, making executable dir...; \
mkdir $(TEST_DIR)/param_unit_test/bin; \
fi
That calls a second makefile after it creates and populates the directory structure.
The second makefile is as follows:
#Nice, wonderful makefile written by Jason
CC=g++
CFLAGS=-c -Wall
SOURCE_DIR:=./source
OBJ_DIR:=./obj
EXE_DIR:=./bin
$(EXE_DIR)/molecule_test : $(OBJ_DIR)/main.o \
$(OBJ_DIR)/parameter_manager_lj_molecule.o \
$(OBJ_DIR)/parameter_manager.o $(OBJ_DIR)/parser.o \
$(OBJ_DIR)/molecule_manager.o $(OBJ_DIR)/molecule_manager_main.o \
$(OBJ_DIR)/molecule_reader.o \
$(OBJ_DIR)/molecule_reader_psf_pdb.o
#$(CC) $(OBJ_DIR)/main.o $(OBJ_DIR)/parameter_manager.o \
$(OBJ_DIR)/parser.o $(OBJ_DIR)/parameter_manager_lj_molecule.o \
$(OBJ_DIR)/molecule_manager.o $(OBJ_DIR)/molecule_manager_main.o \
$(OBJ_DIR)/molecule_reader.o \
$(OBJ_DIR)/molecule_reader_psf_pdb.o \
-o molecule_test
#mv molecule_test $(EXE_DIR)/
$(OBJ_DIR)/main.o: $(SOURCE_DIR)/parameter_manager.h \
$(SOURCE_DIR)/parameter_manager_lj_molecule.h \
$(SOURCE_DIR)/molecule_manager.h \
$(SOURCE_DIR)/molecule_manager_main.h \
$(SOURCE_DIR)/molecule_reader.h \
$(SOURCE_DIR)/molecule_reader_psf_pdb.h \
$(SOURCE_DIR)/common.h $(SOURCE_DIR)/main.cc
$(CC) $(CFLAGS) $(SOURCE_DIR)/main.cc
#mv main.o $(OBJ_DIR)/
$(OBJ_DIR)/molecule_reader.o: $(SOURCE_DIR)/parameter_manager.h \
$(SOURCE_DIR)/parameter_manager_lj_molecule.h \
$(SOURCE_DIR)/molecule_manager.h \
$(SOURCE_DIR)/molecule_manager_main.h \
$(SOURCE_DIR)/molecule_reader.h \
$(SOURCE_DIR)/common.h
$(CC) $(CFLAGS) $(SOURCE_DIR)/molecule_reader.cpp
#mv molecule_reader.o $(OBJ_DIR)/
$(OBJ_DIR)/molecule_reader_psf_pdb.o: $(SOURCE_DIR)/parameter_manager.h \
$(SOURCE_DIR)/parameter_manager_lj_molecule.h \
$(SOURCE_DIR)/molecule_manager.h \
$(SOURCE_DIR)/molecule_manager_main.h \
$(SOURCE_DIR)/molecule_reader.h \
$(SOURCE_DIR)/molecule_reader_psf_pdb.h \
$(SOURCE_DIR)/common.h
$(CC) $(CFLAGS) $(SOURCE_DIR)/molecule_reader_psf_pdb.cpp
#mv molecule_reader_psf_pdb.o $(OBJ_DIR)/
$(OBJ_DIR)/molecule_manager.o: $(SOURCE_DIR)/molecule_manager.h \
$(SOURCE_DIR)/common.h
$(CC) $(CFLAGS) $(SOURCE_DIR)/molecule_manager.cpp
#mv molecule_manager.o $(OBJ_DIR)/
$(OBJ_DIR)/molecule_manager_main.o: $(SOURCE_DIR)/molecule_manager.h \
$(SOURCE_DIR)/molecule_manager_main.h \
$(SOURCE_DIR)/common.h
$(CC) $(CFLAGS) $(SOURCE_DIR)/molecule_manager_main.cpp
#mv molecule_manager_main.o $(OBJ_DIR)/
$(OBJ_DIR)/parameter_manager_lj_molecule.o: $(SOURCE_DIR)/common.h \
$(SOURCE_DIR)/parameter_manager.h \
$(SOURCE_DIR)/parser.h
$(CC) $(CFLAGS) $(SOURCE_DIR)/parameter_manager_lj_molecule.cpp
#mv parameter_manager_lj_molecule.o $(OBJ_DIR)/
$(OBJ_DIR)/parameter_manager.o: $(SOURCE_DIR)/common.h
$(CC) $(CFLAGS) $(SOURCE_DIR)/parameter_manager.cpp
#mv parameter_manager.o $(OBJ_DIR)/
$(OBJ_DIR)/parser.o: $(SOURCE_DIR)/parser.h
#$(CC) $(CFLAGS) $(SOURCE_DIR)/parser.cpp
#mv parser.o $(OBJ_DIR)/
$(OBJ_DIR)/common.o: $(SOURCE_DIR)/common.h
$(CC) $(CFLAGS) $(SOURCE_DIR)/common.h
mv common.h.gch $(OBJ_DIR)/
I admit I'm somewhat of a novice at Makefiles. I would both like advice as to how to streamline these files (without too much "magic") and how to fix these two errors...
First I have to say everything is working, so to speak. When I build my target it creates all the directories right and generates the executable. And all my files get copied properly and get recompiled when I touch the files in my base-level source directory. So these aren't "real" errors so to speak, just annoying error text I want to get rid of...
The first error occurs when I run a build make molecule_test which requires it to do something. Whatever it needs to do gets done, but I also get:
g++: g++: No such file or directory
g++: g++: No such file or directory
g++: g++: No such file or directory
g++: g++: No such file or directory
g++: g++: No such file or directory
g++: g++: No such file or directory
make: *** [molecule_test] Error 1
..AGAIN the build succeeds, creating the executable properly
The second error I get occurs when there's nothing to be done...when that happens I get:
/bin/sh: -c: line 0: unexpected EOF while looking for matching ``'
/bin/sh: -c: line 1: syntax error: unexpected end of file
Please be gentle... I've read basic makefile tutorials, including the gnu makefile tutorial, but there seems to be a leap between creating a small program with a handful of local sources and a large program with the need for nested directories, data files, etc. I'm trying to make that leap... unfortunately I have no best practice makefiles from past code as I'm at a small research group at a university, not a corporate atmosphere.
My basic approach is to create a base directory with the following
[dir] source/
[dir] data/
[dir] makefiles/
[dir] build/ **gets created
Makefile
The top level makefile creates a subdirectory in the build directory, copies the needed sources (say for a particular test program, and needed data files, and a makefile to make all the sources. The top level makefile then calls the build-level makefile.
I'd be open to ideas on how to streamline this process, but would appreciate if we FIRST resolve the errors.
Thanks in advance!!!
P.S. I'm running on Centos 5.4, GNU Make 3.81, gcc version 4.1.2 20080704 (Red Hat 4.1.2-44) .... GNU Make and gcc are both 64-bit versions...
This fix seems to clear up both of your errors:
molecule_test : molecule_test_prepare_sources molecule_test_prepare_makefiles \
molecule_test_prepare_data_files
#cd $(TEST_DIR)/molecule_unit_test && $(MAKE) ./bin/molecule_test
The $(shell ...) command is for invoking a shell outside of a rule. There's no need to use it here, since this is a command in a rule-- it's already happening in a subshell. Also note that this uses $(MAKE) instead of make (the reasons are a little subtle, just think of it as a good habit).
You can do it even more concisely and quietly:
molecule_test : molecule_test_prepare_sources molecule_test_prepare_makefiles \
molecule_test_prepare_data_files
#$(MAKE) -s -C $(TEST_DIR)/molecule_unit_test ./bin/molecule_test
As for streamlining, there's a lot you can do. You can reduce the length of your second makefile by about half, and fix what appear to be a number of bugs, and with the first one you can do even better. It depends on how much "magic" you can tolerate. Here's a quick attempt at streamlining your second Makefile (since I don't have your files to test it with I can't promise it'll work without some touchups).
CC=g++
CFLAGS=-c -Wall
SOURCE_DIR:=./source
INCDIRS := -I$(SOURCE_DIR)
OBJ_DIR:=./obj
EXE_DIR:=./bin
VPATH = $(SOURCE_DIR)
$(EXE_DIR)/molecule_test : $(OBJ_DIR)/main.o \
$(OBJ_DIR)/parameter_manager_lj_molecule.o \
$(OBJ_DIR)/parameter_manager.o $(OBJ_DIR)/parser.o \
$(OBJ_DIR)/molecule_manager.o $(OBJ_DIR)/molecule_manager_main.o \
$(OBJ_DIR)/molecule_reader.o \
$(OBJ_DIR)/molecule_reader_psf_pdb.o
#$(CC) $^ -o $#
$(OBJ_DIR)/main.o $(OBJ_DIR)/molecule_reader.o \
$(OBJ_DIR)/molecule_reader_psf_pdb.o: \
molecule_manager.h \
molecule_manager_main.h \
parameter_manager.h \
parameter_manager_lj_molecule.h
$(OBJ_DIR)/main.o: main.cpp \
molecule_reader.h \
molecule_reader_psf_pdb.h common.h
$(CC) $(CFLAGS) $(INCDIRS) $< $#
$(OBJ_DIR)/molecule_reader_psf_pdb.o: molecule_reader.h
$(OBJ_DIR)/parameter_manager_lj_molecule.o: parser.h
%.o: %.cpp %.h common.h
$(CC) $(CFLAGS) $(INCDIRS) $< -o $#
For a start you can get rid of all the mv commands and use make's built-in variables, e.g.
$(OBJ_DIR)/parameter_manager.o: $(SOURCE_DIR)/parameter_manager.cpp $(SOURCE_DIR)/common.h
$(CC) $(CFLAGS) -o $# $<