I tried to create folder in my local git repo using mkdir. It didn't work, but
mkdir -p works.
Why?
I'm using Mac OS by the way. I checked the definition of mkdir -p. But I still don't quite understand.
Say you're in the directory:
/home/Users/john
And you want to make 3 new sub directories to end up with:
/home/Users/john/long/dir/path
While staying in "/home/Users/john", this will fail:
mkdir long/dir/path
You would have to make three separate calls:
mkdir long
mkdir long/dir
mkdir long/dir/path
The reason is that mkdir by default only creates directories one level down. By adding the "-p" flag, mkdir will make the entire set in one pass. That is, while this won't work:
mkdir long/dir/path
this will work:
mkdir -p long/dir/path
and create all three directories.
From the help of mkdir:
-p, --parents no error if existing, make parent directories as needed
So you failed maybe just because you wanted to create both parent and child folder in one shoot without -p option.
That flag will create parent directories when necessary. You were probably trying to create something with subdirectories and failing due to missing the -p flag
Related
I'm getting this error when I am trying to reinstall elk with wazuh
We need more precise information about your use case in order to help you to troubleshoot this problem.
I recommend you follow the uninstallation guide from the official documentation https://documentation.wazuh.com/current/user-manual/uninstall/elastic-stack.html, and after that, install it again https://documentation.wazuh.com/current/installation-guide/more-installation-alternatives/elastic-stack/all-in-one-deployment/unattended-installation.html.
If you want to preserve your configuration make sure to backup the following files
cp -p /var/ossec/etc/ /var/ossec_backup/etc/client.keys
cp -p /var/ossec/etc/ /var/ossec_backup/etc/ossec.conf
cp -p /var/ossec/queue/rids/sender_counter /var/ossec_backup/queue/rids/sender_counter
If you have made local changes to any of the following then also backup them:
cp -p /var/ossec/etc/local_internal_options.conf /var/ossec_backup/etc/local_internal_options.conf
cp -p /var/ossec/etc/rules/local_rules.xml /var/ossec_backup/rules/local_rules.xml
cp -p /var/ossec/etc/decoders/local_decoder.xml /var/ossec_backup/etc/local_decoder.xml
If you have the centralized configuration you must preserve:
cp -p /var/ossec/etc/shared/default/agent.conf /var/ossec_backup/etc/shared/agent.conf
Optionally the following files can be restored to preserve alert log files and syscheck/rootcheck databases:
cp -rp /var/ossec/logs/archives /var/ossec_backup/logs/archives/*
cp -rp /var/ossec/logs/alerts /var/ossec_backup/logs/alerts/*
cp -rp /var/ossec/queue/rootcheck /var/ossec_backup/queue/rootcheck/*
cp -rp /var/ossec/queue/syscheck /var/ossec_backup/queue/syscheck/*
After reinstalling, you need to place those files in their original path.
Also, in case you want to preserve your indexes after restarting consider making a backup of your indexes following this blog https://wazuh.com/blog/index-backup-management/.
I'm not sure if this is possible. I've been fiddling with a MWE for awhile.
I'm using rsync in relative mode (-R) to copy a remote directory from a server onto my local machine. The remote directory may have symlinks. Sometimes the symlinks point within the scope of the relative directory, and sometimes they point outside of it.
When the links point inside the relative directory, I want to only copy the links (because the files are already synced, so those links should resolve).
When the links point outside the relative directory, I want to copy the files themselves because otherwise the links would likely not resolve.
I've setup a MWE to test various ways of doing this:
TEST_BASE=rsync_test
REMOTE_DPATH=$HOME/tmp/rsync-test/remote
LOCAL_DPATH=$HOME/tmp/rsync-test/local
REMOTE_URI=$REMOTE_DPATH
REMOTE_MOUNT=$REMOTE_DPATH
reset_rsync_test_remote()
{
# Clean
if [ -d "$REMOTE_DPATH" ]; then
rm -rf $REMOTE_DPATH
fi
mkdir -p $REMOTE_DPATH
# Setup remote data
mkdir -p $REMOTE_DPATH/$TEST_BASE/root/dir_L0_X0_A
mkdir -p $REMOTE_DPATH/$TEST_BASE/root/dir_L0_X0_A/dir_L1_X0_B
mkdir -p $REMOTE_DPATH/$TEST_BASE/root/dir_L0_X1_C
mkdir -p $REMOTE_DPATH/$TEST_BASE/root/inside_dir
mkdir -p $REMOTE_DPATH/$TEST_BASE/root/links
mkdir -p $REMOTE_DPATH/$TEST_BASE/outside_dir/
touch $REMOTE_DPATH/$TEST_BASE/root/file_L0_X0_a.txt
touch $REMOTE_DPATH/$TEST_BASE/root/dir_L0_X0_A/file_L1_X0_b.txt
touch $REMOTE_DPATH/$TEST_BASE/root/dir_L0_X1_C/file_L1_X0_c.txt
touch $REMOTE_DPATH/$TEST_BASE/root/inside_dir/inside_file.txt
touch $REMOTE_DPATH/$TEST_BASE/outside_dir/outside_file.txt
# Create links to inside and outside the sync root
ln -s $REMOTE_DPATH/$TEST_BASE/root/inside_dir/inside_file.txt $REMOTE_DPATH/$TEST_BASE/root/links/inside_flink.txt
ln -s $REMOTE_DPATH/$TEST_BASE/outside_dir/outside_file.txt $REMOTE_DPATH/$TEST_BASE/root/links/outside_flink.txt
ln -s $REMOTE_DPATH/$TEST_BASE/outside_dir $REMOTE_DPATH/$TEST_BASE/root/links/outside_dlink
ln -s $REMOTE_DPATH/$TEST_BASE/root/inside_dir $REMOTE_DPATH/$TEST_BASE/root/links/inside_dlink
ln -sr $REMOTE_DPATH/$TEST_BASE/root/inside_dir/inside_file.txt $REMOTE_DPATH/$TEST_BASE/root/links/rel_inside_flink.txt
ln -sr $REMOTE_DPATH/$TEST_BASE/outside_dir/outside_file.txt $REMOTE_DPATH/$TEST_BASE/root/links/rel_outside_flink.txt
ln -sr $REMOTE_DPATH/$TEST_BASE/outside_dir $REMOTE_DPATH/$TEST_BASE/root/links/rel_outside_dlink
ln -sr $REMOTE_DPATH/$TEST_BASE/root/inside_dir $REMOTE_DPATH/$TEST_BASE/root/links/rel_inside_dlink
tree $REMOTE_DPATH/
}
reset_rsync_test_local(){
# Setup home data
echo "LOCAL_DPATH = $LOCAL_DPATH"
if [ -d "$LOCAL_DPATH" ]; then
rm -rf $LOCAL_DPATH
fi
mkdir -p $LOCAL_DPATH
mkdir -p $LOCAL_DPATH/$TEST_BASE
# Make an existing link on the destination that we will sync to
mkdir -p $LOCAL_DPATH/link-dest1
mkdir -p $LOCAL_DPATH/link-dest2
ln -s $LOCAL_DPATH/link-dest1 $LOCAL_DPATH/rsync_test-link
ln -s $LOCAL_DPATH/link-dest2 $LOCAL_DPATH/rsync_test-link/root
tree $LOCAL_DPATH
}
reset_rsync_test_remote
This will setup my fake "remote" directory, which looks like this:
The relative directory that we are going to sync is "root".
Notice the links on the bottom. Some of them point inside "root" (in which case they are have inside_ in their name) and some point outside of "root", (in which case they have outside_ in their name). Then half of them are absolute links and the other half are relative links (rel_ prefix). This enumerates all 8 possibilities I'm concerned with here.
I'm going to rsync the "root" directory to an existing symlink within a symlink on my "local" machine to simulate my normal use case. This shouldn't matter too much, it just means we have to specify (-K) when running rsync.
So far I have two methods for doing roughly what I want to do but each has flaws. In the first method I use (-L), which simply resolves all links to their hard files.
# Method 1 with -KL
# The -K is important when syncing to a destination dir that is a symlink
# The -L will resolve any symlinks
# this grabs everything however, all files will be copied over as hard files
reset_rsync_test_local
rsync -avPRKL $REMOTE_URI/$TEST_BASE/./root $LOCAL_DPATH/rsync_test-link
ls -al $LOCAL_DPATH/
ls -al $LOCAL_DPATH/rsync_test-link/
tree $LOCAL_DPATH/link-dest2
Method 2 gets closer, in that it at least links the rel_inside_flink but doesn't link the rel_inside_dlink and it fails to resolve the rel_outside_dlink.
# Method 2: with -Kk --copy-unsafe-links
# Alternatively using -k --copy-unsafe-links will get almost everything
# links inside the relative directory are copied as links, links outside
# the relative dir are copied as files, except relative outside files for
# whatever reason.
reset_rsync_test_local
rsync -avPRKk --copy-unsafe-links $REMOTE_URI/$TEST_BASE/./root $LOCAL_DPATH/rsync_test-link
ls -al $LOCAL_DPATH/
ls -al $LOCAL_DPATH/rsync_test-link/
tree $LOCAL_DPATH/link-dest2
What I'd like is information on how to accomplish either of the following:
In the better than what I have case:
What I'd like is for any rel_inside link would be copied directly as a link, while all other links were resolved and copied as files.
In the absolutely ideal case: it would convert the absolute links that point inside the relative directory to either relative or absolute links at the destination (e.g. inside_dlink would either convert itself to ../inside_dir or /home/joncrall/tmp/rsync-test/local/rsync_test/root/inside_dir).
I am using mkdir to create directories under FreeBSD 10.2. I know -p option enables me to create a/b/c very easily (mkdir -p a/b/c). Now I want a to have two son directories b and c(a/c,a/b). Is it possible to do that by using only one mkdir command ? I have searched the net and found :
mkdir -p project/{lib/ext,bin,src,doc/{html,info,pdf},demo/stat/a}
which claims to generate the following result:
project/
project/lib/ext
project/bin
project/src
project/doc/html
project/doc/info
project/doc/pdf
project/demo/stat/a
However, this doesn't work in FreeBSD. Anybody can explain ? Thanks
It works fine here (FreeBSD 10.2-STABLE amd64) using the default tcsh shell;
> mkdir -p project/{lib/ext,bin,src,doc/{html,info,pdf},demo/stat/a}
> find .
.
./project
./project/lib
./project/lib/ext
./project/bin
./project/src
./project/doc
./project/doc/html
./project/doc/info
./project/doc/pdf
./project/demo
./project/demo/stat
./project/demo/stat/a
It does not work in the Bourne shell, sh.
I'm looking for an rsync-like program which will create any missing parent directories on the remote side.
For example, if I have /top/a/b/c/d on one server and only /top/a exists on the remote server, I want to copy d to the remote server and have the b and c directories created as well.
The command:
rsync /top/a/b/c/d remote:/top/a/b/c
won't work because /tmp/a/b doesn't exist on the remote server. And if it did exist then the file d would get copied to the path /top/a/b/c.
This is possible to do with rsync using --include and --exclude switches, but it is very involved, e.g.:
rsync -v -r a dest:dir \
--include 'a/b' \
--include 'a/b/c' \
--include 'a/b/c/d' \
--include 'a/b/c/d/e' \
--exclude 'a/*' \
--exclude 'a/b/*' \
--exclude 'a/b/c/*' \
--exclude 'a/b/c/d/*'
will only copy a/b/c/d/e to dest:dir/a/b/c/d/e even if the intermediate directories have files. (Note - the includes must precede the excludes.)
Are there any other options?
You may be looking for
rsync -aR
for example:
rsync -a --relative /top/a/b/c/d remote:/
See also this trick in other question.
rsync -aq --rsync-path='mkdir -p /tmp/imaginary/ && rsync' file user#remote:/tmp/imaginary/
From http://www.schwertly.com/2013/07/forcing-rsync-to-create-a-remote-path-using-rsync-path/, but don't copy and paste from there, his syntax is butchered.
it lets you execute arbitrary command to setup the path for rsync executables.
As of version 3.2.3 (6 Aug 2020), rynsc has a flag for this purpose.
From the rsync manual page (man rsync):
--mkpath create the destination's path component
i suggest that you enforce the existence manually:
ssh user#remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
this creates the target folder if it does not exists already.
According to https://unix.stackexchange.com/a/496181/5783, since rsync 2.6.7, --relative works if you use . to anchor the starting parent directory to create at the destination:
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top1/a/b/c/d
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top2/a
derek#DESKTOP-2F2F59O:~/projects/rsync$ rsync --recursive --relative --verbose top1/a/./b/c/d top2/a/
sending incremental file list
b/
b/c/
b/c/d/
sent 99 bytes received 28 bytes 254.00 bytes/sec
total size is 0 speedup is 0.00
--relative does not work for me since I had different setup.
Maybe I just didn't understood how --relative works, but I found that the
ssh remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
is easy to understand and does the job.
I was looking for a better solution, but mine seems to be better suited when you have too many sub-directories to create them manually.
Simply use cp as an intermediate step with the --parents option
cp --parents /your/path/sub/dir/ /tmp/localcopy
rsync [options] /tmp/localcopy/* remote:/destination/path/
cp --parents will create the structure for you.
You can call it from any subfolder if you want only one subset of the parent folders to be copied.
A shorter way in Linux to create rsync destination paths is to use the '$_' Special Variable. (I think, but cannot confirm, that it is also the same in OSX).
'$_' holds the value of the last argument of the previous command executed. So the question could be answered with:
ssh remote mkdir -p /top/a/b/c/ && rsync -avz /top/a/b/c/d remote:$_
I would like to rsync from local computer to server. On a directory that does not exist, and I want rsync to create that directory on the server first.
How can I do that?
If you have more than the last leaf directory to be created, you can either run a separate ssh ... mkdir -p first, or use the --rsync-path trick as explained here :
rsync -a --rsync-path="mkdir -p /tmp/x/y/z/ && rsync" $source user#remote:/tmp/x/y/z/
Or use the --relative option as suggested by Tony. In that case, you only specify the root of the destination, which must exist, and not the directory structure of the source, which will be created:
rsync -a --relative /new/x/y/z/ user#remote:/pre_existing/dir/
This way, you will end up with /pre_existing/dir/new/x/y/z/
And if you want to have "y/z/" created, but not inside "new/x/", you can add ./ where you want --relativeto begin:
rsync -a --relative /new/x/./y/z/ user#remote:/pre_existing/dir/
would create /pre_existing/dir/y/z/.
From the rsync manual page (man rsync):
--mkpath create the destination's path component
--mkpath was added in rsync 3.2.3 (6 Aug 2020).
Assuming you are using ssh to connect rsync, what about to send a ssh command before:
ssh user#server mkdir -p existingdir/newdir
if it already exists, nothing happens
The -R, --relative option will do this.
For example: if you want to backup /var/named/chroot and create the same directory structure on the remote server then -R will do just that.
this worked for me:
rsync /dev/null node:existing-dir/new-dir/
I do get this message :
skipping non-regular file "null"
but I don't have to worry about having an empty directory hanging around.
I don't think you can do it with one rsync command, but you can 'pre-create' the extra directory first like this:
rsync --recursive emptydir/ destination/newdir
where 'emptydir' is a local empty directory (which you might have to create as a temporary directory first).
It's a bit of a hack, but it works for me.
cheers
Chris
This answer uses bits of other answers, but hopefully it'll be a bit clearer as to the circumstances. You never specified what you were rsyncing - a single directory entry or multiple files.
So let's assume you are moving a source directory entry across, and not just moving the files contained in it.
Let's say you have a directory locally called data/myappdata/ and you have a load of subdirectories underneath this.
You have data/ on your target machine but no data/myappdata/ - this is easy enough:
rsync -rvv /path/to/data/myappdata/ user#host:/remote/path/to/data/myappdata
You can even use a different name for the remote directory:
rsync -rvv --recursive /path/to/data/myappdata user#host:/remote/path/to/data/newdirname
If you're just moving some files and not moving the directory entry that contains them then you would do:
rsync -rvv /path/to/data/myappdata/*.txt user#host:/remote/path/to/data/myappdata/
and it will create the myappdata directory for you on the remote machine to place your files in. Again, the data/ directory must exist on the remote machine.
Incidentally, my use of -rvv flag is to get doubly verbose output so it is clear about what it does, as well as the necessary recursive behaviour.
Just to show you what I get when using rsync (3.0.9 on Ubuntu 12.04)
$ rsync -rvv *.txt user#remote.machine:/tmp/newdir/
opening connection using: ssh -l user remote.machine rsync --server -vvre.iLsf . /tmp/newdir/
user#remote.machine's password:
sending incremental file list
created directory /tmp/newdir
delta-transmission enabled
bar.txt
foo.txt
total: matches=0 hash_hits=0 false_alarms=0 data=0
Hope this clears this up a little bit.
eg:
from: /xxx/a/b/c/d/e/1.html
to: user#remote:/pre_existing/dir/b/c/d/e/1.html
rsync:
cd /xxx/a/ && rsync -auvR b/c/d/e/ user#remote:/pre_existing/dir/
rsync source.pdf user1#192.168.56.100:~/not-created/target.pdf
If the target file is fully specified, the directory ~/not-created is not created.
rsync source.pdf user1#192.168.56.100:~/will-be-created/
But the target is specified with only a directory, the directory ~/will-be-created is created. / must be followed to let rsync know will-be-created is a directory.
use rsync twice~
1: tranfer a temp file, make sure remote relative directories has been created.
tempfile=/Users/temp/Dir0/Dir1/Dir2/temp.txt
# Dir0/Dir1/Dir2/ is directory that wanted.
rsync -aq /Users/temp/ rsync://remote
2: then you can specify the remote directory for transfer files/directory
tempfile|dir=/Users/XX/data|/Users/XX/data/
rsync -avc /Users/XX/data rsync://remote/Dir0/Dir1/Dir2
# Tips: [SRC] with/without '/' is different
This creates the dir tree /usr/local/bin in the destination and then syncs all containing files and folders recursively:
rsync --archive --include="/usr" --include="/usr/local" --include="/usr/local/bin" --include="/usr/local/bin/**" --exclude="*" user#remote:/ /home/user
Compared to mkdir -p, the dir tree even has the same perms as the source.
If you are using a version or rsync that doesn't have 'mkpath', then --files-from can help. Suppose you need to create 'mysubdir' in the target directory
Create 'filelist.txt' to contain
mysubdir/dummy
mkdir -p source_dir/mysubdir/
touch source_dir/mysubdir/dummy
rsync --files-from='filelist.txt' source_dir target_dir
rsync will copy mysubdir/dummy to target_dir, creating mysubdir in the process. Tested with rsync 3.1.3 on Raspberry Pi OS (debian).