How to extract the ports, src, sys trees to /user/ports, /usr/src, usr/src/sys in OpenBSD. Command to do the download and the untar.
Check the PortsFetch section of the faq:
Once you have decided which flavor of the ports tree you want, you can get it from different sources. The table below gives an overview of where you can find the different flavors, and in which form. An 'o' marks availability and '-' means it is not available through that specific source.
Look for a file named ports.tar.gz on the mirrors.
$ cd /tmp
$ ftp https://ftp.openbsd.org/pub/OpenBSD/$(uname -r)/{ports.tar.gz,SHA256.sig}
$ signify -Cp /etc/signify/openbsd-$(uname -r | cut -c 1,3)-base.pub -x SHA256.sig ports.tar.gz
You want to untar this file in the /usr directory, which will create /usr/ports and all the directories under it.
# cd /usr
# tar xzf /tmp/ports.tar.gz
Related
I want to develop a Rsync script to recursively copy the list of files/folders that are created on one server to another server. I'm using the below command to copy a particular file. However whether we can include a list of files that should be ignored while copying the files from source to destination.
rsync -av --include /sourcepath/* --exclude filename user#destinationserver:/destinationpath
We can refer [1] [1]: https://www.tutorialspoint.com/unix_commands/rsync.html
rsync -lapv --include /sourcecpath/* --exclude-from exclude-list-one user#destinationserver:/destinationpath
-l, --links copy symlinks as symlinks
-a, --archive archive mode; same as -rlptgoD (no -H)
-v, --verbose increase verbosity
-p, --perms preserve permissions
--exclude-from=FILE read exclude patterns from FILE
e.g. from the Mac man pages:
man ln
-h If the NewLinkFile (or directory) is a symbolic link, do not follow
it. This is most useful with the -f option, to replace a symlink
which can point to a directory.
I am writing a script which once a day iterates over hundreds of files and symlinks them all and then periodically through out the day checks for new files and symlinks them. Would be great to use this -h check or something similar as at the moment it is done very awkwardly.
A GNU equivalent of the BSD ln -sh is ln -sn.
But GNU's ln -sT is safer, because it will skip both symbolic links and directories.
Compare:
cd "$(mktemp -d)"
mkdir bin
for d in bin usr etc; do ln -sn "/$d" "$d"; done
# will create a bin/bin -> /bin symlink
vs.
cd "$(mktemp -d)"
mkdir bin
for d in bin usr etc; do ln -sT "/$d" "$d"; done
ln: failed to create symbolic link 'bin': File exists
I'm looking for an rsync-like program which will create any missing parent directories on the remote side.
For example, if I have /top/a/b/c/d on one server and only /top/a exists on the remote server, I want to copy d to the remote server and have the b and c directories created as well.
The command:
rsync /top/a/b/c/d remote:/top/a/b/c
won't work because /tmp/a/b doesn't exist on the remote server. And if it did exist then the file d would get copied to the path /top/a/b/c.
This is possible to do with rsync using --include and --exclude switches, but it is very involved, e.g.:
rsync -v -r a dest:dir \
--include 'a/b' \
--include 'a/b/c' \
--include 'a/b/c/d' \
--include 'a/b/c/d/e' \
--exclude 'a/*' \
--exclude 'a/b/*' \
--exclude 'a/b/c/*' \
--exclude 'a/b/c/d/*'
will only copy a/b/c/d/e to dest:dir/a/b/c/d/e even if the intermediate directories have files. (Note - the includes must precede the excludes.)
Are there any other options?
You may be looking for
rsync -aR
for example:
rsync -a --relative /top/a/b/c/d remote:/
See also this trick in other question.
rsync -aq --rsync-path='mkdir -p /tmp/imaginary/ && rsync' file user#remote:/tmp/imaginary/
From http://www.schwertly.com/2013/07/forcing-rsync-to-create-a-remote-path-using-rsync-path/, but don't copy and paste from there, his syntax is butchered.
it lets you execute arbitrary command to setup the path for rsync executables.
As of version 3.2.3 (6 Aug 2020), rynsc has a flag for this purpose.
From the rsync manual page (man rsync):
--mkpath create the destination's path component
i suggest that you enforce the existence manually:
ssh user#remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
this creates the target folder if it does not exists already.
According to https://unix.stackexchange.com/a/496181/5783, since rsync 2.6.7, --relative works if you use . to anchor the starting parent directory to create at the destination:
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top1/a/b/c/d
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top2/a
derek#DESKTOP-2F2F59O:~/projects/rsync$ rsync --recursive --relative --verbose top1/a/./b/c/d top2/a/
sending incremental file list
b/
b/c/
b/c/d/
sent 99 bytes received 28 bytes 254.00 bytes/sec
total size is 0 speedup is 0.00
--relative does not work for me since I had different setup.
Maybe I just didn't understood how --relative works, but I found that the
ssh remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
is easy to understand and does the job.
I was looking for a better solution, but mine seems to be better suited when you have too many sub-directories to create them manually.
Simply use cp as an intermediate step with the --parents option
cp --parents /your/path/sub/dir/ /tmp/localcopy
rsync [options] /tmp/localcopy/* remote:/destination/path/
cp --parents will create the structure for you.
You can call it from any subfolder if you want only one subset of the parent folders to be copied.
A shorter way in Linux to create rsync destination paths is to use the '$_' Special Variable. (I think, but cannot confirm, that it is also the same in OSX).
'$_' holds the value of the last argument of the previous command executed. So the question could be answered with:
ssh remote mkdir -p /top/a/b/c/ && rsync -avz /top/a/b/c/d remote:$_
Hi everyone I'm not to ksh. What i'm trying to do is I'm writing a script to scp a(or many) zip file from a local directory to a remote host. Then get the script to ssh into the remote host to gunzip the files I just scp over. Is there any simple way to do this. I keep trying but once I ssh over to the remote host the rest of my commands no longer run like the cd /file/directory and then gzip -d /files etc.....
NB: don't confuse "zip" and "gzip", two different animals
This should work:
cd <local_directory>
# collect files names as $1 $2 ... $N
set -- *.gz # or use your own filter like "dumps*.gz"
# put source file a tar archive and send it as input to ssh
# then, on the other side, untar the file then decompress
tar cf - $* | ssh <user>#<remote_host> "cd <remote dir> && tar xf - && gunzip $*
Note: using "&&" instead of ";" to prevent "tar" command to be executed if "cd" fails for any reason
I would like to rsync from local computer to server. On a directory that does not exist, and I want rsync to create that directory on the server first.
How can I do that?
If you have more than the last leaf directory to be created, you can either run a separate ssh ... mkdir -p first, or use the --rsync-path trick as explained here :
rsync -a --rsync-path="mkdir -p /tmp/x/y/z/ && rsync" $source user#remote:/tmp/x/y/z/
Or use the --relative option as suggested by Tony. In that case, you only specify the root of the destination, which must exist, and not the directory structure of the source, which will be created:
rsync -a --relative /new/x/y/z/ user#remote:/pre_existing/dir/
This way, you will end up with /pre_existing/dir/new/x/y/z/
And if you want to have "y/z/" created, but not inside "new/x/", you can add ./ where you want --relativeto begin:
rsync -a --relative /new/x/./y/z/ user#remote:/pre_existing/dir/
would create /pre_existing/dir/y/z/.
From the rsync manual page (man rsync):
--mkpath create the destination's path component
--mkpath was added in rsync 3.2.3 (6 Aug 2020).
Assuming you are using ssh to connect rsync, what about to send a ssh command before:
ssh user#server mkdir -p existingdir/newdir
if it already exists, nothing happens
The -R, --relative option will do this.
For example: if you want to backup /var/named/chroot and create the same directory structure on the remote server then -R will do just that.
this worked for me:
rsync /dev/null node:existing-dir/new-dir/
I do get this message :
skipping non-regular file "null"
but I don't have to worry about having an empty directory hanging around.
I don't think you can do it with one rsync command, but you can 'pre-create' the extra directory first like this:
rsync --recursive emptydir/ destination/newdir
where 'emptydir' is a local empty directory (which you might have to create as a temporary directory first).
It's a bit of a hack, but it works for me.
cheers
Chris
This answer uses bits of other answers, but hopefully it'll be a bit clearer as to the circumstances. You never specified what you were rsyncing - a single directory entry or multiple files.
So let's assume you are moving a source directory entry across, and not just moving the files contained in it.
Let's say you have a directory locally called data/myappdata/ and you have a load of subdirectories underneath this.
You have data/ on your target machine but no data/myappdata/ - this is easy enough:
rsync -rvv /path/to/data/myappdata/ user#host:/remote/path/to/data/myappdata
You can even use a different name for the remote directory:
rsync -rvv --recursive /path/to/data/myappdata user#host:/remote/path/to/data/newdirname
If you're just moving some files and not moving the directory entry that contains them then you would do:
rsync -rvv /path/to/data/myappdata/*.txt user#host:/remote/path/to/data/myappdata/
and it will create the myappdata directory for you on the remote machine to place your files in. Again, the data/ directory must exist on the remote machine.
Incidentally, my use of -rvv flag is to get doubly verbose output so it is clear about what it does, as well as the necessary recursive behaviour.
Just to show you what I get when using rsync (3.0.9 on Ubuntu 12.04)
$ rsync -rvv *.txt user#remote.machine:/tmp/newdir/
opening connection using: ssh -l user remote.machine rsync --server -vvre.iLsf . /tmp/newdir/
user#remote.machine's password:
sending incremental file list
created directory /tmp/newdir
delta-transmission enabled
bar.txt
foo.txt
total: matches=0 hash_hits=0 false_alarms=0 data=0
Hope this clears this up a little bit.
eg:
from: /xxx/a/b/c/d/e/1.html
to: user#remote:/pre_existing/dir/b/c/d/e/1.html
rsync:
cd /xxx/a/ && rsync -auvR b/c/d/e/ user#remote:/pre_existing/dir/
rsync source.pdf user1#192.168.56.100:~/not-created/target.pdf
If the target file is fully specified, the directory ~/not-created is not created.
rsync source.pdf user1#192.168.56.100:~/will-be-created/
But the target is specified with only a directory, the directory ~/will-be-created is created. / must be followed to let rsync know will-be-created is a directory.
use rsync twice~
1: tranfer a temp file, make sure remote relative directories has been created.
tempfile=/Users/temp/Dir0/Dir1/Dir2/temp.txt
# Dir0/Dir1/Dir2/ is directory that wanted.
rsync -aq /Users/temp/ rsync://remote
2: then you can specify the remote directory for transfer files/directory
tempfile|dir=/Users/XX/data|/Users/XX/data/
rsync -avc /Users/XX/data rsync://remote/Dir0/Dir1/Dir2
# Tips: [SRC] with/without '/' is different
This creates the dir tree /usr/local/bin in the destination and then syncs all containing files and folders recursively:
rsync --archive --include="/usr" --include="/usr/local" --include="/usr/local/bin" --include="/usr/local/bin/**" --exclude="*" user#remote:/ /home/user
Compared to mkdir -p, the dir tree even has the same perms as the source.
If you are using a version or rsync that doesn't have 'mkpath', then --files-from can help. Suppose you need to create 'mysubdir' in the target directory
Create 'filelist.txt' to contain
mysubdir/dummy
mkdir -p source_dir/mysubdir/
touch source_dir/mysubdir/dummy
rsync --files-from='filelist.txt' source_dir target_dir
rsync will copy mysubdir/dummy to target_dir, creating mysubdir in the process. Tested with rsync 3.1.3 on Raspberry Pi OS (debian).