mkdir's "-p" option - unix

So this doesn't seem like a terribly complicated question I have, but it's one I can't find the answer to. I'm confused about what the -p option does in Unix. I used it for a lab assignment while creating a subdirectory and then another subdirectory within that one. It looked like this:
mkdir -p cmps012m/lab1
This is in a private directory with normal rights (rlidwka). Oh, and would someone mind giving a little explanation of what rlidwka means? I'm not a total noob to Unix, but I'm not really familiar with what this means. Hopefully that's not too vague of a question.

The man pages is the best source of information you can find... and is at your fingertips: man mkdir yields this about -p switch:
-p, --parents
no error if existing, make parent directories as needed
Use case example: Assume I want to create directories hello/goodbye but none exist:
$mkdir hello/goodbye
mkdir:cannot create directory 'hello/goodbye': No such file or directory
$mkdir -p hello/goodbye
$
-p created both, hello and goodbye
This means that the command will create all the directories necessaries to fulfill your request, not returning any error in case that directory exists.
About rlidwka, Google has a very good memory for acronyms :). My search returned this for example: http://www.cs.cmu.edu/~help/afs/afs_acls.html
Directory permissions
l (lookup)
Allows one to list the contents of a directory. It does not allow the reading of files.
i (insert)
Allows one to create new files in a directory or copy new files to a directory.
d (delete)
Allows one to remove files and sub-directories from a directory.
a (administer)
Allows one to change a directory's ACL. The owner of a directory can always change the ACL of a directory that s/he owns, along with the ACLs of any subdirectories in that directory.
File permissions
r (read)
Allows one to read the contents of file in the directory.
w (write)
Allows one to modify the contents of files in a directory and use chmod on them.
k (lock)
Allows programs to lock files in a directory.
Hence rlidwka means: All permissions on.
It's worth mentioning, as #KeithThompson pointed out in the comments, that not all Unix systems support ACL. So probably the rlidwka concept doesn't apply here.

-p|--parent will be used if you are trying to create a directory with top-down approach. That will create the parent directory then child and so on iff none exists.
-p, --parents
no error if existing, make parent directories as needed
About rlidwka it means giving full or administrative access. Found it here https://itservices.stanford.edu/service/afs/intro/permissions/unix.

mkdir [-switch] foldername
-p is a switch, which is optional. It will create a subfolder and a parent folder as well, even if parent folder doesn't exist.
From the man page:
-p, --parents no error if existing, make parent directories as needed
Example:
mkdir -p storage/framework/{sessions,views,cache}
This will create subfolder sessions,views,cache inside framework folder irrespective of whether 'framework' was available earlier or not.

PATH: Answered long ago, however, it maybe more helpful to think of -p as "Path" (easier to remember), as in this causes mkdir to create every part of the path that isn't already there.
mkdir -p /usr/bin/comm/diff/er/fence
if /usr/bin/comm already exists, it acts like:
mkdir /usr/bin/comm/diff
mkdir /usr/bin/comm/diff/er
mkdir /usr/bin/comm/diff/er/fence
As you can see, it saves you a bit of typing, and thinking, since you don't have to figure out what's already there and what isn't.

Note that -p is an argument to the mkdir command specifically, not the whole of Unix. Every command can have whatever arguments it needs.
In this case it means "parents", meaning mkdir will create a directory and any parents that don't already exist.

Related

Folderstructure with rsync in bash

I looked up the forum but didn't find an article which matches my problem. Maybe there is some, and you can help me out with it.
My problem is I want to sync an folder with the command rsync -a -v. The point is I got 5 different Maschinen. On every maschine is a scratch folder I want to sync into the folder: ~/work_dir/scratch_maschines and inside the /scratch_maschines folder should be a folder for maschine_a, maschine_b and so on.
On the maschines it is always the same path: /scratch/my_name. So when I use now this command for the first two maschines:
rsync -a -v --exclude='*.chk' --exclude='*.rwf' --exclude='*.fchk' --delete sp02:/scratch/my_name ~/work_dir/scratch_maschine01; rsync -a -v --exclude='*.chk' --exclude='*.rwf' --exclude='*.fchk' --delete maschine02:/scratch/my_name ~/work_dir/scratch_maschine02
I got a folders for scratch_maschine01 and scratch_maschine02 in my working directory but inside these folders are not direct my data there is first a folder inside with my_name and this folder contains the data. So my question is how can I use the rsync command and get the files from the scratch directorys straight to the folders for each machine?
You might want to consider reformulating your commands similar to the following:
START=`pwd`
EXCLUDES="--exclude='*.chk' --exclude='*.rwf' --exclude='*.fchk'"
{ SOURCE="sp02:/scratch/my_name"
REMOTE="${HOME}/work_dir/scratch_maschine01"
cd "${SOURCE}"
rsync --recursive -v --delete ${EXCLUDES} "./" "${REMOTE}/"
}>${START}/job.log 2>${START}/job.err
The key elements there are
the --recursive which will rsync will expand to include all content and subdirs of the SOURCE directory.
the / behind the ${SOURCE} notifies rsync to limit itself to content of the SOURCE directory, but not the directory itself.
the / behind the ${REMOTE} notifies rsync to limit itself to depositing content into that directory and expect it to already exist, to specifically fail if that does not already exist at REMOTE; this ensures that the remote site doesn't attempt a failsafe PWD and deposit files elsewhere than expected.
The above approach lends itself to a function form that could be placed into a loop with pre-attempt condition checks, along with having a complementary case for all variable assignments grouped under a destination heading (i.e. case statements).
Using such an approach with meaningful labels for variables lends itself to a type of implicit documentation, making the code more meaningful to someone not familiar with the code, as well as a refresher for yourself after a long period of not working or using the code.
I try to avoid the "~" because I prefer to always enclose definitions for variables in double quotes, to avoid issues that might arise from paths that may include unexpected characters or spaces. That way, you are sure to have your defined paths correctly interpreted by commands in scripts.
Lastly, I prefer to use the long form for the rsync options (and almost every other command) so that I don't have to refer to the manual every time to translate the single-character options when trying to understand what is coded, if the need arises for troubleshooting unexpected errors (I have always had poor memory).
My own backup command is as follows. The only reason why the
${PathMirror}${dirC}/
is not encapsulated in single quotes within the double quotes for COM is because I know those variables all evaluate to non-complex strings which cannot be misinterpreted.

Navigating directories when you don't know the name of the directory

I have a directory that is created through an external process. The directory is named 2021-12-08_1345 (YYYY-MM-DD_HHMM) based on the date and time when the process is executed. While this is the only directory in the path, I won't know the precise name of the directory. Is there a way to navigate to this folder knowing that it's the first and only directory?
The solution is cd $(ls -d -1 */ |sed -n '1p') where 1p is the nth directory that you want to navigate to. I came across the solution on an Ubuntu StackExchange https://askubuntu.com/questions/454688/how-do-you-cd-into-the-first-available-folder-without-typing-out-the-name#comment1800653_455113
I verified that this works on macOS 11.5.2+
If you are sure it is the only directory you can use
cd */
or
cd /path/to/*/
but this will fail if there is more than one directory.
Otherwise I suggest to use the solution from Ed Knittel's answer.
If you know that there is only one directory and no files you can even omit the trailing / from these commands, e.g. cd *.

Rsync copy "unsafe" symlinks but don't update modification time on the symlink targets

Is it possible to have rsync copy "unsafe" symlinks (that is, those that refer to files/dirs outside of the copied tree, see docs here) but not update the times on them?
I'm using rsync -a --delete --omit-dir-times to copy a bunch of files from /home/somebody/foo/bar to a destination machine, but running into the following error: rsync: failed to set times on "/home/somebody/foo/bar/symlink": Operation not permitted (1), where /home/somebody/foo/bar/smylink refers to something in /usr/lib/ owned by root at the destination and lacking proper permission for the rsync user to update it.
Essentially rsync tries to update the time on the symlink like all other files it copies, but gets blocked by permissions because it's not root at the destination.
What I'd like to do is copy the link, but not touch the symlink target at all during the copy. I just want the link. I could change permissions on the target file, but I'd like to avoid that.
Is this achievable? Is this a terrible idea and I'd be abusing rsync? Suggestions for alternative approaches in the latter case?
There is another option for rsync --omit-link-times which will probably do what you are looking for. See man page at:
http://manpages.ubuntu.com/manpages/bionic/man1/rsync.1.html

Creating multiple directories for users

I am using gatekeeper for access to pages on server.
This is done by creating directories with an index file in them. This then directs whoever inputted the password to a specific page.
I would like to be able to produce lots of directories with either not long random names or assigned names from say a database as creating many by a manual process is not practical.
Can someone tell me how to generate lots of directories on the fly?
Would be even better if users could create their own directory but thats probably something else.
Thanks
If you have bash (shell) access on your server, you can execute a simple bash script to create directories with a file in each.
for f in foo/bar{00..50}; do mkdir -p $f && touch $f/index.txt; done
Replace:
foo/bar with your directory
50 with the number of directories
index.txt with the name of the file
If you want to additionally write text to each file, then do this instead
for f in foo/bar{00..50}; do mkdir -p $f && printf "text\n goes\n here" > $f/index.txt; done

Symlink dotfiles

I am having trouble symlinking dotfiles. I have a folder in my home directory ~/dotfiles which I have synced to a github repo. I am trying to take my .vimrc file in ~/dotfiles/.vimrc and create a symbolic link to put it at ~/.vimrc. To do this I type in
ln -s ~/dotfiles/.vimrc ~/.vimrc
But when I run that it says
ln: /Users/me/.vimrc: File exists
What am I doing wrong?
That error message means that you already have a file at ~/.vimrc, which ln is refusing to overwrite. Either delete the ~/.vimrc and run ln again or let ln delete it for you by passing the -f option:
ln -s -f ~/dotfiles/.vimrc ~/.vimrc
There is a better solution for managing dotfiles without using symlinks or any other tool, just a git repo initialized with --bare.
A bare repository is special in a way that they omit working directory, so you can create your repo anywhere and set the --work-tree=$HOME then you don't need to do any work to maintain it.
Approach
first thing to do is, create a bare repo
git init --bare $HOME/.dotfiles
To use this bare repo, you need to specify --git-dir=$HOME/.dotfiles/ and --work-tree=$HOME, better is to create an alias
alias dotfiles='/usr/bin/git --git-dir=$HOME/.dotfiles/ --work-tree=$HOME
At this point, all your configuration files are being tracked, and you can easily use the newly registered dotfiles command to manage the repository, ex :-
# to check the status of the tracked and untracked files
dotfiles status
# to add a file
dotfiles commit .tmux.conf -m ".tmux.conf added"
# push new files or changes to the github
dotfiles push origin main
I also use this way to sync and store my dotfiles, see my dotfiles repository and can read at Storing dotfiles with Git where I wrote about managing for multiple devices.
How to symlink all dotfiles in a directory recursively
Have a dotfiles directory that is structured as to how they should be structured at $HOME
dotfiles_home=~/dotfiles/home # for example
cp -rsf "$dotfiles_home"/. ~
-r: Recursive, create the necessary directory for each file
-s: Create symlinks instead of copying
-f: Overwrite existing files (previously created symlinks, default .bashrc, etc)
/.: Make sure cp "copy" the contents of home instead of the home directory itself.
Tips
Just like ln, if you want no headache or drama, use an absolute path for the first argument like the example above.
Note
This only works with GNU cp (preinstalled in Ubuntu), not POSIX cp. Check your man cp, you can install GNU coreutils if needed.
Thanks
To this and this.

Resources