This is how I store my projects on the server dev.anuary.com/mike-gamaroff/2011 10 04 unbreakable-embrace/, i.e. server + client name + project data and name.
The problem is, that some clients started to complain about their names appearing in the URL, when it is visible. Is there a way to get to the URL path using some sort of hash of the original directory path?
That dev.anuary.com/mike-gamaroff/2011 10 04 unbreakable-embrace/ would become dev.anuary.com/2989dh42[whatever]098dfsdfsf/, though, I could still keep the original directory tree?
Create symbolic links on the server?
PROJECT_NAME="$server/$clientName/$projectData $name/"
ln -s /var/www/vhosts/dev.anuary.com/htdocs/$server/$projectData\ $name /var/www/vhosts/dev.anuary.com/htdocs/mike-gamaroff/`echo $PROJECT_NAME | md5sum`
Just a quick guess ...
Then, the site would be available via the normal URL you are speaking about, or via the md5sum URL.
If you want to have this enabled for all projects, you'll need to either manually create them, or do some fun recursion with a shell script down through all the various directories. The benefit to doing this is that you can update a master file that provides a proper reference between project name and hash .. for quick reference
for dir in $(find ./ -type d); do
# do some fancy symbolic linking if the criteria is right
done
Related
When creating filepaths and URLs, I noticed that many times the path starts with ./ or ~/.
What is the difference between filepaths that start with ./ and ~/?
What do each of them mean?
For the sake of completeness ...
Just path is a file or directory named path in the current directory.
./path is a file or directory named path in the current directory, with the directory spelled out. The dot directory . represents the current directory, and path is the name of the file or directory within this directory.
~/path is a shorthand for $HOME/path where $HOME is a variable which refers to your home directory. Typically your home directory will be somewhere like /home/you or /Users/you where you is your account name. (The command echo "$HOME" will display your home directory.) The expanded value is an absolute path (unless you have messed up the value of $HOME thoroughly), as indicated by the initial slash.
/path is an absolute path which refers to a file or directory named path which is in the root directory /. Every file on Unix is ultimately somewhere in the directory tree which starts with the root directory.
A file name which begins with $ includes the value of a shell variable in its name (like for example $HOME above); you have to know the value of that variable to determine whether it ends up containing a relative or an absolute path. Similarly, ~ at the beginning of a file name gets replaced ("expanded") by the shell to a different string, as outlined above.
(Technically, it's possible for a file name to begin with a literal dollar sign or tilde, too; you would then have to quote or backslash-escape that character to avoid having the shell expand it to something else. This is rather inconvenient, so these file names tend to be rare in practice.)
In the following exposition, we refer to the result of any such replacements, and ignore the complication of possible quoting.
Every file name which begins with / is an absolute path (aka full path) which explains how to reach a particular node starting from the root directory. For example, /var/tmp/you/reminder.txt refers to a file or directory reminder.txt (probably a file, judging from the name; but Unix doesn't care what you call your files or directories) which is in the directory you which is in the directory tmp which is in the directory var which is in the root directory.
Every file name which doesn't begin with / is a relative path which indicates how to reach a particular file or directory starting from the current directory. The special directory .. is the parent directory (that is, the directory which contains this directory) and the special directory . is the current directory. So path/there refers to the file or directory there inside the directory path in the current directory; and (hover the mouse over the gray area to display the spoiler)
there/.././and/back/.. is a (wicked complicated) way to refer to the directory and in the current directory, where we traverse the there directory and then move back to the current directory; then stay in the current directory; then refer to the directory back inside the directory and, but then move back to the parent directory of that, ending up with ./and.
In addition to ~/ for the current user's home directory, some shells and applications allow the notation ~them/ to refer to the home directory of the user account them. Also, some web server configurations allow each user to have a public web site in their directory ~/public_html and the URL notation http://server/~them/ would serve up the site of the user account them for outside visitors.
The current directory is a convenience which the shell provides so you don't have to type long paths all the time. You can, if you want to.
/bin/ls /home/you/Documents/unix-101/directories.txt
is a longwinded but perfectly valid way to say (assuming you are in your home directory),
ls Documents/unix-101/directories.txt
You could also say
cd Documents/unix-101
ls directories.txt
and until you cd again, all your commands will run in this directory.
See What exactly is current working directory? for a longer exposition of this related concept.
A "directory" is sometimes called a "folder" by people who are not yet old enough to prefer the former.
Tangentially, don't confuse the directory name . with the Bourne shell command which comprises a single dot (also known by its Bash alias source). The command
. ./scriptname
runs the commands from the file ./scriptname in the context of the current shell instance, as opposed to in a separate subshell (which is what just ./scriptname does). In other words, this command line invokes the dot command on a file scriptname in the dot directory.
The Bourne shell (and derivatives like Bash, Zsh, etc) use single quotes to prevent variable expansion and wildcard expansion, and double quotes to permit variable expansion, but inhibit wildcard expansion in a string. The quoting rules on Windows are different, and generally use double quotes to keep whitespace-separated values as a single string (and % instead of $ for variable substitutions).
./ means "starting from the current directory". . refers to the current working directory, so something like ./foo.bar would be looking for a file called foo.bar in the current directory. (As a side note, .. means refers to the parent directory of the current directory. So ../foo.bar would be looking for that file one directory above.)
~/ means "starting from the home directory". This could have different meanings in different scenarios. For example, in a Unix environment ~/foo.bar would be looking for a file called foo.bar in your home directory, something like /home/totzam/foo.bar. In many web applications, ~/foo.bar would be looking for a file called foo.bar in the web application root, something like /var/http/mywebapp/foo.bar.
./ is the current directory
~/ is the home directory of the current user
./ means that path is relative to your current position.
~/ means that path is relative to your home directory.
I will explain a simple example of it. As developers mentioned:
./ is current directory.
~/ is the home directory of the current user.
How both of the file path expressions can help us? Suppose you want to execute a script (.sh) and you're in the same directory where file exists then you can simply do it ./filename.sh
I mostly use ~/ to access my home directory files like .bashrc when I want to add any config in it. It's easier since the file path expression (for home directory) feels much easier and makes accessibility to the file from anywhere, without worrying about the path or changing the path.
. represents current directory
.. represents the parent directory
~ represents the home directory for the current user. Home directory is also represented by HOME env variable. you can do echo $HOME on the shell to see it.
These are generally used to specify relative paths. The / in the end of each notation is a separator that you can use when using these notations together.
Ex:
$ cd ../.. # Go 2 directories backwards
$ cd ~ # Takes you to $HOME directory
$ cd . # Does nothing :) As it literally means go to the directory that you are already present in.
$ cd ~/dir1 $ go to `$HOME/dir1`
On Unix, in any directory if you do ls -a you would see that . and .. will be mentioned (even for empty directory). Like mentioned, these have special meaning and are generated by default in Unix systems and are generally helpful to specify relative paths (i.e, path to a different directory relative to your current directory)
cd command is harmless. So, just play around by combining notations with cd command. You will eventually get a grip of them.
I am trying to use rsync to do backups. I have an include file called /etc/daily.rsync and it contains the following:
+ /home/demo
- *
Then I run the command below:
$ sudo rsync -acvv --delete --include-from=/etc/daily.rsync /mnt/offsite_backup/home/
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
drwxrwxr-x 6 2021/02/22 14:09:13 .
total: matches=0 hash_hits=0 false_alarms=0 data=0
sent 52 bytes received 131 bytes 366.00 bytes/sec
total size is 0 speedup is 0.00
When I go look in the directory I see nothing. What I think is that it is trying to rsync from the current directory which btw is empty. So this leaves me to believe that it is not getting the data form the include file.
This command runs as expected:
sudo rsync -acvv --delete /home/demo /mnt/offsite_backup/home/
The different posts made many suggestions, and I have tried them. I am just stuck. Any thoughts would be very welcome.
I think you're misunderstanding what a filter file (like the one you specified with --include-from) does. It does not specify where to sync files from; it specifies which files within the source directory to sync.
You need to specify both the source and destination as part of the command line. In the command:
sudo rsync -acvv --delete --include-from=/etc/daily.rsync /mnt/offsite_backup/home/
You only specified one directory, /mnt/offsite_backup/home/, so rsync has assumed it's the source, and there is no destination. According to the rsync man page:
As a special case, if a single source arg is specified without a
destination, the files are listed in an output format similar to "ls -l".
So, basically, it's listing the contents of /mnt/offsite_backup/home/, and apparently that's empty.
The second command you gave specifies both the source and destination, which is why it works correctly. If you want to add a filter file to, be aware that the paths in the filter will be relative to the source. So if you used
sudo rsync -acvv --delete --include-from=/etc/daily.rsync /home/demo /mnt/offsite_backup/home/
...it's going to try to include the file/directory /home/demo/home/demo, which probably doesn't exist. Except it actually won't do that, because the - * line will exclude /home/demo/home, so if it did exist, it and its contents will be excluded. You need to include the parent directories of anything you want to include in the sync operation. Again, from the man page:
The concept path exclusion is particularly important when using a
trailing '*' rule. For instance, this won't work:
+ /some/path/this-file-will-not-be-found
+ /file-is-included
- *
This fails because the parent directory "some" is excluded by the '*' rule, so rsync never visits any of the files in the "some" or
"some/path" directories. One solution is to ask for all directories in
the hierarchy to be included by using a single rule: "+ */" (put it
somewhere before the "- *" rule), and perhaps use the
--prune-empty-dirs option. Another solution is to add specific include rules for all the parent dirs that need to be visited. For instance,
this set of rules works fine:
+ /some/
+ /some/path/
+ /some/path/this-file-is-found
+ /file-also-included
- *
ok, so after walking away from the problem I realized that, I never specified what actual directory I wanted to sync. The include can't work from thin air. so the command is:
sudo rsync -acv --delete /home/ --include-from=/etc/weekly.rsync /mnt/offline_backup/home/
The include file had to change as well.
+ demo/***
+ truenorth/***
- *
To have it decend into the directory structure, I needed the ***. I hope this can help someone else out.
I am using gatekeeper for access to pages on server.
This is done by creating directories with an index file in them. This then directs whoever inputted the password to a specific page.
I would like to be able to produce lots of directories with either not long random names or assigned names from say a database as creating many by a manual process is not practical.
Can someone tell me how to generate lots of directories on the fly?
Would be even better if users could create their own directory but thats probably something else.
Thanks
If you have bash (shell) access on your server, you can execute a simple bash script to create directories with a file in each.
for f in foo/bar{00..50}; do mkdir -p $f && touch $f/index.txt; done
Replace:
foo/bar with your directory
50 with the number of directories
index.txt with the name of the file
If you want to additionally write text to each file, then do this instead
for f in foo/bar{00..50}; do mkdir -p $f && printf "text\n goes\n here" > $f/index.txt; done
When uploading changed files to a live webserver, how do I enforce at any given moment, my entire file set is either in the old state, or new, but never something in between?
With 'in between' I mean some files old and some new, or some particular file being overwritten halfway, et cetera.
Suppose I have a directory called foo/ on my live webserver containing a framework or bunch of script files including eachother, and I need to replace a whole bunch of files at the same time. I'd call this an 'atomic update'.
The closest I can get is uploading my new foo/ directory (containing the new files) to a different name, e.g. bar/, and then on a shell on the webserver I do:
mv foo foo-old; mv bar foo;
But this way there is still a tiny fraction of a second where foo/ doesn't exist, when the old dir has just been renamed and the new dir is about to.
Is there a 100% correct way of doing this? I guess I need some sort of 'atomic swap', to rename or swap two directory names as a single, atomic action on file system level.
In case it's OS dependent: I'm using a webserver running CentOS and got SSH access.
Near-atomic
The simplest way to achieve a near-atomic change of directories is to use a symbolic link as your web root, which you can re-point to a different location when upgrading.
$ mkdir old
$ mkdir new
$ ln -s old live
$ ls -l
live -> old
new
old
...
$ ln -snf new live
live -> new
new
old
Changing the target of a symbolic link is actually a 2 step operation internally, unlink followed by symlink.
Atomic
An atomic change of directories can be achieved by creating a secondary symbolic link which points to a new directory, and then renaming the new symbolic link into the old symbolic link.
$ mkdir old new
$ ln -s old live
$ ln -s new live_new
$ mv -fT live_new live
mv command will use a single atomic operation (rename) to overwrite the old symbolic link with a new one.
Network layer atomic
Use 2 separate hosts (physical or virtual) and route every new user from a defined moment in time to a new host which contains the upgraded website.
So this doesn't seem like a terribly complicated question I have, but it's one I can't find the answer to. I'm confused about what the -p option does in Unix. I used it for a lab assignment while creating a subdirectory and then another subdirectory within that one. It looked like this:
mkdir -p cmps012m/lab1
This is in a private directory with normal rights (rlidwka). Oh, and would someone mind giving a little explanation of what rlidwka means? I'm not a total noob to Unix, but I'm not really familiar with what this means. Hopefully that's not too vague of a question.
The man pages is the best source of information you can find... and is at your fingertips: man mkdir yields this about -p switch:
-p, --parents
no error if existing, make parent directories as needed
Use case example: Assume I want to create directories hello/goodbye but none exist:
$mkdir hello/goodbye
mkdir:cannot create directory 'hello/goodbye': No such file or directory
$mkdir -p hello/goodbye
$
-p created both, hello and goodbye
This means that the command will create all the directories necessaries to fulfill your request, not returning any error in case that directory exists.
About rlidwka, Google has a very good memory for acronyms :). My search returned this for example: http://www.cs.cmu.edu/~help/afs/afs_acls.html
Directory permissions
l (lookup)
Allows one to list the contents of a directory. It does not allow the reading of files.
i (insert)
Allows one to create new files in a directory or copy new files to a directory.
d (delete)
Allows one to remove files and sub-directories from a directory.
a (administer)
Allows one to change a directory's ACL. The owner of a directory can always change the ACL of a directory that s/he owns, along with the ACLs of any subdirectories in that directory.
File permissions
r (read)
Allows one to read the contents of file in the directory.
w (write)
Allows one to modify the contents of files in a directory and use chmod on them.
k (lock)
Allows programs to lock files in a directory.
Hence rlidwka means: All permissions on.
It's worth mentioning, as #KeithThompson pointed out in the comments, that not all Unix systems support ACL. So probably the rlidwka concept doesn't apply here.
-p|--parent will be used if you are trying to create a directory with top-down approach. That will create the parent directory then child and so on iff none exists.
-p, --parents
no error if existing, make parent directories as needed
About rlidwka it means giving full or administrative access. Found it here https://itservices.stanford.edu/service/afs/intro/permissions/unix.
mkdir [-switch] foldername
-p is a switch, which is optional. It will create a subfolder and a parent folder as well, even if parent folder doesn't exist.
From the man page:
-p, --parents no error if existing, make parent directories as needed
Example:
mkdir -p storage/framework/{sessions,views,cache}
This will create subfolder sessions,views,cache inside framework folder irrespective of whether 'framework' was available earlier or not.
PATH: Answered long ago, however, it maybe more helpful to think of -p as "Path" (easier to remember), as in this causes mkdir to create every part of the path that isn't already there.
mkdir -p /usr/bin/comm/diff/er/fence
if /usr/bin/comm already exists, it acts like:
mkdir /usr/bin/comm/diff
mkdir /usr/bin/comm/diff/er
mkdir /usr/bin/comm/diff/er/fence
As you can see, it saves you a bit of typing, and thinking, since you don't have to figure out what's already there and what isn't.
Note that -p is an argument to the mkdir command specifically, not the whole of Unix. Every command can have whatever arguments it needs.
In this case it means "parents", meaning mkdir will create a directory and any parents that don't already exist.