I have a SunOS system, which is what complicates this situation from being simple.
I have some scripts that run from different paths (unavoidable) and the system has a path structure that has the "System Environment" in the path, which I can then extract from the path. I have a simple script which is called before or sourced from every other script to get the Environment and set several other common variables. The problem is, now that there are 3 different areas that may be calling this script, it doesn't properly extract the Environment from the path.
Here are simple examples of the 3 paths that might exist:
/dir1/dir2/${ENV}/bin/script1.ksh
/dir1/dir2/${ENV}/services/service_name/script2.ksh
/dir1/dir2/${ENV}/services/service_name/log/script3.ksh
I'd like to have 1 script, that would be able to get ${ENV}, not matter which one of the paths was provided as opposed to my current strategy of 3 separate ones.
Here is how I currently get the first ${ENV}:
#!/bin/ksh
export BASE_DIR=${0%/*/*}
export ENV=${BASE_DIR##*/}
2nd Script:
#!/bin/ksh
export CURR_DIR=$( cd -- "$(dirname -- "$(command -v -- "$0")")" && pwd)
export BASE_DIR=${CURR_DIR%/*/*}
export ENV=${BASE_DIR##*/}
As I stated, this is a SunOS system, so it has an old limited version of KSH. No set -A or substitution.
Any ideas on the best strategy to limit my repetitiveness of scripts?
Thanks.
It looks from your example that your ${ENV} directory is a fixed depth from root, in which case you can easily get the name of the directory by starting from the other end;
export ENV=`pwd | sed -e "s%\(/dir1/dir2/\)\([^/]*\).*%\2%"`
I'm using '%' so I can match '/' without escaping. Without knowing specifics about what version of SunOS/Solaris you're using I can't be certain how compliant your sed is but Bruce Barnett includes it in his tutorials which are very closely aligned with late SunOS and early Solaris versions.
If your scripts are all called by the same user, then you might want to include the above in that user's .profile, then the ENV variable will be accessible to all scripts owned/executed by that user.
UPDATE: Lee E. McMahon's "SED -- A Non-Interactive Text Editor" - written in 1978 - includes pattern grouping using escaped parentheses, so it should work for you on SunOS. :)
Related
Instead of always giving the --indent n flag on the command line, I would like to specify this in a config file (or library or module file – I'm not sure what the correct terminology is).
I tried putting various things in ~/.jq, such as indent: 4 or --indent 4, but they always gave me a compile error when I invoked jq. I looked at the manual but couldn't see any information on how to do this, nor find anything in the issues on GitHub.
Is it possible?
The ~/.jq file contains jq code such as function definitions. It's not a config file in the classical sense.
Most shells support aliases and you could set one yourself, e.g. in ~/.profile or ~/.bashrc/~/.bash_aliases with:
alias jq='jq --indent 4'
When invoked in an interactive shell, the alias will be resolved to the command including the option. Note however that aliases are ignored in shell scripts (i.e. non-interactive mode) by default.
I looked up the forum but didn't find an article which matches my problem. Maybe there is some, and you can help me out with it.
My problem is I want to sync an folder with the command rsync -a -v. The point is I got 5 different Maschinen. On every maschine is a scratch folder I want to sync into the folder: ~/work_dir/scratch_maschines and inside the /scratch_maschines folder should be a folder for maschine_a, maschine_b and so on.
On the maschines it is always the same path: /scratch/my_name. So when I use now this command for the first two maschines:
rsync -a -v --exclude='*.chk' --exclude='*.rwf' --exclude='*.fchk' --delete sp02:/scratch/my_name ~/work_dir/scratch_maschine01; rsync -a -v --exclude='*.chk' --exclude='*.rwf' --exclude='*.fchk' --delete maschine02:/scratch/my_name ~/work_dir/scratch_maschine02
I got a folders for scratch_maschine01 and scratch_maschine02 in my working directory but inside these folders are not direct my data there is first a folder inside with my_name and this folder contains the data. So my question is how can I use the rsync command and get the files from the scratch directorys straight to the folders for each machine?
You might want to consider reformulating your commands similar to the following:
START=`pwd`
EXCLUDES="--exclude='*.chk' --exclude='*.rwf' --exclude='*.fchk'"
{ SOURCE="sp02:/scratch/my_name"
REMOTE="${HOME}/work_dir/scratch_maschine01"
cd "${SOURCE}"
rsync --recursive -v --delete ${EXCLUDES} "./" "${REMOTE}/"
}>${START}/job.log 2>${START}/job.err
The key elements there are
the --recursive which will rsync will expand to include all content and subdirs of the SOURCE directory.
the / behind the ${SOURCE} notifies rsync to limit itself to content of the SOURCE directory, but not the directory itself.
the / behind the ${REMOTE} notifies rsync to limit itself to depositing content into that directory and expect it to already exist, to specifically fail if that does not already exist at REMOTE; this ensures that the remote site doesn't attempt a failsafe PWD and deposit files elsewhere than expected.
The above approach lends itself to a function form that could be placed into a loop with pre-attempt condition checks, along with having a complementary case for all variable assignments grouped under a destination heading (i.e. case statements).
Using such an approach with meaningful labels for variables lends itself to a type of implicit documentation, making the code more meaningful to someone not familiar with the code, as well as a refresher for yourself after a long period of not working or using the code.
I try to avoid the "~" because I prefer to always enclose definitions for variables in double quotes, to avoid issues that might arise from paths that may include unexpected characters or spaces. That way, you are sure to have your defined paths correctly interpreted by commands in scripts.
Lastly, I prefer to use the long form for the rsync options (and almost every other command) so that I don't have to refer to the manual every time to translate the single-character options when trying to understand what is coded, if the need arises for troubleshooting unexpected errors (I have always had poor memory).
My own backup command is as follows. The only reason why the
${PathMirror}${dirC}/
is not encapsulated in single quotes within the double quotes for COM is because I know those variables all evaluate to non-complex strings which cannot be misinterpreted.
In the terminal, is there a way to create a directory with a file in it in one step?
Currently I do this in 2 steps:
1. mkdir foo
2. touch foo/bar.txt
Apparently, touch foo/bar.txt doesn't work.
With only standard unix tools, the most direct way to create a directory and a file in this directory is
mkdir foo && touch foo/bar.txt
Unix is built around the philosophy of simple, single-purpose tools with the shell as a glue to combine them. So to create a directory and a file, you instruct a shell to run the directory creation utility then the file creation utility.
I won't swear that there isn't some bizarre way of using a standard tool that lets you do it with a single command. (In fact, there is: unpack an archive — except that you'll need to provide that archive as a file, with predefined owner, date and other metadata, or else use another command to build an archive.) But whatever it is would be convoluted.
We work with Make files and want to create a precommit check in HG to check Makefile syntax. Originally, our check was just going to be
make -n FOO.mk
However, we realized that if a Makefile were syntactically correct but required some environment variable to be set, the test could fail.
Any ideas? Our default is to resort to writing our own python scripts to check for a limited subset of common Makefile mistakes.
We are using GNUmake.
$ make --dry-run > /dev/null
$ echo $?
0
The output is of no value to me so I always redirect to /dev/null (often stderr too) and rely on exit code. The man page https://linux.die.net/man/1/make explains:
-n, --just-print, --dry-run, --recon
Print the commands that would be executed, but do not execute them.
A syntax error would result in the sample output:
$ make --dry-run > /dev/null
Makefile:11: *** unterminated variable reference. Stop.
It is not a good idea to have makefiles depend on environment variables. Precisely because of the issue you mentioned.
Variables from the Environment:
... use of variables from the environment is not recommended. It is not wise for makefiles to depend for their functioning on environment variables set up outside their control, since this would cause different users to get different results from the same makefile. This is against the whole purpose of most makefiles.
References to an environment variable in the recipe need a $$ prefix so it is not that hard to find references to the pattern '[$][$][{] or the pattern [$][$][A-Z] which will find the direct references. A pretty simple perl filter (sed script) finds them all.
To find the indirect ones I would try the recipe with only PATH set and HOME set to /dev/null, and SHELL set to /bin/false. Make's macro SHELL is not the environment $SHELL, so you can get the recipes to run, you'll have to set SHELL=/bin/sh in the recipe file to run the command from the recipe. That should shake out enough data to help you find the depends.
What you do about the results is another issue.
At a unix command line, what's the difference between executing a program by simply typing it's name, vs. executing a program by typing a . (dot) followed by the program name? e.g.:
runme
vs.
. runme
. name sources the file called name into the current shell. So if a file contains this
A=hello
Then if you sources that, afterwards you can refer to a variable called A which will contain hello. But if you execute the file (given proper execution rights and #!/interpreterline), then such things won't work, since the variable and other things that script sets will only affects its subshell it is run in.
Sourcing a binary file will not make any sense: Shell wouldn't know how to interpret the binary stuff (remember it inserts the things appearing in that file into the current shell - much like the good old #include <file> mechanism in C). Example:
head -c 10 /dev/urandom > foo.sh; . foo.sh # don't do this at home!
bash: �ǻD$�/�: file or directory not found
Executing a binary file, however, does make a lot of sense, of course. So normally you want to just name the file you want to execute, and in special cases, like the A=hello case above, you want to source a file.
Using "source" or "." causes the commands to run in the current process. Running the script as an executable gives it its own process.
This matters most if you are trying to set environment variable in current shell (which you can't do in a separate process) or want to abort the script without aborting your shell (which you can only do in a separate process).
The first executes the command. The second is shorthand for including a shell script inside another.
This syntax is used to "load" and parse a script. It's most useful when you have a script that has common functionality to a bunch of other scripts, and you can just "dot include" it. See http://tldp.org/LDP/abs/html/internal.html for details (scroll down to the "dot" command).
Running "runme" will create a new process which will go on its merry little way and not affect your shell.
Running ". runme" will allow the script "runme" to change your environment variables, change directories, and all sorts of other things that you might want it to do for you. It can only do this because it's being interpreted by the shell process that's already running for you. As a consequence, if you're running bash as your login shell, you can only use the "." notation with a bash script, not (for example) a binary on C shell script.