How can you export a variable to Make's 'shell' function? - gnu-make

Consider the following:
$ cat a.sh
#!/bin/sh
echo in a.sh, BANANA=$BANANA
$ cat Makefile
.PHONY: foo
export BANANA = I am a banana
foo:
$(eval F=`./a.sh`) # BANANA is set in a.sh
echo $F
$(eval G=$(shell ./a.sh)) # BANANA is *not* set in a.sh
echo $G
$ make
# BANANA is set in a.sh
echo `./a.sh`
in a.sh, BANANA=I am a banana
# BANANA is *not* set in a.sh
echo in a.sh, BANANA=
in a.sh, BANANA=
As demonstrated, the export directive to Make tells make to set the variable BANANA in the environment of its children. But that setting does not apply to the shell function. It does seem to apply to the backticks. Is this a bug? How can I easily set make variables to the environment of the shell function?
Note:
$ make --version
GNU Make 4.0
Built for x86_64-unknown-linux-gnu
Copyright (C) 1988-2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

First, be clear about the meaning of your recipe:
foo:
$(eval F=`./a.sh`) # BANANA is set in a.sh
echo $F
$(eval G=$(shell ./a.sh)) # BANANA is *not* set in a.sh
echo $G
This recipe contains two (not four) commands:
foo:
echo $F
echo $G
and the two make-functions:
$(eval F=`./a.sh`)
$(eval G=$(shell ./a.sh))
will be evaluated, in that order, for the scope of the two-line recipe when make decides to
run it. If you are surpised by this point, read this question
and answer.
Be clear also that F and G are both make variables, not shell variables. You
only get away with referring to $F and $G rather than $(F) and $(G) thanks
to the last para of 6.1 Basics of Variable References
A dollar sign followed by a character other than a dollar sign, open-parenthesis or open-brace
treats that single character as the variable name. Thus, you could reference the variable x with ‘$x’.
However, this practice is strongly discouraged, except in the case of the automatic variables
It wouldn't work for, say, FF and GG.
So the normal way to write your makefile would be:
.PHONY: foo
export BANANA = I am a banana
foo: F=`./a.sh`
foo: G=$(shell ./a.sh)
foo:
echo $(F)
echo $(G)
which has exactly the same effect.
And this perhaps clarifies the difference between the output of echo $(F)
and echo $(G).
$(shell ./a.sh) invokes a make function that executes ./a.sh in a shell
directly spawned by make and returns the stdout of so doing. Thus for target foo, make-variable G will
be defined as the stdout of executing ./a.sh in a child shell of make.
`./a.sh` does not invoke any make-function. As far as make is concerned, it is just
a string. For the target foo, make-variable F will be defined as `./a.sh `
The exported make-variable BANANA is not injected into the environment of a shell spawned by
$(shell ...). 5.7.2 Communicating Variables to a Sub-make
To pass down, or export, a variable, make adds the variable and its value to the
environment for running each line of the recipe
An exported variable and its definition is only injected into the environments of the shells that
run the lines of recipes.
Thus BANANA is not defined in the environment of a.sh when it is run by $(shell ./a.sh)
to generate the definition of G. But it is defined in the environment of the shell
that that runs the recipe line echo $(F), with $(F) = `a.sh`. That shell (not make) interprets
`a.sh` as a back-tick invocation of a subshell, which inherits the definition of
BANANA.
To get BANANA exported into the environment of $(shell ...), you have to do
it yourself since it is not done by make:
G=$(shell export BANANA='$(BANANA)'; ./a.sh)

Related

Make: run common pre/post rules for a set of targets

I want to build the following dependency graph, but with pre and post being artifact free:
Before creating/updating any of a, b, or c the command pre should run once and afterwards post should run once. Both do not and preferably should not produce artifacts. And of course, these should only be run if any of a b c have changed. This should all be triggered by a phony all target, i.e. a is never run independently.
Using order-only prerequisites a: | pre does not help because these are always run. Making post depend on a b c won't work because then it is also run all the time because post does not create an artifact.
If this is impossible and artifacts are required after all, how would pre (the more interesting of the two) only run if any of the targets which depend on it have changed?
Note: a etc. are normal makefile targets (which could be called independently), e.g.:
a: a.in a.dependency
#echo Creating a
#mkabc a.in > a
There is only one way in make to force a command to execute before target X is built, but only if target X needs to be built, and that's put the command as the first thing in the recipe for target X. There's no way to manipulate the dependency graph in GNU make so that make determines if a target needs to be built and, if so, first builds some other target before the recipe runs.
So you will definitely have to use recursive make here, putting the command to build the pre target into the recipe of the other targets. However, of course that will cause it to be built multiple times which you don't want.
One way to get around that is to play a trick using eval. Try this (untested):
BUILD_PRE = $(shell $(MAKE) -j1 pre >/dev/null)
post: a b c
echo $#
pre:
echo $#
a b c:
$(BUILD_PRE)$(eval BUILD_PRE =)
touch $#
.PHONY: pre post
In the rule for a, b, and c we first expand the BUILD_PRE variable which results in a recursive make invocation via the shell call. Then the eval expansion will reset the value of BUILD_PRE so that it's now empty; this means in subsequent rules for b and c this first line will expand to the empty string and pre will not be run again.
You may ask, why do we need to use shell here? Can't we just use:
BUILD_PRE = $(MAKE) -j1 pre
so that the first recipe contained a recursive make? The problem with this is that it won't work with parallel make. Suppose the first target make attempts to build is a (it will always be of course). That recipe will contain a recursive make invocation and make will start it. But if you are using -j make does not wait for that recipe to complete: it will go try to start b and c. Since BUILD_PRE is now empty you only get one build of pre, but b and c are not waiting for pre to be completed.
By using a shell function the recursive invocation is forced to complete when the recipe is expanded, before any other recipe is started.
I should say, I suspect there may be a few odd things about this. In particular when make normally invokes a recursive build it does some setup etc. that won't happen when a recursive build is invoked through shell. But, it should work.
Edit: Final Makefile with '+' prefix to mark recursive make calls:
all: allabc
BUILD_PRE = $(shell $(MAKE) pre)
BUILD_POST =
pre:
#echo PRE abc >&2
post:
#echo POST abc >&2
allabc: a b c
#+$(BUILD_POST) > /dev/null
a:
+$(BUILD_PRE)$(eval BUILD_PRE = )
touch "$#"
$(eval BUILD_POST = $$(MAKE) post)
b:
+$(BUILD_PRE)$(eval BUILD_PRE = )
touch "$#"
$(eval BUILD_POST = $$(MAKE) post)
c:
+$(BUILD_PRE)$(eval BUILD_PRE = )
touch "$#"
$(eval BUILD_POST = $$(MAKE) post)
clean:
rm -f a b c
Not sure I understand all the details but assuming you want to build your 5 targets when invoking make all, with the dependencies you show (and maybe a, b and c in parallel), you can, for instance:
.PHONY: all pre post
all:
$(MAKE) pre
$(MAKE) a b c
$(MAKE) post
pre:
<pre-recipe>
post:
<post-recipe>
a:
<a-recipe>
...

Initializing variables in bash

I'm migrating windows CMD script to bin/bash on unix.
The goal of initial script was to setting up some variables, so after anything run from this cmd window was using that variables.
How can I do same in UNIX? Looks like simple
MyVar="value"
doesn't work. It visible only in script itself, not from terminal where it was run.
You can initialize shell variables with simple assignments
$ foo="fooval"
$ echo $foo
fooval
These variables won't spread to unrelated child processes:
$ foo=fooval
$ sh -c 'printf "\"%s\"" $foo'
""
To make them spread, you need to export them into the process's (shell's)
environment (make them into "environment variables" (these are commonly capitalized, i.e.,
FOO instead of foo)
$ export foo
$ sh -c 'echo $foo'
fooval
You can assign and export in one step:
$ export foo=fooval
Environment variables will never spread anywhere but down the process hierarchy.
(Only to children, never to parents or completely unrelated processes)
Therefore, if you have a script with variable assignments, you need to source it, not execute it:
$ ./envvars #won't affect the parent shell
$ . ./envvars #this will
There are no per-terminal variables (though there are per-terminal configurations with fixed keys accessible manipulatable with the stty tool).
Create a file test.sh
add the following line:
export b="key"
Now goto the terminal and do the following :
source ./test.sh
echo $b
Output:
key

Renaming multiple files using parameter unix

I have a rename script below rename.sh. I want to introduce a variable such that I can pass a date argument when executing the script like./rename.sh 20151103 such that 20151103 replaces 20140306 in the script.
for f in *.CDR*; do
echo mv "$f" "${f/-20140306/-0-20140306}"
done
Thinking of automating this as I don't want to manually edit the script each time i'm doing a rename. Any other method will be highly welcomed.
#!/bin/bash
pattern="$1"
for f in *.CDR*; do
echo mv "$f" "${f/-${pattern}/-0-${pattern}}"
done
Explanation:
The #!-line says we're running this as a bash script.
The script will populate the variables $1, $2 etc. with the arguments handed to it on the command line. These are called the positional parameters ($0 usually holds the name of the script).
We take $1, because we know that should contain the pattern we're replacing, and assign it to the variable $pattern. In much more complex scripts, here is where we would handle command line switches (with getopts, but that's an answer for another day).
We quote $1, just because. (It's good practice to quote user input, just to be sure no shell-globbing characters, such as * gets expanded).
The rest is the script like you had from before, but with the string 20140306 replaced by ${pattern}. I'm using ${pattern} rather than $pattern here for readability only. In general, you need to use ${a} rather than $a if you, for example, interpolate a string like "${a}nospaceafter".
Then it should just be a matter of making the script executable before testing it:
$ chmod +x rename.sh
This is the one of the method you can consider:
#!/bin/bash
input=$1
for f in *.CDR*; do
echo mv "$f" "${f/-$input/-0-$input}"
done

Make zsh complete arguments from a file

zsh is great but its completion system is very diverse. And the documentation lacks good examples. Is there a template for completing for a specific application. The completion would get its match data from a file, separated by newlines?
I tried modifying an older example of mine that takes match data "live":
~ % cat .zsh/completers/_jazzup
#compdef jazz_up
_arguments "2: :(`mpc lsplaylists|sed -e 's# #\\\\ #g'`)"
I could supply cat my_file there instead of mpc invocation and so on but would there be a more elegant way to do this simple task? And that completion there is placement-specific: can you provide an example where zsh would attempt to complete at any point after the program name is recognized?
The match data will have whitespaces and so on, the completion should escape the WS. Example of that:
Foo bar
Barbaric
Get it (42)
Now if that completion would be configured for a command Say, we should get this kind of behaviour out of zsh:
$ Say Fo<TAB>
$ Say Foo\ bar
$ Say Ge<TAB>
$ Say Get\ it\ \(42\)
Simple completion needs are better addressed with _describe, it pairs an array holding completion options and a description for them (you can use multiple array/description pairs, check the manual).
(_arguments is great but too complex.)
[...]
First create a file
echo "foo\nbar\nbaz\nwith spac e s\noh:noes\noh\:yes" >! ~/simple-complete
Then create a file _simple somewhere in your $fpath:
#compdef simple
# you may wish to modify the expansion options here
# PS: 'f' is the flag making one entry per line
cmds=( ${(uf)"$(< ~/simple-complete)"} )
# main advantage here is that it is easy to understand, see alternative below
_describe 'a description of the completion options' cmds
# this is the equivalent _arguments command... too complex for what it does
## _arguments '*:foo:(${cmds})'
then
function simple() { echo $* }
autoload _simple # do not forget BEFORE the next cmd!
compdef _simple simple # binds the completion function to a command
simple [TAB]
it works. Just make sure the completion file _simple is placed somewhere in your fpath.
Notice that : in the option list is supposed to be used for separating an option from their (individual) description (oh:noes). So that won't work with _describe unless you quote it (oh\:yes). The commented out _arguments example will not use the : as a separator.
Without changing anything further in .zshrc (I already have autoload -Uz compinit
compinit) I added the following as /usr/local/share/zsh/site-functions/_drush
#compdef drush
_arguments "1: :($(/usr/local/bin/aliases-drush.php))"
Where /usr/local/bin/aliases-drush.php just prints a list of strings, each string being a potential first argument for the command drush. You could use ($(< filename)) to complete from filename.
I based this on https://unix.stackexchange.com/a/458850/9452 -- it's surprising how simple this is at the end of the day.

How can I tell if a makefile is being run from an interactive shell?

I have a makefile which runs commands that can take a while. I'd like those commands to be chatty if the build is initiated from an interactive shell but quieter if not (specifically, by cron). Something along the lines of (pseudocode):
foo_opts = -a -b -c
if (make was invoked from an interactive shell):
foo_opts += --verbose
all: bar baz
foo $(foo_opts)
This is GNU make. If the specifics of what I'm doing matter, I can edit the question.
It isn't strictly determining whether it is invoked from an interactive shell or not, but for a cron job in which the output is redirected to a file, the answer to this question would be the same as for How to detect if my shell script is running through a pipe?:
if [ -t 0 ]
then
# input is from a terminal
fi
Edit: To use this to set a variable in a Makefile (in GNU make, that is):
INTERACTIVE:=$(shell [ -t 0 ] && echo 1)
ifdef INTERACTIVE
# is a terminal
else
# cron job
endif
http://www.faqs.org/faqs/unix-faq/faq/part5/section-5.html
5.5) How can I tell if I am running an interactive shell?
In the C shell category, look for the variable $prompt.
In the Bourne shell category, you can look for the variable $PS1,
however, it is better to check the variable $-. If $- contains
an 'i', the shell is interactive. Test like so:
case $- in
*i*) # do things for interactive shell
;;
*) # do things for non-interactive shell
;;
esac
I do not think you can easily find out. I suggest adopting an alternative strategy, probably by quelling the verbose output from the cron job. I would look to do that using a makefile like this:
VERBOSE = --verbose
foo_opts = -a -b -c ${VERBOSE}
all: bar baz
foo $(foo_opts)
Then, in the cron job, specify:
make VERBOSE=
This command-line specification of VERBOSE overrides the one in the makefile (and cannot be changed by the makefile). That way, the specialized task (cron job) that you set up once and use many times will be done without the verbose output; the general task of building will be done verbosely (unless you elect to override the verbose-ness on the command line).
One minor advantage of this technique is that it will work with any variant of make; it does not depend on any GNU Make facility.
I’m not really sure what "am interactive" means. Do you mean if you have a valid /dev/tty? If so, then you could check that. Most of us check isatty on stdin, though, because it answers the questions we want to know: is there someone there to type something.
Just a note: you can also see the related discussion that I had about detecting redirection of STDOUT from inside a Makefile.
I believe it will be helpful to readers of this question - executive summary:
-include piped.mk
all: piped.mk
ifeq ($(PIPED),1)
#echo Output of make is piped because PIPED is ${PIPED}
else
#echo Output of make is NOT piped because PIPED is ${PIPED}
endif
#rm -f piped.mk
piped.mk:
#[ -t 1 ] && PIPED=0 || PIPED=1 ; echo "PIPED=$${PIPED}" > piped.mk
$ make
Output of make is NOT piped because PIPED is 0
$ make | more
Output of make is piped because PIPED is 1
In my answer there I explain why the [-t 1] has to be done in an action and not in a variable assignment (as in the recommended answer here), as well as the various pitfalls regarding re-evaluation of a generated Makefile (i.e. the piped.mk above).
The term interactive in this question seems to imply redirection of STDIN... in which case replacing [ -t 1 ] with [ -t 0 ] in my code above should work as-is.
Hope this helps.

Resources