Can I write a wrapper makefile that will cd one level up and execute there make with all the command options I have given the wrapper?
In more detail:
Directory project contains a real Makefile with some different targets.
Directory project/resources contains the wrapper Makefile which should call Makefile in project.
When I am in my shell in directory project/resources, I execute
make TARGET
and the Makefile there just cds one directory up and calls
make TARGET
in the directory project.
Is this possible? And how?
You could use a very simple Makefile for all your sub-directories:
%:
$(MAKE) -C .. $#
% is a last resort match-anything pattern rule that will match any target... for which there is no implicit rule (GNU make has an incredibly large number of implicit rules). So, if none of your targets are covered by an implicit rule, this should work. Else you will have to tell make not to use the implicit rules it knows. This can be done (with GNU make) by calling make with the -r option:
cd project/resources
make -r <anything>
will call make in project for target <anything>. The main drawback is that the -r flag is passed to the sub-make and so the implicit rules will not apply neither in project, which can be a problem. If it is you can obtain the same effect by adding an empty .SUFFIXES target to theMakefile in project/resources:
.SUFFIXES:
%:
$(MAKE) -C .. $#
With my version of GNU make (3.82) it works like a charm and the sub-make has all the default implicit rules.
Yes, you can have a makefile which works for "any" target.
The GNU make manual discusses this in the Overriding Part of Another Makefile section:
Sometimes it is useful to have a makefile that is mostly just like another makefile. You can often use the ‘include’ directive to include one in the other, and add more targets or variable definitions. However, it is invalid for two makefiles to give different recipes for the same target. But there is another way.
In the containing makefile (the one that wants to include the other), you can use a match-anything pattern rule to say that to remake any target that cannot be made from the information in the containing makefile, make should look in another makefile. See Pattern Rules, for more information on pattern rules.
For example, if you have a makefile called Makefile that says how to make the target ‘foo’ (and other targets), you can write a makefile called GNUmakefile that contains:
foo:
frobnicate > foo
%: force
#$(MAKE) -f Makefile $#
force: ;
If you say ‘make foo’, make will find GNUmakefile, read it, and see that to make foo, it needs to run the recipe ‘frobnicate > foo’. If you say ‘make bar’, make will find no way to make bar in GNUmakefile, so it will use the recipe from the pattern rule: ‘make -f Makefile bar’. If Makefile provides a rule for updating bar, make will apply the rule. And likewise for any other target that GNUmakefile does not say how to make.
The way this works is that the pattern rule has a pattern of just ‘%’, so it matches any target whatever. The rule specifies a prerequisite force, to guarantee that the recipe will be run even if the target file already exists. We give the force target an empty recipe to prevent make from searching for an implicit rule to build it—otherwise it would apply the same match-anything rule to force itself and create a prerequisite loop!
One option: use a wrapper file to execute the commands to do that. Just be sure your target make files don't include the child directory that has the wrapper, or else you can create an endless loop. For example,
clean:
pushd .. && make clean && popd
Using the comment of user Renaud Pacalet and the answer to a different question the following one-liner is as close as I could get. The whole Makefile reads:
IGNORE := $(shell $(MAKE) -C .. $(MAKECMDGOALS))
This solutions comes with a few caveats:
Command line option -B does not get passed through to the subsequent make call.
The output of the subsequently called make process (in the project directory) is not printed to stdout.
The wrapper make process reports for any given target at the end :
make: *** No rule to make target TARGET. Stop.
Related
I am trying to write a recursive make recipe. In this recipe, each target is dependent on a file with an equal name on the parent directory. A minimal (non-working) example:
foo/.dirstamp:
mkdir $(dir $#)
touch $#
.SECONDEXPANSION:
%/.dirstamp: $$(dir $$*).dirstamp
mkdir $(dir $#)
touch $#
With this example, I would expect make foo/bar/qux/lol/.dirstamp to generate the whole directory tree (if it does not exist), touching all .dirstamp files along the way. However, it does not work:
$ ls # note that there is nothing, make is meant to create the dir tree
Makefile
$ make --debug=v foo/bar/qux/lol/.dirstamp
GNU Make 4.0
[...]
Reading makefiles...
Reading makefile 'Makefile'...
Updating goal targets....
Considering target file 'foo/bar/qux/lol/.dirstamp'.
File 'foo/bar/qux/lol/.dirstamp' does not exist.
Finished prerequisites of target file 'foo/bar/qux/lol/.dirstamp'.
Must remake target 'foo/bar/qux/lol/.dirstamp'.
make: *** No rule to make target 'foo/bar/qux/lol/.dirstamp'. Stop.
It works fine as long as the recursive recipe only needs to be expanded twice, e.g., make foo/bar/.dirstamp works fine.
How can this work for an arbitrary number of levels? How can I handle a recursive expansion for the target and prerequisites names?
Note: my real problem is that the prerequisites of my recipes are in a root
directory different from the target so I am using the recipe above to duplicate the directory tree. I know about mkdir -p which seems to work fine in GNU systems. I am still interested on knowing how I would solve the recursion problem for arbitrary levels. which no longer works because part of the team is using Mac and mounting this directories over smb.
More details on the actual problem: prerequisites are in data/x/y/z while targets go into results/x/y/z. However, the results directory tree does not exist and needs to be created as needed. To solve this, I made the creation of parent directories an order-only prerequisite (via the .dirstamp files on my minimal example above).
can't copy data into results, that's several TB of data;
can't have the targets created in data, that's read-only;
can't use mkdir -p because the results directory will not be local, mounted over smb, and others may use non-GNU systems;
After an hint from #EtanReisner on the question:
make won't apply a rule more than once. That's a built-in (intentional) limitation. Without working around that with manual recursion or manually building the set of targets and using a static pattern rule (which may or may not actually work I'm not sure) there's not much you can do about this.
I worked up this solution:
RESULT_DIRS := $(patsubst data/%, results/%, $(shell find data/* -type d -print))
DIRSTAMPS := $(addsuffix /.dirstamp, $(RESULT_DIRS))
results/.dirstamp:
mkdir $(dir $#)
touch $#
.SECONDEXPANSION:
$(DIRSTAMPS): $$(dir $$(patsubst %/.dirstamp, %, $$#)).dirstamp
mkdir $(dir $#)
touch $#
It will duplicate the data directory tree in results as the dirstamp files are required. They are required by making them prerequisites of the other recipes (note the | which makes them order-only prerequisites):
results/%/foo.analysis: data/%/foo.data | results/%/.dirstamp
$(SOME_ANALYSIS_PROGRAM) $^ > $#
I'm writing my very first makefile and I'm stuck on a problem.
I have a bunch of prerequisites, of which the first one is a template that needs to be in a special position. I get to do this like so:
target : req1 req2 req3
command $(filter-out $<,$^) $# --template=$<
The thing is, sometimes I need to switch that template for another one while leaving the other prerequisites alone, so that
# Changing just the first prerequisite
target : req1b req2 req3
command $(filter-out $<,$^) $# --template=$<
I'm searching for a way to achieve this using the goal I have right now, without writing an ad-hoc explicit goal, maybe calling make with an argument or something similar, but I know too little about makefiles to get it done.
The general idea is you will want to use a variable, how you set that variable is up to you. One way is to pass a variable via the command line. Your Makefile would look like:
target : $(REQ_ONE) req2 req3
command $(filter-out $<,$^) $# --template=$<
and then do make target REQ_ONE=reg1 or make target REQ_ONE=reg1b
If you have a preferred default that you wish to use (say req1) and you want to use the alternative in rarer circumstances you could use the modified forms of the previous example.
# only set if the variable doesn't exist
REQ_ONE ?= req1
target : $(REQ_ONE) req2 req3
command $(filter-out $<,$^) $# --template=$<
Finally, a variant on this approach is to have your Makefile call make with a variable assignment:
# only set if the variable doesn't exist
REQ_ONE ?= req1
target2:
$(MAKE) target REQ_ONE=req1b
target : $(REQ_ONE) req2 req3
command $(filter-out $<,$^) $# --template=$<
Another solution is to use secondary expansion as demonstrated in this SO post on target specific variables as a prerequisites.
Is there a way to specify, in a .pro file, extra commands to be added to a standard target in the Makefile that qmake generates? For example, consider distclean, extra commands might be desired to:
Remove *~ files.
Clean out runtime-generated output files from the source tree.
Etc.
I want to use the normal target and not a custom target because I want this to be completely transparent in my workflow. That is (again using distclean as an example), I don't want to...
... require knowledge in a multi-project setup that certain Makefiles use a custom rule instead of distclean.
... document custom rules, even for stand-alone projects, as distclean is already well-known and intuitive†.
I found How to add custom targets in a qmake generated Makefile?, but this describes adding custom targets (which is already documented, even back in 4.6) rather than adding rules to existing targets. While it does contain some hints, all of them require adding new custom targets, as specifying the same target more than once in a Makefile replaces (not adds) commands from the previous target.
The only thing I could really think of to try was to add target.commands += new commands to the .pro file as a wild guess (e.g distclean.commands += rm \"*~\"). This has no effect.
How can I transparently add custom commands to existing targets with qmake?
† For the distclean example: While maintainer-clean is also on that "standard target" list, in practice I have found it to be rarely used, and in any case qmake doesn't generate it by default; I consider it to be unsuitable.
There are two straightforward ways to accomplish this, depending on how self-contained / portable you want your solution to be and how lenient you want to be with the order of command execution.
Option 1
The first option is to create a custom target in the .pro file for the new commands, then add that target as a prerequisite to the standard target that you are modifying. Going back to the distclean example, let's say you want to add a command to remove all *~ files:
Create a custom target in your .pro file. Note that you have to escape quotes and slashes in .pro files. For example, add:
extraclean.commands = find . -name \"*~\" -exec rm -v {} \\;
Add this target as a dependency of the target you are modifying:
distclean.depends = extraclean
This won't actually modify the distclean rule just yet, as this method can't be used to modify existing rules. However...
Add both your new target and the target you are modifying as extra targets:
QMAKE_EXTRA_TARGETS += distclean extraclean
This will add a second specification of distclean to the Makefile, but this works because you can add dependencies to existing targets in make in separate rules, even though you can't add commands that way. If you were to also specify distclean.commands in your .pro file, you would break the existing distclean by replacing its default recipe.
So, putting that all together, in the .pro file:
extraclean.commands = find . -name \"*~\" -exec rm -v {} \\;
distclean.depends = extraclean
QMAKE_EXTRA_TARGETS += distclean extraclean
Where extraclean is some custom target with the commands you want to add, and distclean is the existing target that you wish to modify.
Pros:
Completely self-contained in a .pro file.
As portable as you can get, leaves the actual Makefile syntax and generation up to qmake.
Cons:
Your new commands aren't appended to the existing recipe. Rather, they happen after all prerequisite targets are satisfied but before the existing recipe. In the distclean example, with the version of qmake that I'm using, this places the commands after the source tree clean but before Makefile itself is deleted (which is the only action the default recipe takes). This is not an issue for this example, but may be an issue for you.
Option 2
The second option is to change the name of the Makefile that qmake generates, and create your own custom Makefile that defers to the generated one, rather than includes + overrides it. This is also a straightforward option; while not as self-contained as option 1, it gives you the ability to execute commands both before and after the default generated recipe.
You don't want to include + override the existing Makefile, because you don't want to replace the default recipes. If you do, you have to re-implement the default, but this can be an issue as that default may change (and you have to keep up with the changes). It's best to let qmake do as much work as possible, and not repeat its work.
To do this:
First, change the name of the file that qmake generates. This can be accomplished by adding a line such as this to the .pro file:
MAKEFILE = RealMakefile
That will cause qmake to output RealMakefile instead of Makefile.
The next step is to create your own Makefile with your custom commands. However, there are some caveats here. First, a full example, again using distclean. In a file named Makefile:
.DEFAULT_GOAL := all
%:
#$(MAKE) -f RealMakefile $#
distclean:
#$(MAKE) -f RealMakefile $#
#find . -name "*~" -exec rm -v {} \;
Some notes about this:
We set .DEFAULT_GOAL because otherwise distclean would be the default. An alternative to this, if you're not comfortable with .DEFAULT_GOAL, is to specify an all rule using #$(MAKE) -f RealMakefile $# as the recipe.
The % target matches any target that isn't otherwise defined in this Makefile. It simply delegates processing to RealMakefile.
The distclean target is where we add our commands. We still need to delegate to RealMakefile, but additional commands can be added both before and after that happens.
Pros:
More control over command order. Commands can be added both before and after the default recipe.
Cons:
Not self-contained in a .pro.
Not as portable: It doesn't leave all Makefile generation up to qmake, and also I'm not actually sure what parts are specific to GNU make here (comments welcome).
So, while this answer may be a little long, both of these methods are very straightforward. I would prefer option 1 unless the command execution order is an issue.
Another solution is to add files you want to delete to the QMAKE_CLEAN and QMAKE_DISTCLEAN qmake variables.
build_tests {
TINYORM_SQLITE_DATABASE = $$quote($$TINYORM_BUILD_TREE/tests/q_tinyorm_test_1.sqlite3)
QMAKE_CLEAN = $$TINYORM_SQLITE_DATABASE
QMAKE_DISTCLEAN = $$TINYORM_SQLITE_DATABASE
}
It is relevant only, when do you know files you want to delete, so in this case, you can not use rm command or some sort of globbing.
It's probably trivial to do this but I can't see how.
I want to have a parent Makefile to decide which Makefile to call recursively based on the value of a variable passed in the command line.
I.e., I want to be able to call my main Makefile with:
make some_rule TARGET=a
or
make some_rule TARGET=b
and have my main Makefile decide based on the value of TARGET which makefile to invoke to run make some_rule. (For example, decide whether to call sub_directory_a/Makefile or sub_directory_b/Makefile to execute rule some_rule.)
Note: I have many different rules, so I do not want my main Makefile to list all the possible rules and for each of them call recursively the correct Makefile. I am hoping my main Makefile can only be a few lines long and not have to be updated whenever I create new rules.
You could do what you describe with
default_target:
%:
$(MAKE) -C some_directory_$(TARGET) $#
The %: rule is a pattern rule in which the pattern matches all rules (called a match-anything rule by the GNU make manual); $# is the current target. Note that the default_target: rule doesn't have a recipe, so calling make without a target will use the recipe of the match-anything rule (the only one that applies and has a recipe) to try to build default_target.
The caveat of this approach is that targets cannot be declared phony. If you want to have phony targets, you'll have to specify the recipe for those targets again, for example
PHONY_TARGETS = all clean distclean
.PHONY: $(PHONY_TARGETS)
$(PHONY_TARGETS):
$(MAKE) -C some_directory_$(TARGET) $#
%:
$(MAKE) -C some_directory_$(TARGET) $#
Unfortunately, I do not know a trick to declare all targets phony, which is what you'd really want to do.
Note that you can use ifeq etc. with the variables you set at the command line if you want to allow more fancy values for TARGET than parts of directory names, such as
%:
ifeq ($(TARGET),gibson)
echo 'Planet $# was successfully hacked.'
else
$(MAKE) -C some_directory_$(TARGET) $#
endif
Also note that a more common way to set common variables for many Makefiles is to put them into a file, often common.mk, and include it from the other Makefiles:
include ../common.mk # to include common.mk from some_directory_a/Makefile
But you'll have to decide yourself which approach is a better fit for your project.
I would like to have a makefile like this:
cudaLib :
# Create shared library with nvcc
ocelotLib :
# Create shared library for gpuocelot
build-cuda : cudaLib
make build
build-ocelot : ocelotLib
make build
build :
# build and link with the shared library
I.e. the *Lib tasks create a library that runs cuda directly on the device, or on gpuocelot respectively.
For both build tasks I need to run the same build steps, only creating the library differs.
Is there an alternative to running make directly?
make build
Kind of a post-requisite?
Note: This answer focuses on the aspect of a robust recursive invocation of a different target in a given makefile.
To complement Jack Kelly's helpful answer, here's a GNU makefile snippet that demonstrates the use of $(MAKE) to robustly invoke a different target in the same makefile (ensuring that the same make binary is called, and that the same makefile is targeted):
# Determine this makefile's path.
# Be sure to place this BEFORE `include` directives, if any.
THIS_FILE := $(lastword $(MAKEFILE_LIST))
target:
#echo $# # print target name
#$(MAKE) -f $(THIS_FILE) other-target # invoke other target
other-target:
#echo $# # print target name
Output:
$ make target
target
other-target
Using $(lastword $(MAKEFILE_LIST)) and -f ... ensures that the $(MAKE) command uses the same makefile, even if that makefile was passed with an explicit path (-f ...) when make was originally invoked.
Note: While GNU make does have features for recursive invocations - for instance, variable $(MAKE) specifically exists to enable them - their focus is on invoking subordinate makefiles, not on calling a different target in the same makefile.
That said, even though the workaround above is somewhat cumbersome and obscure, it does use regular features and should be robust.
Here is the link to the manual section covering recursive invocations ("sub-makes"):
Recursive Use of make
Most versions of make set a variable $(MAKE) that you can use for recursive invocations.
As you have written it, the build target will need to do something different depending on whether you have just done an ocelot or cuda build. That's another way of saying you have to parameterise build in some way. I suggest separate build targets (much like you already have), with associated variables. Something like:
build-cuda: cudaLib
build-ocelot: ocelotLib
build-cuda build-ocelot:
shell commands
which invoke ${opts-$#}
On the command-line you type make build-cuda (say). Make first builds cudaLib, then it carries out the recipe for build-cuda. It expands the macros before calling the shell. $# in this case is build-cuda, thus ${opts-$#} is first expanded to ${opts-build-cuda}. Make now goes on to expand ${opts-build-cuda}. You will have defined opts-build-cuda (and of course its sister opts-build-ocelot) elsewhere in the makefile.
P.S. Since build-cuda et. al. are not real files, you had better tell make this (.PHONY: build-cuda).