I would like to have a makefile like this:
cudaLib :
# Create shared library with nvcc
ocelotLib :
# Create shared library for gpuocelot
build-cuda : cudaLib
make build
build-ocelot : ocelotLib
make build
build :
# build and link with the shared library
I.e. the *Lib tasks create a library that runs cuda directly on the device, or on gpuocelot respectively.
For both build tasks I need to run the same build steps, only creating the library differs.
Is there an alternative to running make directly?
make build
Kind of a post-requisite?
Note: This answer focuses on the aspect of a robust recursive invocation of a different target in a given makefile.
To complement Jack Kelly's helpful answer, here's a GNU makefile snippet that demonstrates the use of $(MAKE) to robustly invoke a different target in the same makefile (ensuring that the same make binary is called, and that the same makefile is targeted):
# Determine this makefile's path.
# Be sure to place this BEFORE `include` directives, if any.
THIS_FILE := $(lastword $(MAKEFILE_LIST))
target:
#echo $# # print target name
#$(MAKE) -f $(THIS_FILE) other-target # invoke other target
other-target:
#echo $# # print target name
Output:
$ make target
target
other-target
Using $(lastword $(MAKEFILE_LIST)) and -f ... ensures that the $(MAKE) command uses the same makefile, even if that makefile was passed with an explicit path (-f ...) when make was originally invoked.
Note: While GNU make does have features for recursive invocations - for instance, variable $(MAKE) specifically exists to enable them - their focus is on invoking subordinate makefiles, not on calling a different target in the same makefile.
That said, even though the workaround above is somewhat cumbersome and obscure, it does use regular features and should be robust.
Here is the link to the manual section covering recursive invocations ("sub-makes"):
Recursive Use of make
Most versions of make set a variable $(MAKE) that you can use for recursive invocations.
As you have written it, the build target will need to do something different depending on whether you have just done an ocelot or cuda build. That's another way of saying you have to parameterise build in some way. I suggest separate build targets (much like you already have), with associated variables. Something like:
build-cuda: cudaLib
build-ocelot: ocelotLib
build-cuda build-ocelot:
shell commands
which invoke ${opts-$#}
On the command-line you type make build-cuda (say). Make first builds cudaLib, then it carries out the recipe for build-cuda. It expands the macros before calling the shell. $# in this case is build-cuda, thus ${opts-$#} is first expanded to ${opts-build-cuda}. Make now goes on to expand ${opts-build-cuda}. You will have defined opts-build-cuda (and of course its sister opts-build-ocelot) elsewhere in the makefile.
P.S. Since build-cuda et. al. are not real files, you had better tell make this (.PHONY: build-cuda).
Related
Can I write a wrapper makefile that will cd one level up and execute there make with all the command options I have given the wrapper?
In more detail:
Directory project contains a real Makefile with some different targets.
Directory project/resources contains the wrapper Makefile which should call Makefile in project.
When I am in my shell in directory project/resources, I execute
make TARGET
and the Makefile there just cds one directory up and calls
make TARGET
in the directory project.
Is this possible? And how?
You could use a very simple Makefile for all your sub-directories:
%:
$(MAKE) -C .. $#
% is a last resort match-anything pattern rule that will match any target... for which there is no implicit rule (GNU make has an incredibly large number of implicit rules). So, if none of your targets are covered by an implicit rule, this should work. Else you will have to tell make not to use the implicit rules it knows. This can be done (with GNU make) by calling make with the -r option:
cd project/resources
make -r <anything>
will call make in project for target <anything>. The main drawback is that the -r flag is passed to the sub-make and so the implicit rules will not apply neither in project, which can be a problem. If it is you can obtain the same effect by adding an empty .SUFFIXES target to theMakefile in project/resources:
.SUFFIXES:
%:
$(MAKE) -C .. $#
With my version of GNU make (3.82) it works like a charm and the sub-make has all the default implicit rules.
Yes, you can have a makefile which works for "any" target.
The GNU make manual discusses this in the Overriding Part of Another Makefile section:
Sometimes it is useful to have a makefile that is mostly just like another makefile. You can often use the ‘include’ directive to include one in the other, and add more targets or variable definitions. However, it is invalid for two makefiles to give different recipes for the same target. But there is another way.
In the containing makefile (the one that wants to include the other), you can use a match-anything pattern rule to say that to remake any target that cannot be made from the information in the containing makefile, make should look in another makefile. See Pattern Rules, for more information on pattern rules.
For example, if you have a makefile called Makefile that says how to make the target ‘foo’ (and other targets), you can write a makefile called GNUmakefile that contains:
foo:
frobnicate > foo
%: force
#$(MAKE) -f Makefile $#
force: ;
If you say ‘make foo’, make will find GNUmakefile, read it, and see that to make foo, it needs to run the recipe ‘frobnicate > foo’. If you say ‘make bar’, make will find no way to make bar in GNUmakefile, so it will use the recipe from the pattern rule: ‘make -f Makefile bar’. If Makefile provides a rule for updating bar, make will apply the rule. And likewise for any other target that GNUmakefile does not say how to make.
The way this works is that the pattern rule has a pattern of just ‘%’, so it matches any target whatever. The rule specifies a prerequisite force, to guarantee that the recipe will be run even if the target file already exists. We give the force target an empty recipe to prevent make from searching for an implicit rule to build it—otherwise it would apply the same match-anything rule to force itself and create a prerequisite loop!
One option: use a wrapper file to execute the commands to do that. Just be sure your target make files don't include the child directory that has the wrapper, or else you can create an endless loop. For example,
clean:
pushd .. && make clean && popd
Using the comment of user Renaud Pacalet and the answer to a different question the following one-liner is as close as I could get. The whole Makefile reads:
IGNORE := $(shell $(MAKE) -C .. $(MAKECMDGOALS))
This solutions comes with a few caveats:
Command line option -B does not get passed through to the subsequent make call.
The output of the subsequently called make process (in the project directory) is not printed to stdout.
The wrapper make process reports for any given target at the end :
make: *** No rule to make target TARGET. Stop.
I'm writing my very first makefile and I'm stuck on a problem.
I have a bunch of prerequisites, of which the first one is a template that needs to be in a special position. I get to do this like so:
target : req1 req2 req3
command $(filter-out $<,$^) $# --template=$<
The thing is, sometimes I need to switch that template for another one while leaving the other prerequisites alone, so that
# Changing just the first prerequisite
target : req1b req2 req3
command $(filter-out $<,$^) $# --template=$<
I'm searching for a way to achieve this using the goal I have right now, without writing an ad-hoc explicit goal, maybe calling make with an argument or something similar, but I know too little about makefiles to get it done.
The general idea is you will want to use a variable, how you set that variable is up to you. One way is to pass a variable via the command line. Your Makefile would look like:
target : $(REQ_ONE) req2 req3
command $(filter-out $<,$^) $# --template=$<
and then do make target REQ_ONE=reg1 or make target REQ_ONE=reg1b
If you have a preferred default that you wish to use (say req1) and you want to use the alternative in rarer circumstances you could use the modified forms of the previous example.
# only set if the variable doesn't exist
REQ_ONE ?= req1
target : $(REQ_ONE) req2 req3
command $(filter-out $<,$^) $# --template=$<
Finally, a variant on this approach is to have your Makefile call make with a variable assignment:
# only set if the variable doesn't exist
REQ_ONE ?= req1
target2:
$(MAKE) target REQ_ONE=req1b
target : $(REQ_ONE) req2 req3
command $(filter-out $<,$^) $# --template=$<
Another solution is to use secondary expansion as demonstrated in this SO post on target specific variables as a prerequisites.
I am thinking I may need to use the file function in GNU make, and just can not follow the example they give. I have looked online, but don't see any post with more explanation. Here is the example they give:
program: $(OBJECTS)
$(file >$#.in,$^)
$(CMD) $(CMDFLAGS) #$#.in
#rm $#.in
I think I know what it is doing at a high level as it is explained in the manual.
$#.in
is a list of all the target files
$^
is a list of the source files
I am not sure how #$#.in is used on the third line or what there is an # sign at the beginning. What does that mean please? What does it supposed to do?
The key to the operation of that recipe is given in the prose immediately preceding it in the manual:
Many commands use the convention that an argument prefixed with an # specifies a file containing more arguments. Then you might write your recipe in this way:
program: $(OBJECTS)
$(file >$#.in,$^)
$(CMD) $(CMDFLAGS) #$#.in
#rm $#.in
$# is the target file (there is only one of those in any given recipe)
$#.in is the target file with .in added to the end of the name.
$^ is the "list" of the all the prerequisites of the target.
#$#.in is the name of the target with .in at the end and # at the start.
So the $(file ...) call in that recipe writes the list of prerequisites of the target into a file called program.in in "overwrite" mode and then passes that file name to the $(CMD) command using the #filename convention that was mentioned.
I have a Makefile that I run with multithreading (-j8 specifically).
I want to force a make clean and make all operation if I'm missing a specific marker file identifying the version compiled.
(This file should be introduced when the make is completed after the second make all.)
I can't seem to make this work properly. I either get stuck in loops or it just doesn't happen at all.
(This is part of a huge system so I can't just change any paradigms and I have to work with what I have)
Here's the relevant section I have so far. This wasn't the original plan but I shifted so many things around this is the current situation:
VERSION = 2.8
.DEFAULT_GOAL := all
.PHONY : all
all : {some targets} | marker_file
###########################
.PHONY : marker_file
marker_file : build/$(VERSION).marker
.PHONY : check_marker
check_marker :
ifeq (,$(wildcard build/$(VERSION).marker))
#echo -e "\e[41mYOU ARE ON NEW PREREQUISITES $(VERSION)! FORCING MAKE CLEAN BEFORE REBUILDING\e[0m"
$(MAKE) clean
#mkdir -p build
#touch build/$(VERSION).marker
$(MAKE) $(MAKECMDGOALS)
endif
# if the marker file needs generation, force clean and rebuild
build/$(VERSION).marker : check_marker
Can anyone figure out how to properly plan the rules and dependencies so that I can generate the file on the second time?
You definitely don't want to use order-only prerequisites. That forces the prerequisite to always run, but doesn't use the results in determining whether to run the target. That's almost the exact opposite of what you want.
Also you cannot use make preprocessor constructs like ifeq inside a recipe (indented by a TAB). Recipes are passed to the shell, and the shell is not make and does not understand make constructs like ifeq.
You can use make's auto-re-exec feature: if an included file changes then make will re-exec itself. So:
VERSION = 2.8
.DEFAULT_GOAL := all
.PHONY : all
all : {some targets}
###########################
MARKER_FILE = build/$(VERSION).marker
$(MARKER_FILE) :
#echo -e "\e[41mYOU ARE ON NEW PREREQUISITES $(VERSION)! FORCING MAKE CLEAN BEFORE REBUILDING\e[0m"
$(MAKE) clean MARKER_FILE=
#mkdir -p $(#D)
#touch $#
include $(MARKER_FILE)
When running a CMake generated Makefile with multiple processes (make -jN), the output often gets messed up like this:
[ 8%] [ 8%] [ 9%] Building CXX object App/CMakeFiles/App.dir/src/File1.cpp.o
Building CXX object App/CMakeFiles/App.dir/src/File2.cpp.o
Building CXX object App/CMakeFiles/App.dir/src/File3.cpp.o
I'm not sure, but I think this behavior is also there for Makefiles not generated by CMake. I'd say it happens when multiple processes write to stdout at the same time.
I know I'm probably being pedantic, but is there any (simple) fix to this? ;)
If you're using GNU make, you can do it by redefining SHELL such that commands are wrapped by a trivial utility that ensures atomicity of information printed to standard output. Here's a more detailed description, including sample source for the wrapper utility.
I tried to get the CMake people to fix this, but apparently they don't want to. See http://www.cmake.org/Bug/view.php?id=7062.
The specific CMake bug related to interleaved make output using -jN with N>1 is CMake bug 0012991: "Parallel build output mess". It is still open in the "backlog" state waiting to be fixed.
This bug is actually annoying enough that it's a strong reason to switch to Ninja instead of make. Plus the fact that Ninja is faster than make. Ninja also uses an appropriate number of parallel jobs based on the number of CPU cores present. Also cool is how Ninja is, by default, very quiet: all progress happens on a single line in the terminal unless the build process emits messages or a build step fails. If a build step fails, Ninja prints the full command line that invoked it and displays the output. It's really nice since it makes any warning or error messages stand out. Although currently there is no colored terminal output: that would be a nice improvement but for me the advantages of Ninja over make are tremendous.
Looks like it is already fixed. Add a -Oline parameter to the command line:
make -j 8 -Oline
Version of make:
GNU Make 4.3
Built for x86_64-pc-msys
Sun's (now Oracle's) dmake available on Linux and Solaris takes care of that.
See here and there.
Here is a simple working example of using a wrapper for Make. I'm not sure if I'd encourage it's use, but it's an idea.
# Makefile
SHELL = /tmp/test/wrapper
test: test1 test2
test1:
$(eval export TARGET=$#)
env
test2:
$(eval export TARGET=$#)
env
and this:
#!/usr/bin/env bash
# wrapper
bash $# | sed -e "s/^/${TARGET} /"