I have makefile structured in the following way.
.PHONY: DEPTARG1 DEPTARG2
$(info BUILD_AGAIN is $(BUILD_AGAIN))
TARGET: DEPTARG1 DEPTARG2
ifeq ($(BUILD_AGAIN),y)
DEPTARG1:
#echo Building $#
DEPTARG2:
#echo Building $#
endif
I anticipated this makefile to throw an error if BUILD-AGAIN is n, since it wont have the dependencies available.But It didn't. Any thoughts for it are welcome.
As an additional information DETARG1 and DEPTARG2 are phony targets.
Output when BUILD_AGAIN is y
BUILD_AGAIN is y
Building DEPTARG1
Building DEPTARG2
Output when BUILD_AGAIN is n
BUILD_AGAIN is n
make: Nothing to be done for `TARGET'.
Moving the .PHONY: declaration inside your conditional would eliminate this error.
That being said you seem to be going through a lot of shenanigans to re-create something that make has built in. Writing up or not writing up recipes is no the way to run or not run them. Make has a flag for that, make -B will force a rebuild even if the targets are up to date. Replacing the whole setting a BUILD_AGAIN variable shenanigans just running with or without that flag should do the trick.
Related
I am trying to write a recursive make recipe. In this recipe, each target is dependent on a file with an equal name on the parent directory. A minimal (non-working) example:
foo/.dirstamp:
mkdir $(dir $#)
touch $#
.SECONDEXPANSION:
%/.dirstamp: $$(dir $$*).dirstamp
mkdir $(dir $#)
touch $#
With this example, I would expect make foo/bar/qux/lol/.dirstamp to generate the whole directory tree (if it does not exist), touching all .dirstamp files along the way. However, it does not work:
$ ls # note that there is nothing, make is meant to create the dir tree
Makefile
$ make --debug=v foo/bar/qux/lol/.dirstamp
GNU Make 4.0
[...]
Reading makefiles...
Reading makefile 'Makefile'...
Updating goal targets....
Considering target file 'foo/bar/qux/lol/.dirstamp'.
File 'foo/bar/qux/lol/.dirstamp' does not exist.
Finished prerequisites of target file 'foo/bar/qux/lol/.dirstamp'.
Must remake target 'foo/bar/qux/lol/.dirstamp'.
make: *** No rule to make target 'foo/bar/qux/lol/.dirstamp'. Stop.
It works fine as long as the recursive recipe only needs to be expanded twice, e.g., make foo/bar/.dirstamp works fine.
How can this work for an arbitrary number of levels? How can I handle a recursive expansion for the target and prerequisites names?
Note: my real problem is that the prerequisites of my recipes are in a root
directory different from the target so I am using the recipe above to duplicate the directory tree. I know about mkdir -p which seems to work fine in GNU systems. I am still interested on knowing how I would solve the recursion problem for arbitrary levels. which no longer works because part of the team is using Mac and mounting this directories over smb.
More details on the actual problem: prerequisites are in data/x/y/z while targets go into results/x/y/z. However, the results directory tree does not exist and needs to be created as needed. To solve this, I made the creation of parent directories an order-only prerequisite (via the .dirstamp files on my minimal example above).
can't copy data into results, that's several TB of data;
can't have the targets created in data, that's read-only;
can't use mkdir -p because the results directory will not be local, mounted over smb, and others may use non-GNU systems;
After an hint from #EtanReisner on the question:
make won't apply a rule more than once. That's a built-in (intentional) limitation. Without working around that with manual recursion or manually building the set of targets and using a static pattern rule (which may or may not actually work I'm not sure) there's not much you can do about this.
I worked up this solution:
RESULT_DIRS := $(patsubst data/%, results/%, $(shell find data/* -type d -print))
DIRSTAMPS := $(addsuffix /.dirstamp, $(RESULT_DIRS))
results/.dirstamp:
mkdir $(dir $#)
touch $#
.SECONDEXPANSION:
$(DIRSTAMPS): $$(dir $$(patsubst %/.dirstamp, %, $$#)).dirstamp
mkdir $(dir $#)
touch $#
It will duplicate the data directory tree in results as the dirstamp files are required. They are required by making them prerequisites of the other recipes (note the | which makes them order-only prerequisites):
results/%/foo.analysis: data/%/foo.data | results/%/.dirstamp
$(SOME_ANALYSIS_PROGRAM) $^ > $#
Can I write a wrapper makefile that will cd one level up and execute there make with all the command options I have given the wrapper?
In more detail:
Directory project contains a real Makefile with some different targets.
Directory project/resources contains the wrapper Makefile which should call Makefile in project.
When I am in my shell in directory project/resources, I execute
make TARGET
and the Makefile there just cds one directory up and calls
make TARGET
in the directory project.
Is this possible? And how?
You could use a very simple Makefile for all your sub-directories:
%:
$(MAKE) -C .. $#
% is a last resort match-anything pattern rule that will match any target... for which there is no implicit rule (GNU make has an incredibly large number of implicit rules). So, if none of your targets are covered by an implicit rule, this should work. Else you will have to tell make not to use the implicit rules it knows. This can be done (with GNU make) by calling make with the -r option:
cd project/resources
make -r <anything>
will call make in project for target <anything>. The main drawback is that the -r flag is passed to the sub-make and so the implicit rules will not apply neither in project, which can be a problem. If it is you can obtain the same effect by adding an empty .SUFFIXES target to theMakefile in project/resources:
.SUFFIXES:
%:
$(MAKE) -C .. $#
With my version of GNU make (3.82) it works like a charm and the sub-make has all the default implicit rules.
Yes, you can have a makefile which works for "any" target.
The GNU make manual discusses this in the Overriding Part of Another Makefile section:
Sometimes it is useful to have a makefile that is mostly just like another makefile. You can often use the ‘include’ directive to include one in the other, and add more targets or variable definitions. However, it is invalid for two makefiles to give different recipes for the same target. But there is another way.
In the containing makefile (the one that wants to include the other), you can use a match-anything pattern rule to say that to remake any target that cannot be made from the information in the containing makefile, make should look in another makefile. See Pattern Rules, for more information on pattern rules.
For example, if you have a makefile called Makefile that says how to make the target ‘foo’ (and other targets), you can write a makefile called GNUmakefile that contains:
foo:
frobnicate > foo
%: force
#$(MAKE) -f Makefile $#
force: ;
If you say ‘make foo’, make will find GNUmakefile, read it, and see that to make foo, it needs to run the recipe ‘frobnicate > foo’. If you say ‘make bar’, make will find no way to make bar in GNUmakefile, so it will use the recipe from the pattern rule: ‘make -f Makefile bar’. If Makefile provides a rule for updating bar, make will apply the rule. And likewise for any other target that GNUmakefile does not say how to make.
The way this works is that the pattern rule has a pattern of just ‘%’, so it matches any target whatever. The rule specifies a prerequisite force, to guarantee that the recipe will be run even if the target file already exists. We give the force target an empty recipe to prevent make from searching for an implicit rule to build it—otherwise it would apply the same match-anything rule to force itself and create a prerequisite loop!
One option: use a wrapper file to execute the commands to do that. Just be sure your target make files don't include the child directory that has the wrapper, or else you can create an endless loop. For example,
clean:
pushd .. && make clean && popd
Using the comment of user Renaud Pacalet and the answer to a different question the following one-liner is as close as I could get. The whole Makefile reads:
IGNORE := $(shell $(MAKE) -C .. $(MAKECMDGOALS))
This solutions comes with a few caveats:
Command line option -B does not get passed through to the subsequent make call.
The output of the subsequently called make process (in the project directory) is not printed to stdout.
The wrapper make process reports for any given target at the end :
make: *** No rule to make target TARGET. Stop.
I have a Makefile that I run with multithreading (-j8 specifically).
I want to force a make clean and make all operation if I'm missing a specific marker file identifying the version compiled.
(This file should be introduced when the make is completed after the second make all.)
I can't seem to make this work properly. I either get stuck in loops or it just doesn't happen at all.
(This is part of a huge system so I can't just change any paradigms and I have to work with what I have)
Here's the relevant section I have so far. This wasn't the original plan but I shifted so many things around this is the current situation:
VERSION = 2.8
.DEFAULT_GOAL := all
.PHONY : all
all : {some targets} | marker_file
###########################
.PHONY : marker_file
marker_file : build/$(VERSION).marker
.PHONY : check_marker
check_marker :
ifeq (,$(wildcard build/$(VERSION).marker))
#echo -e "\e[41mYOU ARE ON NEW PREREQUISITES $(VERSION)! FORCING MAKE CLEAN BEFORE REBUILDING\e[0m"
$(MAKE) clean
#mkdir -p build
#touch build/$(VERSION).marker
$(MAKE) $(MAKECMDGOALS)
endif
# if the marker file needs generation, force clean and rebuild
build/$(VERSION).marker : check_marker
Can anyone figure out how to properly plan the rules and dependencies so that I can generate the file on the second time?
You definitely don't want to use order-only prerequisites. That forces the prerequisite to always run, but doesn't use the results in determining whether to run the target. That's almost the exact opposite of what you want.
Also you cannot use make preprocessor constructs like ifeq inside a recipe (indented by a TAB). Recipes are passed to the shell, and the shell is not make and does not understand make constructs like ifeq.
You can use make's auto-re-exec feature: if an included file changes then make will re-exec itself. So:
VERSION = 2.8
.DEFAULT_GOAL := all
.PHONY : all
all : {some targets}
###########################
MARKER_FILE = build/$(VERSION).marker
$(MARKER_FILE) :
#echo -e "\e[41mYOU ARE ON NEW PREREQUISITES $(VERSION)! FORCING MAKE CLEAN BEFORE REBUILDING\e[0m"
$(MAKE) clean MARKER_FILE=
#mkdir -p $(#D)
#touch $#
include $(MARKER_FILE)
I would like to have a makefile like this:
cudaLib :
# Create shared library with nvcc
ocelotLib :
# Create shared library for gpuocelot
build-cuda : cudaLib
make build
build-ocelot : ocelotLib
make build
build :
# build and link with the shared library
I.e. the *Lib tasks create a library that runs cuda directly on the device, or on gpuocelot respectively.
For both build tasks I need to run the same build steps, only creating the library differs.
Is there an alternative to running make directly?
make build
Kind of a post-requisite?
Note: This answer focuses on the aspect of a robust recursive invocation of a different target in a given makefile.
To complement Jack Kelly's helpful answer, here's a GNU makefile snippet that demonstrates the use of $(MAKE) to robustly invoke a different target in the same makefile (ensuring that the same make binary is called, and that the same makefile is targeted):
# Determine this makefile's path.
# Be sure to place this BEFORE `include` directives, if any.
THIS_FILE := $(lastword $(MAKEFILE_LIST))
target:
#echo $# # print target name
#$(MAKE) -f $(THIS_FILE) other-target # invoke other target
other-target:
#echo $# # print target name
Output:
$ make target
target
other-target
Using $(lastword $(MAKEFILE_LIST)) and -f ... ensures that the $(MAKE) command uses the same makefile, even if that makefile was passed with an explicit path (-f ...) when make was originally invoked.
Note: While GNU make does have features for recursive invocations - for instance, variable $(MAKE) specifically exists to enable them - their focus is on invoking subordinate makefiles, not on calling a different target in the same makefile.
That said, even though the workaround above is somewhat cumbersome and obscure, it does use regular features and should be robust.
Here is the link to the manual section covering recursive invocations ("sub-makes"):
Recursive Use of make
Most versions of make set a variable $(MAKE) that you can use for recursive invocations.
As you have written it, the build target will need to do something different depending on whether you have just done an ocelot or cuda build. That's another way of saying you have to parameterise build in some way. I suggest separate build targets (much like you already have), with associated variables. Something like:
build-cuda: cudaLib
build-ocelot: ocelotLib
build-cuda build-ocelot:
shell commands
which invoke ${opts-$#}
On the command-line you type make build-cuda (say). Make first builds cudaLib, then it carries out the recipe for build-cuda. It expands the macros before calling the shell. $# in this case is build-cuda, thus ${opts-$#} is first expanded to ${opts-build-cuda}. Make now goes on to expand ${opts-build-cuda}. You will have defined opts-build-cuda (and of course its sister opts-build-ocelot) elsewhere in the makefile.
P.S. Since build-cuda et. al. are not real files, you had better tell make this (.PHONY: build-cuda).
When running a CMake generated Makefile with multiple processes (make -jN), the output often gets messed up like this:
[ 8%] [ 8%] [ 9%] Building CXX object App/CMakeFiles/App.dir/src/File1.cpp.o
Building CXX object App/CMakeFiles/App.dir/src/File2.cpp.o
Building CXX object App/CMakeFiles/App.dir/src/File3.cpp.o
I'm not sure, but I think this behavior is also there for Makefiles not generated by CMake. I'd say it happens when multiple processes write to stdout at the same time.
I know I'm probably being pedantic, but is there any (simple) fix to this? ;)
If you're using GNU make, you can do it by redefining SHELL such that commands are wrapped by a trivial utility that ensures atomicity of information printed to standard output. Here's a more detailed description, including sample source for the wrapper utility.
I tried to get the CMake people to fix this, but apparently they don't want to. See http://www.cmake.org/Bug/view.php?id=7062.
The specific CMake bug related to interleaved make output using -jN with N>1 is CMake bug 0012991: "Parallel build output mess". It is still open in the "backlog" state waiting to be fixed.
This bug is actually annoying enough that it's a strong reason to switch to Ninja instead of make. Plus the fact that Ninja is faster than make. Ninja also uses an appropriate number of parallel jobs based on the number of CPU cores present. Also cool is how Ninja is, by default, very quiet: all progress happens on a single line in the terminal unless the build process emits messages or a build step fails. If a build step fails, Ninja prints the full command line that invoked it and displays the output. It's really nice since it makes any warning or error messages stand out. Although currently there is no colored terminal output: that would be a nice improvement but for me the advantages of Ninja over make are tremendous.
Looks like it is already fixed. Add a -Oline parameter to the command line:
make -j 8 -Oline
Version of make:
GNU Make 4.3
Built for x86_64-pc-msys
Sun's (now Oracle's) dmake available on Linux and Solaris takes care of that.
See here and there.
Here is a simple working example of using a wrapper for Make. I'm not sure if I'd encourage it's use, but it's an idea.
# Makefile
SHELL = /tmp/test/wrapper
test: test1 test2
test1:
$(eval export TARGET=$#)
env
test2:
$(eval export TARGET=$#)
env
and this:
#!/usr/bin/env bash
# wrapper
bash $# | sed -e "s/^/${TARGET} /"