I'm expecting to see files foo1 and foo3 created by the makefile below. However only a file foo3 is created. To me it seems that the canned recipe make-foo is simply ignored by make. The debug outcome of targets foo1 and foo2 (empty recipe) is identical.
# why canned recipe doesn't work ?
# http://www.gnu.org/software/make/manual/make.html#Canned-Recipes
define make-foo =
echo making $#
touch $#
endef
.PHONY: all
all: foo1 foo2 foo3
# foo1 is not created, but why ?
.PHONY: foo1
foo1:
$(make-foo)
# debug output similar to foo1
.PHONY: foo2
foo2:
# this works
.PHONY: foo3
foo3:
echo making $#
touch $#
Running make:
xxxx#xxxx:/dev/shm$ make -dRr
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for i686-pc-linux-gnu
Reading makefiles...
Reading makefile `makefile'...
Updating makefiles....
Considering target file `makefile'.
Looking for an implicit rule for `makefile'.
No implicit rule found for `makefile'.
Finished prerequisites of target file `makefile'.
No need to remake target `makefile'.
Updating goal targets....
Considering target file `all'.
File `all' does not exist.
Considering target file `foo1'.
File `foo1' does not exist.
Finished prerequisites of target file `foo1'.
Must remake target `foo1'.
Successfully remade target file `foo1'.
Considering target file `foo2'.
File `foo2' does not exist.
Finished prerequisites of target file `foo2'.
Must remake target `foo2'.
Successfully remade target file `foo2'.
Considering target file `foo3'.
File `foo3' does not exist.
Finished prerequisites of target file `foo3'.
Must remake target `foo3'.
echo making foo3
Putting child 0x0914c5f0 (foo3) PID 3132 on the chain.
Live child 0x0914c5f0 (foo3) PID 3132
making foo3
Reaping winning child 0x0914c5f0 PID 3132
touch foo3
Live child 0x0914c5f0 (foo3) PID 3133
Reaping winning child 0x0914c5f0 PID 3133
Removing child 0x0914c5f0 PID 3133 from chain.
Successfully remade target file `foo3'.
Finished prerequisites of target file `all'.
Must remake target `all'.
Successfully remade target file `all'.
Missing foo1:
xxxx#xxxx:/dev/shm$ ll foo*
-rw-r--r-- 1 xxxx xxxx 0 2011-02-17 20:04 foo3
I think you don't want the = at the end of the define line. This makefile works here for me:
define make-foo
echo making $#
touch $#
endef
.PHONY: foo1
foo1:
$(make-foo)
Example:
$ make
echo making foo1
making foo1
touch foo1
$ ls
Makefile foo1
The GNU make manual seems to indicate that the = should be just fine, but like you, I get different behaviour if I have it there.
Edit: I just asked:
GNU make differences in multiline variable declarations
To get some clarification on what's happening here...
Related
For clarity, I am running this on windows with GnuWin32 make.
I have a set of directories with markdown files in at several different levels - theoretically they could be in the branch nodes, but I think currently they are only in the leaf nodes. I have a set of pandoc/LaTeX commands to run to turn the markdown files into PDFs - and obviously only want to recreate the PDFs if the markdown file has been updated, so a makefile seems appropriate.
What I would like is a single makefile in the root, which iterates over any and all sub-directories (to any depth) and applies the make rule I'll specify for running pandoc.
From what I've been able to find, recursive makefiles require you to have a makefile in each sub-directory (which seems like an administrative overhead that I would like to avoid) and/or require you to list out all the sub-directories at the start of the makefile (again, would prefer to avoid this).
Theoretical folder structure:
root
|-make
|-Folder AB
| |-File1.md
| \-File2.md
|-Folder C
| \-File3.md
\-Folder D
|-Folder E
| \-File4.md
|-Folder F
\-File5.md
How do I write a makefile to deal with this situation?
Here is a small set of Makefile rules that hopefuly would get you going
%.pdf : %.md
pandoc -o $# --pdf-engine=xelatex $^
PDF_FILES=FolderA/File1.pdf FolderA/File2.pdf \
FolderC/File3.pdf FolderD/FolderE/File4.pdf FolderD/FolderF/File5.pdf
all: ${PDF_FILES}
Let me explain what is going on here. First we have a pattern rule that tells make how to convert a Markdown file to a PDF file. The --pdf-engine=xelatex option is here just for the purpose of illustration.
Then we need to tell Make which files to consider. We put the names together in a single variable PDF_FILES. This value for this variable can be build via a separate scripts that scans all subdirectories for .md files.
Note that one has to be extra careful if filenames or directory names contain spaces.
Then we ask Make to check if any of the PDF_FILES should be updated.
If you have other targets in your makefile, make sure that all is the first non-pattern target, or call make as make all
Updating the Makefile
If shell functions works for you and basic utilities such as sed and find are available, you could make your makefile dynamic with a single line.
%.pdf : %.md
pandoc -o $# --pdf-engine=xelatex $^
PDF_FILES:=$(shell find -name "*.md" | xargs echo | sed 's/\.md/\.pdf/g' )
all: ${PDF_FILES}
MadScientist suggested just that in the comments
Otherwise you could implement a script using the tools available on your operating system and add an additional target update: that would compute the list of files and replace the line starting with PDF_FILES with an updated list of files.
Final version of the code that worked for Windows, based on #DmitiChubarov and #MadScientist's suggestions is as follows:
%.pdf: %.md
pandoc $^ -o $#
PDF_FILES:=$(shell dir /s /b *.md | sed "s/\.md/\.pdf/g")
all: ${PDF_FILES}
Say I have a list of source files like this:
SOURCE=aaa.src bbb.src ccc.src
I want to create a rule to build each corresponding target in its own folder, the name of which depends on the soruce file.
Sourcefile ----> Corresponding output file / target
aaa.src -------------------> aaa/aaa.out
bbb.src -------------------> bbb/bbb.out
ccc.src -------------------> ccc/ccc.out
How do I write a rule for this using GNUMake? My best effort was the following Makefile:
.PHONY: all clean
CC=somecompiler
SOURCE := aaa.src bbb.src ccc.src
RTARGS := $(SOURCE:%.src=%.out)
TDIRS := $(addsuffix /,$(basename $(SOURCE)))
TARGS := $(join $(TDIRS), $(RTARGS))
all:$(TARGS)
%.out: $(SOURCE) # Better source specification?
#[ ! -d "$(dir $*)" ] && mkdir "$(dir $*)"
$(CC) "$(patsubst %/,%,$(dir $*)).src" "$#"
clean:
rm -f $(TARGS)
rmdir --ignore-fail-on-non-empty $(TDIRS)
The problem here is, that any one target (i.e. .out file) depends on every source (i.e. .src file), instead of just the one that has the same basename. I could change the commented line to %.out: %.src but than the source file would have to be in the same directory as the output file. I could also compile it something like this:
%.out: %.src
$(CC) "$>" -o "$*/$#"
But then make would always compile every target regardless of whether it already exists or not.
How do I get make to use the appropriate source file only. What is the proper way of specifying dependences for each target separately in a generic rule?
In GNU Make you can specify prerequisites separately from recipes and still have a generic / pattern rule for all .out files:
.PHONY: all clean
all :
SOURCES := aaa.src bbb.src ccc.src
OUTPUTS := $(foreach src,${SOURCES},$(basename ${src})/${src:%.src=%.out})
# Build dependencies in the form of x/x.out : x.src | x
define ESTABLISH_DEPENDENCY =
$(basename ${1})/${1:%.src=%.out} : ${1} | $(basename ${1}) # Also depend on the output directory.
$(basename ${1}) : # Rule to create the output directory.
mkdir $$#
endef
$(foreach src,${SOURCES},$(eval $(call ESTABLISH_DEPENDENCY,${src})))
all : ${OUTPUTS}
# Generice rule for all .out files.
%.out :
#echo "build $# from $^"
touch $#
%.src : # For debugging only.
touch $#
.PHONY: all
Output:
$ make
touch aaa.src
mkdir aaa
build aaa/aaa.out from aaa.src
touch aaa/aaa.out
touch bbb.src
mkdir bbb
build bbb/bbb.out from bbb.src
touch bbb/bbb.out
touch ccc.src
mkdir ccc
build ccc/ccc.out from ccc.src
touch ccc/ccc.out
Note that it is more efficient and elegant to have order-only dependencies on directories and let make create them, than to have each recipe checking directory existence first.
I am trying to write a recursive make recipe. In this recipe, each target is dependent on a file with an equal name on the parent directory. A minimal (non-working) example:
foo/.dirstamp:
mkdir $(dir $#)
touch $#
.SECONDEXPANSION:
%/.dirstamp: $$(dir $$*).dirstamp
mkdir $(dir $#)
touch $#
With this example, I would expect make foo/bar/qux/lol/.dirstamp to generate the whole directory tree (if it does not exist), touching all .dirstamp files along the way. However, it does not work:
$ ls # note that there is nothing, make is meant to create the dir tree
Makefile
$ make --debug=v foo/bar/qux/lol/.dirstamp
GNU Make 4.0
[...]
Reading makefiles...
Reading makefile 'Makefile'...
Updating goal targets....
Considering target file 'foo/bar/qux/lol/.dirstamp'.
File 'foo/bar/qux/lol/.dirstamp' does not exist.
Finished prerequisites of target file 'foo/bar/qux/lol/.dirstamp'.
Must remake target 'foo/bar/qux/lol/.dirstamp'.
make: *** No rule to make target 'foo/bar/qux/lol/.dirstamp'. Stop.
It works fine as long as the recursive recipe only needs to be expanded twice, e.g., make foo/bar/.dirstamp works fine.
How can this work for an arbitrary number of levels? How can I handle a recursive expansion for the target and prerequisites names?
Note: my real problem is that the prerequisites of my recipes are in a root
directory different from the target so I am using the recipe above to duplicate the directory tree. I know about mkdir -p which seems to work fine in GNU systems. I am still interested on knowing how I would solve the recursion problem for arbitrary levels. which no longer works because part of the team is using Mac and mounting this directories over smb.
More details on the actual problem: prerequisites are in data/x/y/z while targets go into results/x/y/z. However, the results directory tree does not exist and needs to be created as needed. To solve this, I made the creation of parent directories an order-only prerequisite (via the .dirstamp files on my minimal example above).
can't copy data into results, that's several TB of data;
can't have the targets created in data, that's read-only;
can't use mkdir -p because the results directory will not be local, mounted over smb, and others may use non-GNU systems;
After an hint from #EtanReisner on the question:
make won't apply a rule more than once. That's a built-in (intentional) limitation. Without working around that with manual recursion or manually building the set of targets and using a static pattern rule (which may or may not actually work I'm not sure) there's not much you can do about this.
I worked up this solution:
RESULT_DIRS := $(patsubst data/%, results/%, $(shell find data/* -type d -print))
DIRSTAMPS := $(addsuffix /.dirstamp, $(RESULT_DIRS))
results/.dirstamp:
mkdir $(dir $#)
touch $#
.SECONDEXPANSION:
$(DIRSTAMPS): $$(dir $$(patsubst %/.dirstamp, %, $$#)).dirstamp
mkdir $(dir $#)
touch $#
It will duplicate the data directory tree in results as the dirstamp files are required. They are required by making them prerequisites of the other recipes (note the | which makes them order-only prerequisites):
results/%/foo.analysis: data/%/foo.data | results/%/.dirstamp
$(SOME_ANALYSIS_PROGRAM) $^ > $#
Can I write a wrapper makefile that will cd one level up and execute there make with all the command options I have given the wrapper?
In more detail:
Directory project contains a real Makefile with some different targets.
Directory project/resources contains the wrapper Makefile which should call Makefile in project.
When I am in my shell in directory project/resources, I execute
make TARGET
and the Makefile there just cds one directory up and calls
make TARGET
in the directory project.
Is this possible? And how?
You could use a very simple Makefile for all your sub-directories:
%:
$(MAKE) -C .. $#
% is a last resort match-anything pattern rule that will match any target... for which there is no implicit rule (GNU make has an incredibly large number of implicit rules). So, if none of your targets are covered by an implicit rule, this should work. Else you will have to tell make not to use the implicit rules it knows. This can be done (with GNU make) by calling make with the -r option:
cd project/resources
make -r <anything>
will call make in project for target <anything>. The main drawback is that the -r flag is passed to the sub-make and so the implicit rules will not apply neither in project, which can be a problem. If it is you can obtain the same effect by adding an empty .SUFFIXES target to theMakefile in project/resources:
.SUFFIXES:
%:
$(MAKE) -C .. $#
With my version of GNU make (3.82) it works like a charm and the sub-make has all the default implicit rules.
Yes, you can have a makefile which works for "any" target.
The GNU make manual discusses this in the Overriding Part of Another Makefile section:
Sometimes it is useful to have a makefile that is mostly just like another makefile. You can often use the ‘include’ directive to include one in the other, and add more targets or variable definitions. However, it is invalid for two makefiles to give different recipes for the same target. But there is another way.
In the containing makefile (the one that wants to include the other), you can use a match-anything pattern rule to say that to remake any target that cannot be made from the information in the containing makefile, make should look in another makefile. See Pattern Rules, for more information on pattern rules.
For example, if you have a makefile called Makefile that says how to make the target ‘foo’ (and other targets), you can write a makefile called GNUmakefile that contains:
foo:
frobnicate > foo
%: force
#$(MAKE) -f Makefile $#
force: ;
If you say ‘make foo’, make will find GNUmakefile, read it, and see that to make foo, it needs to run the recipe ‘frobnicate > foo’. If you say ‘make bar’, make will find no way to make bar in GNUmakefile, so it will use the recipe from the pattern rule: ‘make -f Makefile bar’. If Makefile provides a rule for updating bar, make will apply the rule. And likewise for any other target that GNUmakefile does not say how to make.
The way this works is that the pattern rule has a pattern of just ‘%’, so it matches any target whatever. The rule specifies a prerequisite force, to guarantee that the recipe will be run even if the target file already exists. We give the force target an empty recipe to prevent make from searching for an implicit rule to build it—otherwise it would apply the same match-anything rule to force itself and create a prerequisite loop!
One option: use a wrapper file to execute the commands to do that. Just be sure your target make files don't include the child directory that has the wrapper, or else you can create an endless loop. For example,
clean:
pushd .. && make clean && popd
Using the comment of user Renaud Pacalet and the answer to a different question the following one-liner is as close as I could get. The whole Makefile reads:
IGNORE := $(shell $(MAKE) -C .. $(MAKECMDGOALS))
This solutions comes with a few caveats:
Command line option -B does not get passed through to the subsequent make call.
The output of the subsequently called make process (in the project directory) is not printed to stdout.
The wrapper make process reports for any given target at the end :
make: *** No rule to make target TARGET. Stop.
I have a Makefile that I run with multithreading (-j8 specifically).
I want to force a make clean and make all operation if I'm missing a specific marker file identifying the version compiled.
(This file should be introduced when the make is completed after the second make all.)
I can't seem to make this work properly. I either get stuck in loops or it just doesn't happen at all.
(This is part of a huge system so I can't just change any paradigms and I have to work with what I have)
Here's the relevant section I have so far. This wasn't the original plan but I shifted so many things around this is the current situation:
VERSION = 2.8
.DEFAULT_GOAL := all
.PHONY : all
all : {some targets} | marker_file
###########################
.PHONY : marker_file
marker_file : build/$(VERSION).marker
.PHONY : check_marker
check_marker :
ifeq (,$(wildcard build/$(VERSION).marker))
#echo -e "\e[41mYOU ARE ON NEW PREREQUISITES $(VERSION)! FORCING MAKE CLEAN BEFORE REBUILDING\e[0m"
$(MAKE) clean
#mkdir -p build
#touch build/$(VERSION).marker
$(MAKE) $(MAKECMDGOALS)
endif
# if the marker file needs generation, force clean and rebuild
build/$(VERSION).marker : check_marker
Can anyone figure out how to properly plan the rules and dependencies so that I can generate the file on the second time?
You definitely don't want to use order-only prerequisites. That forces the prerequisite to always run, but doesn't use the results in determining whether to run the target. That's almost the exact opposite of what you want.
Also you cannot use make preprocessor constructs like ifeq inside a recipe (indented by a TAB). Recipes are passed to the shell, and the shell is not make and does not understand make constructs like ifeq.
You can use make's auto-re-exec feature: if an included file changes then make will re-exec itself. So:
VERSION = 2.8
.DEFAULT_GOAL := all
.PHONY : all
all : {some targets}
###########################
MARKER_FILE = build/$(VERSION).marker
$(MARKER_FILE) :
#echo -e "\e[41mYOU ARE ON NEW PREREQUISITES $(VERSION)! FORCING MAKE CLEAN BEFORE REBUILDING\e[0m"
$(MAKE) clean MARKER_FILE=
#mkdir -p $(#D)
#touch $#
include $(MARKER_FILE)