I have a number of latex files, and I want to be able to compile them all conveniently. My makefile is currently as follows.
targets := $(patsubst %.tex, %.pdf, $(wildcard *.tex))
.PHONY: all
all: $(targets)
.SECONDEXPANSION:
%.pdf: %.tex $$(wildcard $$*_inc/*.*)
pdflatex $<
This way, if an include folder exists, everything within that folder will be treated as a dependency.
However, I also want to automate the running of dependent python scripts. These might be the creation of plots or the calculation of values to be included. I attempted to make the following modification.
targets := $(patsubst %.tex, %.pdf, $(wildcard *.tex))
all: $(targets)
.PHONY: all run_%
.SECONDEXPANSION:
%.pdf: %.tex $$(wildcard $$*_inc/*.*) $$(patsubst %, run_%, $$(wildcard $$*_inc/*.py))
pdflatex $<
run_%: %
$<
However, this does not work as I would expect to. I have the following folder structure.
|
|-Makefile
|-paper.tex
|-paper_inc/
|---a.txt
|---b.py
I would expect this to run b.py, then run pdflatex paper.tex. However, b.py is not run. If I look at the dependencies created, I see that paper_inc/a.txt and paper_inc/b.py are dependencies of paper.pdf, but run_paper_inc/b.py is not.
I'm not entirely certain what is the issue with this, as it feels that it should work. In addition, I tried the following lines as well for the %.pdf rule.
%.pdf: %.tex $$(wildcard $$*_inc/*.*) $$(addprefix run_,$$(wildcard $$*_inc/*.py))
%.pdf: %.tex $$(wildcard $$*_inc/*.*) $$(shell ls $$*_inc/*.py | sed 's/^/run/')
%.pdf: %.tex $$(wildcard $$*_inc/*.*) $$(shell ls $$*_inc/*.py | awk '{print "prefix "$0}')
With each of these rules, though, I receive the error message make: *** No rule to make target 'paper.pdf'. Stop.
What should be be done to fix this error, or, alternatively, what should I use instead of make?
You have two problems.
First:
run_%: %
$<
From the manual:
When the target pattern does not contain a slash (and it usually does
not), directory names in the file names are removed from the file name
before it is compared with the target prefix and suffix. After the
comparison of the file name to the target pattern, the directory
names, along with the slash that ends them, are added on to the
prerequisite file names generated from the pattern rule's prerequisite
patterns and the file name.
This is a little hard to follow, but basically when Make tries to build run_paper_inc/b.py, it looks for a pattern rule that matches b.py. The rule above doesn't fit. But this does:
%.py:
$(subst run_,,$#)
Second:
.SECONDEXPANSION:
%.pdf: ... $$(patsubst %, run_%, $$(wildcard $$*_inc/*.py))
...
One of Make's big shortcomings is its weak handling of wildcards. In this case % can't work for the pattern rule and the patsubst at the same time. But there's a way:
.SECONDEXPANSION:
%.pdf: ... $$(addprefix run_,$$(wildcard %_inc/*.py))
...
Related
i have my source files inside the src folder and when i run make (windows) i get the following error
make: *** No rule to make target '%.o', needed by 'Program.exe'. Stop.
VPATH := src
Program.exe : %.o
g++.exe -o bin/program.exe $<
%.o : %.cpp
echo $<
g++ -c -ILibraries/include -LLibraries/lib $< -lgdi32
The % in make is not a wildcard that matches files. Just as well, because when make starts there won't be any .o files to match.
The % in make is a pattern match and it only works in pattern rules: pattern rules must have a % in the target (like the second rule you have, %.o : %.c). If you don't have a % in the target, then make just thinks that the % in the prerequisite list is a normal character like an a or b or whatever. Make doesn't know how to create a file named, literally, %.o because there is no matching %.cpp file.
Also it's always wrong in make to create a file that is not equal to $#. Here your target is Program.exe but your recipe creates a file bin/program.exe: those are not the same thing so it's wrong.
Also $< is only the FIRST prerequisite: when you want to link lots of files together you want to use $^ which is all the prerequisites.
You need to list all the object files you want to create:
bin/program.exe: src/foo.o src/bar.o src/baz.o
g++.exe -o $# $^
If you want to automatically generate all the object files then, assuming you want to compile all the source files, you can do something like:
OBJS := $(patsubst %.cpp,%.o,$(wildcard src/*.cpp))
bin/program.exe: $(OBJS)
g++.exe -o $# $^
You don't need to set VPATH in this situation; it doesn't help.
In a toplevel (unix, GNU) Makefile I want to iterate over a list of subdirs and execute tasks in there but exclude some of them. In this case exclude all subdirs starting with an underscore (_)
The following is working in iterating but ignores the exclusion regex. It still visits dir "_exclude".
In particular I think it ignores the "start of string" ^ (or it takes it literally). I have tried other regexes and they do work. Any idea how to fix that? Or indeed if you have any idea how to eliminate the shelling out or not using GNU makefile extensions?
SHELL=/bin/bash
EXCLUDE_DIRS_REGEX=^_
# later addition:
ALLMAKEFILES = $(shell find . -maxdepth 2 -type f -name Makefile)
SUBDIRS = $(filter-out ./,$(dir $(ALLMAKEFILES)))
all:
for dir in $(SUBDIRS); do \
if [[ "$$dir" =~ $(EXCLUDE_DIRS_REGEX) ]]; then continue; fi; \
make -C $$dir all; \
done
You have two problems to solve:
compute the list of target directories
call make in each of them
For the first one a mixture of make and shell functions or built-ins could be something like:
REGEX := ^_
ALLMAKEDIRS := $(patsubst %/,%,$(dir $(wildcard */Makefile)))
SUBDIRS := $(shell for d in $(ALLMAKEDIRS); \
do ! [[ "$$d" =~ $(REGEX) ]] && echo "$$d"; done)
I kept the REGEX just in case you have more complex regular expressions or you would like it to be easily modified. But of course if it is exactly ^_ and you never change it the following is simpler:
SUBDIRS := $(filter-out _%,$(patsubst %/,%,$(dir $(wildcard */Makefile))))
For the second one your approach is not very make-ish. A better approach would be to have one rule per subdirectory. The following is an example:
.PHONY: all $(SUBDIRS)
all: $(SUBDIRS)
$(SUBDIRS):
$(MAKE) -C $# all
It has several advantages:
there is no need to loop over subdirectories in a complicated recipe, make will do it for you,
make can launch several jobs in parallel (see the -j option) and this will speed-up your build.
Note: calling make in a Makefile is not recommended. Prefer $(MAKE). See this section of the manual for the details.
For clarity, I am running this on windows with GnuWin32 make.
I have a set of directories with markdown files in at several different levels - theoretically they could be in the branch nodes, but I think currently they are only in the leaf nodes. I have a set of pandoc/LaTeX commands to run to turn the markdown files into PDFs - and obviously only want to recreate the PDFs if the markdown file has been updated, so a makefile seems appropriate.
What I would like is a single makefile in the root, which iterates over any and all sub-directories (to any depth) and applies the make rule I'll specify for running pandoc.
From what I've been able to find, recursive makefiles require you to have a makefile in each sub-directory (which seems like an administrative overhead that I would like to avoid) and/or require you to list out all the sub-directories at the start of the makefile (again, would prefer to avoid this).
Theoretical folder structure:
root
|-make
|-Folder AB
| |-File1.md
| \-File2.md
|-Folder C
| \-File3.md
\-Folder D
|-Folder E
| \-File4.md
|-Folder F
\-File5.md
How do I write a makefile to deal with this situation?
Here is a small set of Makefile rules that hopefuly would get you going
%.pdf : %.md
pandoc -o $# --pdf-engine=xelatex $^
PDF_FILES=FolderA/File1.pdf FolderA/File2.pdf \
FolderC/File3.pdf FolderD/FolderE/File4.pdf FolderD/FolderF/File5.pdf
all: ${PDF_FILES}
Let me explain what is going on here. First we have a pattern rule that tells make how to convert a Markdown file to a PDF file. The --pdf-engine=xelatex option is here just for the purpose of illustration.
Then we need to tell Make which files to consider. We put the names together in a single variable PDF_FILES. This value for this variable can be build via a separate scripts that scans all subdirectories for .md files.
Note that one has to be extra careful if filenames or directory names contain spaces.
Then we ask Make to check if any of the PDF_FILES should be updated.
If you have other targets in your makefile, make sure that all is the first non-pattern target, or call make as make all
Updating the Makefile
If shell functions works for you and basic utilities such as sed and find are available, you could make your makefile dynamic with a single line.
%.pdf : %.md
pandoc -o $# --pdf-engine=xelatex $^
PDF_FILES:=$(shell find -name "*.md" | xargs echo | sed 's/\.md/\.pdf/g' )
all: ${PDF_FILES}
MadScientist suggested just that in the comments
Otherwise you could implement a script using the tools available on your operating system and add an additional target update: that would compute the list of files and replace the line starting with PDF_FILES with an updated list of files.
Final version of the code that worked for Windows, based on #DmitiChubarov and #MadScientist's suggestions is as follows:
%.pdf: %.md
pandoc $^ -o $#
PDF_FILES:=$(shell dir /s /b *.md | sed "s/\.md/\.pdf/g")
all: ${PDF_FILES}
Say I have a list of source files like this:
SOURCE=aaa.src bbb.src ccc.src
I want to create a rule to build each corresponding target in its own folder, the name of which depends on the soruce file.
Sourcefile ----> Corresponding output file / target
aaa.src -------------------> aaa/aaa.out
bbb.src -------------------> bbb/bbb.out
ccc.src -------------------> ccc/ccc.out
How do I write a rule for this using GNUMake? My best effort was the following Makefile:
.PHONY: all clean
CC=somecompiler
SOURCE := aaa.src bbb.src ccc.src
RTARGS := $(SOURCE:%.src=%.out)
TDIRS := $(addsuffix /,$(basename $(SOURCE)))
TARGS := $(join $(TDIRS), $(RTARGS))
all:$(TARGS)
%.out: $(SOURCE) # Better source specification?
#[ ! -d "$(dir $*)" ] && mkdir "$(dir $*)"
$(CC) "$(patsubst %/,%,$(dir $*)).src" "$#"
clean:
rm -f $(TARGS)
rmdir --ignore-fail-on-non-empty $(TDIRS)
The problem here is, that any one target (i.e. .out file) depends on every source (i.e. .src file), instead of just the one that has the same basename. I could change the commented line to %.out: %.src but than the source file would have to be in the same directory as the output file. I could also compile it something like this:
%.out: %.src
$(CC) "$>" -o "$*/$#"
But then make would always compile every target regardless of whether it already exists or not.
How do I get make to use the appropriate source file only. What is the proper way of specifying dependences for each target separately in a generic rule?
In GNU Make you can specify prerequisites separately from recipes and still have a generic / pattern rule for all .out files:
.PHONY: all clean
all :
SOURCES := aaa.src bbb.src ccc.src
OUTPUTS := $(foreach src,${SOURCES},$(basename ${src})/${src:%.src=%.out})
# Build dependencies in the form of x/x.out : x.src | x
define ESTABLISH_DEPENDENCY =
$(basename ${1})/${1:%.src=%.out} : ${1} | $(basename ${1}) # Also depend on the output directory.
$(basename ${1}) : # Rule to create the output directory.
mkdir $$#
endef
$(foreach src,${SOURCES},$(eval $(call ESTABLISH_DEPENDENCY,${src})))
all : ${OUTPUTS}
# Generice rule for all .out files.
%.out :
#echo "build $# from $^"
touch $#
%.src : # For debugging only.
touch $#
.PHONY: all
Output:
$ make
touch aaa.src
mkdir aaa
build aaa/aaa.out from aaa.src
touch aaa/aaa.out
touch bbb.src
mkdir bbb
build bbb/bbb.out from bbb.src
touch bbb/bbb.out
touch ccc.src
mkdir ccc
build ccc/ccc.out from ccc.src
touch ccc/ccc.out
Note that it is more efficient and elegant to have order-only dependencies on directories and let make create them, than to have each recipe checking directory existence first.
This question is similar in spirit to question 2543127.
I have a gnu makefile with a list of header files. Each header file may be located in a different directory, e.g.,
HEADERS = $(wildcard *.h) $(wildcard foo/*.h) $(wildcard bar/*.h)
and I want to have the makefile copy all headers to an include directory
INCDIR = ../include
and when a dummy target, e.g., ALL is invoked, it will update the header files in the include directory appropriately, i.e.,
.PHONY: ALL
ALL : $(addprefix $(INCDIR)/,$(notdir $(HEADERS)))
Obviously, I could accomplish what I want quite easily if I knew what the lists of directories were. If I did, then I could write some rules (something) like so (not entirely correct, but you get the jist):
$(addprefix $(INCDIR)/,$(notdir $(filter foo/%.h,$(HEADERS)))) : %.h : foo/%.h
#cp -f $< $#
$(addprefix $(INCDIR)/,$(notdir $(filter bar/%.h,$(HEADERS)))) : %.h : bar/%.h
#cp -f $< $#
$(addprefix $(INCDIR)/,$(notdir $(filter-out bar/%.h,$(filter-out foo/%.h,$(HEADERS))))) : %.h : %.h
#cp -f $< $#
There are two problems with this approach, (1) It becomes tedious as the number of directories increases and (2) I am writing this in a makefile include, which doesn't know directories, all it knows are the variables INCDIR and HEADERS; it does not directly know the directories foo/, bar/, and ./ other than through $(sort $(dir $(HEADERS)))
Question: How can I write a rule to achieve the desired effect under the constraints of only being provided the INCDIR and HEADERS variables.
This should do it:
HFILES = $(notdir $(HEADERS))
DIRS = $(dir $(HEADERS))
TARGETS = $(addprefix $(INCDIR)/, $(HFILES))
all: $(TARGETS)
$(INCDIR)/%.h: %.h
cp $< $#
vpath %.h $(DIRS)
OK. The answer is pretty "easy", although it requires usage of some gnu make that I haven't previously used. My solution, creates a subroutine that requires 2 arguments: (1) the name of the file (sans directory) and (2) the name of the directory in which it resided.
The "subroutine" is a template for a rule. When one evaluates the call to the subroutine, one initiates another rules, just as if one had written it explicitly.
define COPY_HEADER
$$(INCDIR)/$(2) : $(1)$(2)
#cp -f $$< $$#
endef
One then evaluates this subroutine for every header file and passes in the directory part and the file part of each header file.
$(foreach file,$(HEADERS),$(eval $(call COPY_HEADER,$(dir $(file)),$(notdir $(file)))))