I am trying to create a Makefile
I have a Makefile variable containing dependent directories like
DEPDIR := temp foo bar
I need the absolute path of each directory. So, I wrote a rule:
DIRS := $(foreach dirs, $(DEPDIR), $(shell find $(HOME) -name "$$dirs"))
and I also tried:
DIRS := $(notdir $(foreach dirs, $(DEPDIR), $(shell find $(HOME) -name "$$dirs")))
But variable DIRS in both case is empty.
Am I missing something here?
Using find for this is completely wacky. There may be directories elsewhere in the directory tree with the same name, and anyway, presumably, the ones you want are in the current directory.
If they can be nested within other directories, running find on the current directory might make some sense, but you are probably simply looking for
$(patsubst %,${PWD}/%,$(DIRS))
I've no idea why did you write double $: $$dirs. Try this:
DIRS := $(foreach dir, $(DEPDIR), $(shell find $(HOME) -type d -name "$(dir)" ))
Consider narrowing down the search criteria, since $(HOME) by definition can contain anything.
I guess you will not argue that it'd be better to keep track of the project dependencies using a dedicated directory. And there are many ways to build such kind of directories(generating symlinks, downloading/extracting compressed packages, cloning repositories, using package managers etc.). Therefore, I'll suggest the following pattern as well.
MAKEFILE_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
PROJECT_DIR := $(dir $(MAKEFILE_PATH))
DIRS := $(realpath $(addprefix $(PROJECT_DIR)deps/, dir1 dir2 dir3))
Here we extract filename of the current makefile from MAKEFILE_LIST, then store its absolute path in MAKEFILE_PATH variable. Then the directory part of the path to the makefile is stored in PROJECT_DIR. By means of addprefix we prepend $(PROJECT_DIR)/deps to the front of each dependent directory(dir1 dir2 dir3). Then we resolve symlinks, and construct canonical absolute names for the dependent directories, then store them in DIRS.
Related
I'm trying to make a Makefile. I have one folder called pictures and one called thumbs. In pictures there are pictures called pic1.jpg, pic2.jpg, pic3.jpg, and so on. I want to make thumbnails from these pictures and place them in the thumbs folder. This is what I have tried:
infiles = $(wildcard pictures/*.jpg)
outfiles = $(subst pictures,thumbs, $(infiles))
all : $(outfiles)
convert -thumbnail 100 pictures/*.jpg thumbs/*.jpg
Anyone knows what I'm doing wrong?
Your makefile says "here is how to crate all; it can be made when all the outfiles are up to date", but you don't say how to create those files.
A more idiomatic apprach is to specify a dependency for each individual file, and then Make can take it from there. In other words, say that all depends on outfiles just like you did, but then separately specify how each outfile depends on its respective infile.
infiles = $(wildcard pictures/*.jpg)
outfiles = $(subst pictures,thumbs, $(infiles))
.PHONY: all
all : $(outfiles)
thumbs/%.jpg: pictures/%.jpg
mkdir -p thumbs
# Guesswork here; probably update the command
convert -thumbnail 100 $< $#
The .PHONY declaration marks all as just a target name, not a file which needs to be built only if it doesn't exist, or is older than its dependencies.
For clarity, I am running this on windows with GnuWin32 make.
I have a set of directories with markdown files in at several different levels - theoretically they could be in the branch nodes, but I think currently they are only in the leaf nodes. I have a set of pandoc/LaTeX commands to run to turn the markdown files into PDFs - and obviously only want to recreate the PDFs if the markdown file has been updated, so a makefile seems appropriate.
What I would like is a single makefile in the root, which iterates over any and all sub-directories (to any depth) and applies the make rule I'll specify for running pandoc.
From what I've been able to find, recursive makefiles require you to have a makefile in each sub-directory (which seems like an administrative overhead that I would like to avoid) and/or require you to list out all the sub-directories at the start of the makefile (again, would prefer to avoid this).
Theoretical folder structure:
root
|-make
|-Folder AB
| |-File1.md
| \-File2.md
|-Folder C
| \-File3.md
\-Folder D
|-Folder E
| \-File4.md
|-Folder F
\-File5.md
How do I write a makefile to deal with this situation?
Here is a small set of Makefile rules that hopefuly would get you going
%.pdf : %.md
pandoc -o $# --pdf-engine=xelatex $^
PDF_FILES=FolderA/File1.pdf FolderA/File2.pdf \
FolderC/File3.pdf FolderD/FolderE/File4.pdf FolderD/FolderF/File5.pdf
all: ${PDF_FILES}
Let me explain what is going on here. First we have a pattern rule that tells make how to convert a Markdown file to a PDF file. The --pdf-engine=xelatex option is here just for the purpose of illustration.
Then we need to tell Make which files to consider. We put the names together in a single variable PDF_FILES. This value for this variable can be build via a separate scripts that scans all subdirectories for .md files.
Note that one has to be extra careful if filenames or directory names contain spaces.
Then we ask Make to check if any of the PDF_FILES should be updated.
If you have other targets in your makefile, make sure that all is the first non-pattern target, or call make as make all
Updating the Makefile
If shell functions works for you and basic utilities such as sed and find are available, you could make your makefile dynamic with a single line.
%.pdf : %.md
pandoc -o $# --pdf-engine=xelatex $^
PDF_FILES:=$(shell find -name "*.md" | xargs echo | sed 's/\.md/\.pdf/g' )
all: ${PDF_FILES}
MadScientist suggested just that in the comments
Otherwise you could implement a script using the tools available on your operating system and add an additional target update: that would compute the list of files and replace the line starting with PDF_FILES with an updated list of files.
Final version of the code that worked for Windows, based on #DmitiChubarov and #MadScientist's suggestions is as follows:
%.pdf: %.md
pandoc $^ -o $#
PDF_FILES:=$(shell dir /s /b *.md | sed "s/\.md/\.pdf/g")
all: ${PDF_FILES}
I am trying to write a recursive make recipe. In this recipe, each target is dependent on a file with an equal name on the parent directory. A minimal (non-working) example:
foo/.dirstamp:
mkdir $(dir $#)
touch $#
.SECONDEXPANSION:
%/.dirstamp: $$(dir $$*).dirstamp
mkdir $(dir $#)
touch $#
With this example, I would expect make foo/bar/qux/lol/.dirstamp to generate the whole directory tree (if it does not exist), touching all .dirstamp files along the way. However, it does not work:
$ ls # note that there is nothing, make is meant to create the dir tree
Makefile
$ make --debug=v foo/bar/qux/lol/.dirstamp
GNU Make 4.0
[...]
Reading makefiles...
Reading makefile 'Makefile'...
Updating goal targets....
Considering target file 'foo/bar/qux/lol/.dirstamp'.
File 'foo/bar/qux/lol/.dirstamp' does not exist.
Finished prerequisites of target file 'foo/bar/qux/lol/.dirstamp'.
Must remake target 'foo/bar/qux/lol/.dirstamp'.
make: *** No rule to make target 'foo/bar/qux/lol/.dirstamp'. Stop.
It works fine as long as the recursive recipe only needs to be expanded twice, e.g., make foo/bar/.dirstamp works fine.
How can this work for an arbitrary number of levels? How can I handle a recursive expansion for the target and prerequisites names?
Note: my real problem is that the prerequisites of my recipes are in a root
directory different from the target so I am using the recipe above to duplicate the directory tree. I know about mkdir -p which seems to work fine in GNU systems. I am still interested on knowing how I would solve the recursion problem for arbitrary levels. which no longer works because part of the team is using Mac and mounting this directories over smb.
More details on the actual problem: prerequisites are in data/x/y/z while targets go into results/x/y/z. However, the results directory tree does not exist and needs to be created as needed. To solve this, I made the creation of parent directories an order-only prerequisite (via the .dirstamp files on my minimal example above).
can't copy data into results, that's several TB of data;
can't have the targets created in data, that's read-only;
can't use mkdir -p because the results directory will not be local, mounted over smb, and others may use non-GNU systems;
After an hint from #EtanReisner on the question:
make won't apply a rule more than once. That's a built-in (intentional) limitation. Without working around that with manual recursion or manually building the set of targets and using a static pattern rule (which may or may not actually work I'm not sure) there's not much you can do about this.
I worked up this solution:
RESULT_DIRS := $(patsubst data/%, results/%, $(shell find data/* -type d -print))
DIRSTAMPS := $(addsuffix /.dirstamp, $(RESULT_DIRS))
results/.dirstamp:
mkdir $(dir $#)
touch $#
.SECONDEXPANSION:
$(DIRSTAMPS): $$(dir $$(patsubst %/.dirstamp, %, $$#)).dirstamp
mkdir $(dir $#)
touch $#
It will duplicate the data directory tree in results as the dirstamp files are required. They are required by making them prerequisites of the other recipes (note the | which makes them order-only prerequisites):
results/%/foo.analysis: data/%/foo.data | results/%/.dirstamp
$(SOME_ANALYSIS_PROGRAM) $^ > $#
I have a function that returns all the files in a directory.
# Returns all files in folder recursively that match pattern
#
# $(call rwildcard, folder,pattern)
rwildcard=$(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2) $(filter $(subst *,%,$2),$d))
Argument 1, i.e. folder, is the path of the folder where to search for files recursively and it's user provided
If this argument is "/" this would run out of memory and crash with an exception.
Is there a way to prevent this? besides filtering "/" as an argument.
Note: i'm using cygwin
I suspect you aren't making any progress in the inner rwildcard. If it matches "." every time, are you stuck in a loop?
Can you use another tool to get a list of files?
r := $(shell find $(dir) -type f -name \*$(likethis)\*)
Using Make is there a nice way to depend on a directories contents.
Essentially I have some generated code which the application code depends on. The generated code only needs to change if the contents of a directory changes, not necessarily if the files within change their content. So if a file is removed or added or renamed I need the rule to run.
My first thought is generate a text file listing of the directory and diff that with the last listing. A change means rerun the build. I think I will have to pass off the generate and diff part to a bash script.
I am hoping somehow in their infinite intelligence might have an easier solution.
Kudos to gjulianm who got me on the right track. His solution works perfect for a single directory.
To get it working recursively I did the following.
ASSET_DIRS = $(shell find ../../assets/ -type d)
ASSET_FILES = $(shell find ../../assets/ -type f -name '*')
codegen: ../../assets/ $(ASSET_DIRS) $(ASSET_FILES)
generate-my-code
It appears now any changes to the directory or files (add, delete, rename, modify) will cause this rule to run. There is likely some issue with file names here (spaces might cause issues).
Let's say your directory is called dir, then this makefile will do what you want:
FILES = $(wildcard dir/*)
codegen: dir # Add $(FILES) here if you want the rule to run on file changes too.
generate-my-code
As the comment says, you can also add the FILES variable if you want the code to depend on file contents too.
A disadvantage of having the rule depend on a directory is that any change to that directory will cause the rule to be out-of-date — including creating generated files in that directory. So unless you segregate source and target files into different directories, the rule will trigger on every make.
Here is an alternative approach that allows you to specify a subset of files for which additions, deletions, and changes are relevant. Suppose for example that only *.foo files are relevant.
# replace indentation with tabs if copy-pasting
.PHONY: codegen
codegen:
find . -name '*.foo' |sort >.filelist.new
diff .filelist.current .filelist.new || cp -f .filelist.new .filelist.current
rm -f .filelist.new
$(MAKE) generate
generate: .filelist.current $(shell cat .filelist.current)
generate-my-code
.PHONY: clean
clean:
rm -f .filelist.*
The second line in the codegen rule ensures that .filelist.current is only modified when the list of relevant files changes, avoiding false-positive triggering of the generate rule.