I need some help with my Makefile for a project. The source directory looks something like this.
|-- Makefile
|-- drivers
| |-- Makefile
| |-- tty
| |-- Makefile
| |-- console.c
| |-- keyboard.c
|-- kernel
| |-- Makefile
| |-- kmain.c
In the top Makefile, I have exported a variable OBJECTS that I want to populate with object files so I can build and link them together in the top Makefile.
I want to update OBJECTS in, say, drivers/tty/Makefile by doing something like this:
OBJECTS += $(CURDIR)console.o
OBJECTS += $(CURDIR)keyboard.o
But the change to OBJECTS does not bubble up to the top Makefile. I've been looking at the Makefiles in the Linux source tree, and they seem to be doing something similar. However, I can't get it to work. Am I missing something here?
You appear to be using Make recursively, something like
# Makefile:
export OBJECTS :=
all:
$(MAKE) -C drivers/tty
#echo OBJECTS is $(OBJECTS)
# drivers/tty/Makefile:
OBJECTS += $(CURDIR)console.o
all:
whatever
This doesn't work, because each Make has its own OBJECTS; the child Make can't modify variables in the parent Make. It's export, not import/export or share (there's no such thing as import/export or share, I'm just trying to illustrate).
You can get the effect you want by including the other makefiles instead of invoking them:
# Makefile:
OBJECTS :=
all: DRIVERS_TTY
#echo OBJECTS is $(OBJECTS)
include drivers/tty/Makefile
# drivers/tty/Makefile:
OBJECTS += drivers/tty/console.o
DRIVERS_TTY:
whatever
You'll notice there is some unpleasant location-dependency there; drivers/tty/Makefile has "drivers/tty" spelled out inside it, which makes maintenance a pain. There are ways to fix that, once you have this basic include trick working.
When you recursively run make it opens a new subshell for each subsequent call to make so you can't go back up the chain with your exports. One method would be for each call to a submake to append to an object list file and then possibly include that file. A better solution is to probably do something along the lines of having your main makefile include each of these submake files directly instead of calling make on them. This method allows the OBJECTS variable to be built up using the each of the submake files with your OBJECTS += statements. Another added benefit is that you are running just one instance of make instead of multiple submakes which allows make to do better dependency generation. Take a look at "Recursive make considered harmful" http://aegis.sourceforge.net/auug97.pdf
One cool makefile build system that was posted up here before by user Dan Moulding https://stackoverflow.com/users/95706/dan-moulding really showed off a lot of the cool stuff you can do with submake files all while having just one master makefile. Dan's boilermake project is here: https://github.com/dmoulding/boilermake
Related
I am trying to replicate the bash command mv `ls | head -5` ./subfolder1/ in Rust.
This is meant to move the first five files in a directory and it works fine in the shell. I am using the Command process builder but the following code fails at runtime:
Command::new("mv")
.current_dir("newdir")
.args(&["`ls | head -5`", "newdir"])
.env("PATH", "/bin")
.spawn()
Output:
mv: cannot stat cannot stat '`ls | head -5`': No such file or directory
As with pretty much all such structures, Command is a frontend to fork followed by the exec* family, meaning it executes one command, it's not a subshell, and it does not delegate to a shell.
If you want to chain multiple commands you will have to run them individually and wire them by hand, though there exist libraries to provide a shell-style interface (with the danger and inefficiencies that implies).
Can't rightly see why you'd bother here though, all of this seems reasonably easy to do with std::fs (and possibly a smattering of std::env) e.g.
for entry in fs::read_dir(".")?.take(5) {
let entry = entry?;
fs::rename(entry.path(), dest.join(entry.file_name()))?;
}
For clarity, I am running this on windows with GnuWin32 make.
I have a set of directories with markdown files in at several different levels - theoretically they could be in the branch nodes, but I think currently they are only in the leaf nodes. I have a set of pandoc/LaTeX commands to run to turn the markdown files into PDFs - and obviously only want to recreate the PDFs if the markdown file has been updated, so a makefile seems appropriate.
What I would like is a single makefile in the root, which iterates over any and all sub-directories (to any depth) and applies the make rule I'll specify for running pandoc.
From what I've been able to find, recursive makefiles require you to have a makefile in each sub-directory (which seems like an administrative overhead that I would like to avoid) and/or require you to list out all the sub-directories at the start of the makefile (again, would prefer to avoid this).
Theoretical folder structure:
root
|-make
|-Folder AB
| |-File1.md
| \-File2.md
|-Folder C
| \-File3.md
\-Folder D
|-Folder E
| \-File4.md
|-Folder F
\-File5.md
How do I write a makefile to deal with this situation?
Here is a small set of Makefile rules that hopefuly would get you going
%.pdf : %.md
pandoc -o $# --pdf-engine=xelatex $^
PDF_FILES=FolderA/File1.pdf FolderA/File2.pdf \
FolderC/File3.pdf FolderD/FolderE/File4.pdf FolderD/FolderF/File5.pdf
all: ${PDF_FILES}
Let me explain what is going on here. First we have a pattern rule that tells make how to convert a Markdown file to a PDF file. The --pdf-engine=xelatex option is here just for the purpose of illustration.
Then we need to tell Make which files to consider. We put the names together in a single variable PDF_FILES. This value for this variable can be build via a separate scripts that scans all subdirectories for .md files.
Note that one has to be extra careful if filenames or directory names contain spaces.
Then we ask Make to check if any of the PDF_FILES should be updated.
If you have other targets in your makefile, make sure that all is the first non-pattern target, or call make as make all
Updating the Makefile
If shell functions works for you and basic utilities such as sed and find are available, you could make your makefile dynamic with a single line.
%.pdf : %.md
pandoc -o $# --pdf-engine=xelatex $^
PDF_FILES:=$(shell find -name "*.md" | xargs echo | sed 's/\.md/\.pdf/g' )
all: ${PDF_FILES}
MadScientist suggested just that in the comments
Otherwise you could implement a script using the tools available on your operating system and add an additional target update: that would compute the list of files and replace the line starting with PDF_FILES with an updated list of files.
Final version of the code that worked for Windows, based on #DmitiChubarov and #MadScientist's suggestions is as follows:
%.pdf: %.md
pandoc $^ -o $#
PDF_FILES:=$(shell dir /s /b *.md | sed "s/\.md/\.pdf/g")
all: ${PDF_FILES}
Using Make is there a nice way to depend on a directories contents.
Essentially I have some generated code which the application code depends on. The generated code only needs to change if the contents of a directory changes, not necessarily if the files within change their content. So if a file is removed or added or renamed I need the rule to run.
My first thought is generate a text file listing of the directory and diff that with the last listing. A change means rerun the build. I think I will have to pass off the generate and diff part to a bash script.
I am hoping somehow in their infinite intelligence might have an easier solution.
Kudos to gjulianm who got me on the right track. His solution works perfect for a single directory.
To get it working recursively I did the following.
ASSET_DIRS = $(shell find ../../assets/ -type d)
ASSET_FILES = $(shell find ../../assets/ -type f -name '*')
codegen: ../../assets/ $(ASSET_DIRS) $(ASSET_FILES)
generate-my-code
It appears now any changes to the directory or files (add, delete, rename, modify) will cause this rule to run. There is likely some issue with file names here (spaces might cause issues).
Let's say your directory is called dir, then this makefile will do what you want:
FILES = $(wildcard dir/*)
codegen: dir # Add $(FILES) here if you want the rule to run on file changes too.
generate-my-code
As the comment says, you can also add the FILES variable if you want the code to depend on file contents too.
A disadvantage of having the rule depend on a directory is that any change to that directory will cause the rule to be out-of-date — including creating generated files in that directory. So unless you segregate source and target files into different directories, the rule will trigger on every make.
Here is an alternative approach that allows you to specify a subset of files for which additions, deletions, and changes are relevant. Suppose for example that only *.foo files are relevant.
# replace indentation with tabs if copy-pasting
.PHONY: codegen
codegen:
find . -name '*.foo' |sort >.filelist.new
diff .filelist.current .filelist.new || cp -f .filelist.new .filelist.current
rm -f .filelist.new
$(MAKE) generate
generate: .filelist.current $(shell cat .filelist.current)
generate-my-code
.PHONY: clean
clean:
rm -f .filelist.*
The second line in the codegen rule ensures that .filelist.current is only modified when the list of relevant files changes, avoiding false-positive triggering of the generate rule.
I would like to have a makefile like this:
cudaLib :
# Create shared library with nvcc
ocelotLib :
# Create shared library for gpuocelot
build-cuda : cudaLib
make build
build-ocelot : ocelotLib
make build
build :
# build and link with the shared library
I.e. the *Lib tasks create a library that runs cuda directly on the device, or on gpuocelot respectively.
For both build tasks I need to run the same build steps, only creating the library differs.
Is there an alternative to running make directly?
make build
Kind of a post-requisite?
Note: This answer focuses on the aspect of a robust recursive invocation of a different target in a given makefile.
To complement Jack Kelly's helpful answer, here's a GNU makefile snippet that demonstrates the use of $(MAKE) to robustly invoke a different target in the same makefile (ensuring that the same make binary is called, and that the same makefile is targeted):
# Determine this makefile's path.
# Be sure to place this BEFORE `include` directives, if any.
THIS_FILE := $(lastword $(MAKEFILE_LIST))
target:
#echo $# # print target name
#$(MAKE) -f $(THIS_FILE) other-target # invoke other target
other-target:
#echo $# # print target name
Output:
$ make target
target
other-target
Using $(lastword $(MAKEFILE_LIST)) and -f ... ensures that the $(MAKE) command uses the same makefile, even if that makefile was passed with an explicit path (-f ...) when make was originally invoked.
Note: While GNU make does have features for recursive invocations - for instance, variable $(MAKE) specifically exists to enable them - their focus is on invoking subordinate makefiles, not on calling a different target in the same makefile.
That said, even though the workaround above is somewhat cumbersome and obscure, it does use regular features and should be robust.
Here is the link to the manual section covering recursive invocations ("sub-makes"):
Recursive Use of make
Most versions of make set a variable $(MAKE) that you can use for recursive invocations.
As you have written it, the build target will need to do something different depending on whether you have just done an ocelot or cuda build. That's another way of saying you have to parameterise build in some way. I suggest separate build targets (much like you already have), with associated variables. Something like:
build-cuda: cudaLib
build-ocelot: ocelotLib
build-cuda build-ocelot:
shell commands
which invoke ${opts-$#}
On the command-line you type make build-cuda (say). Make first builds cudaLib, then it carries out the recipe for build-cuda. It expands the macros before calling the shell. $# in this case is build-cuda, thus ${opts-$#} is first expanded to ${opts-build-cuda}. Make now goes on to expand ${opts-build-cuda}. You will have defined opts-build-cuda (and of course its sister opts-build-ocelot) elsewhere in the makefile.
P.S. Since build-cuda et. al. are not real files, you had better tell make this (.PHONY: build-cuda).
In order to keep my Qt project somewhat organized (using Qt Creator), I've got one .pro file and multiple .pri files. Just recently I added a class to one of my .pri files that has the same filename as a class that already existed in a separate .pri file.
The file structure and makefiles generated by qmake appear to be oblivious to the filename collision that ensues. The generated moc_* files all get thrown into the same subdirectory (either release or debug, depending) and one ends up overwriting the other. When I try to make the project, I get several warnings that look like this:
Makefile.Release:318: warning: overriding commands for target `release/moc_file.cpp`
And the project fails to link.
Here is a simple example of what I'm talking about.
Directory structure:
+ project_dir
| + subdir1
| | - file.h
| | - file.cpp
| + subdir2
| | - file.h
| | - file.cpp
| - main.cpp
| - project.pro
| - subdir1.pri
| - subdir2.pri
Contents of project.pro:
TARGET = project
TEMPLATE = app
include(subdir1.pri)
include(subdir2.pri)
SOURCES += main.cpp
Contents of subdir1.pri:
HEADERS += subdir1/file.h
SOURCES += subdir1/file.cpp
Contents of subdir2.pri:
HEADERS += subdir2/file.h
SOURCES += subdir2/file.cpp
Is there a way to tell qmake to generate a system that puts the moc_* files from separate .pri files into separate subdirectories?
In subdir1.pri try appending
MOC_DIR = "subdir1/MOCFiles"
Also for subdir2.pri give
MOC_DIR = "subdir2/MOCFiles"
It isn't tested. Just check it out. Hope it will work.
Edit 1 : Where MOCFiles is your desired folder for your moc files to get into.
Edit 2 : I just stopped mentioning with the MOC files directory since that has been asked specifically in the question. But additionally you may also have to add the following to each of the pri files. (Make sure that the folders are different for different *.pri files)
RCC_DIR = "subdir1/RCCFiles"
UI_DIR = "subdir1/UICFiles"
OBJECTS_DIR = "subdir1/ObjFiles"
I believe having multiple pri files can work without collisions by having the same file names. Since you have accepted an answer (which states it is not possible), make the above changes and give a try. Do let know if it isn't working.
Best thing to do is to make sure that all files have a unique name. There are other tools besides qmake which will also break when you try to do what you're doing; you also potentially make it confusing for yourself (e.g. understanding what #include "file.h" does is more difficult).
I have tried this before. The short answer is to name them differently somehow. Another answer would be to treat each subdirectory as a separate library, with its own .pro file, and use a subdirs type to compile all the library directories.
If you really want to investigate a full answer, you can specify the tool to be used for moc. In this situation, you might be able to mangle the name so that a slightly different name is used for the two different files. However, you would then also need to make sure each differently-named file is added to the list of files to compile and link, and the originally-named moc file is not (or your build will fail).