I need to write a quick makefile for building all my projects. This is C++ code and I am using gmake.
Say I have a list of directories, I want to cd to each, issue gmake command and if it succeeds, go to the next one and so on.
I cooked this by looking at the gmake manual
.PHONY: all clean dirs $(DIRS)
dirs: $(DIRS)
$(DIRS): \n\t
$(MAKE) -C $#
It works for the "all" target - if I just type gmake, it does the right thing. But if I do gmake clean it does nothing.
I am learning gmake as I go, so I am certainly doing something silly here :)
Thanks for any help.
In order to recursively make something other than the first target (I'm guessing all is your first target), you need to give the sub-make an idea of what to build. You can do so with the MAKEFLAGS and MAKECMDGOALS variables.
For example:
$(DIRS):
$(MAKE) -C "$#" $(MAKEFLAGS) $(MAKECMDGOALS)
Your rule was not passing along the target names, e.g. clean, so the sub-make had no work to do (since all was already built).
Related
I have a makefile that I've changed up a bit here to look more generalized
.PHONY:
bash hash.sh
all: .PHONY lab tests.zip
lab: .PHONY lab.cpp
g++ -o lab lab.cpp -g -Wall -std=c++11
tests.zip:
curl -L -O https://url/tests.zip
unzip tests.zip
tests: .PHONY lab tests.zip
bash scripts/test.bash lab.cpp
clean:
rm -rf scripts tests bitset-tests.zip
I am a TA for an entry level computer science course at my university, and I've created a makefile here for my students to use to compile and test their code seamlessly.
One thing I want to do though is to have the makefile update itself every time the remote repository has a new version of it. I know I could just have them update the file themselves, but my job is to make the students focus less on setting things up and more on just coding for now, since it's entry level. So for the purposes of this, I'm sticking with the idea I have.
Currently, I'm achieving this with a script hash.sh which fetches a hash of the makefile from the repo, and compares it to a hash of the makefile in the student's directory. If the hashes don't match, then the updated makefile is fetched and replaces the old one. This is done in the .PHONY recipe. I should also mention that I don't want to add a recipe that updates it like make update, because again I want the process to be seamless. You'd be surprised how many students wouldn't utilize that feature, so I want to build it into the ones they will use for sure.
Is there a better method for this, or am I doing something wrong with this one?
Thomas has the right idea, but you can't use .PHONY here because it would means the makefile is ALWAYS out of date; make knows this so it doesn't re-exec itself if its included makefile is marked .PHONY.
You need to create a way for make to know if the makefile was changed since the last time it was run locally. I recommend you do it like this:
<normal makefile here>
Makefile: FORCE
curl https://.../Makefile -o Makefile.tmp
cmp -s Makefile Makefile.tmp && rm -f Makefile.tmp || mv -f Makefile.tmp Makefile
FORCE:
What does this do? First it uses a FORCE target which is an old-school way to emulate a .PHONY target, which is always out of date, without actually using .PHONY (which as I mentioned above, is handled specially by GNU make in this situation).
Second it retrieves the Makefile but only updates the local makefile if it has changed. If it hasn't changed, it doesn't update the local makefile and so make won't re-exec itself.
The whole stuff with fetching a hash sounds overly complicated. If you're going to do a fetch anyway, why not unconditionally fetch the entire makefile? It saves a network round trip, which is probably the limiting factor; the actual data is probably just a few kB anyway.
If you're using curl, notice the --time-cond option, for example:
curl https://... --time-cond Makefile -o Makefile
This will only fetch and update the Makefile if it's newer than the mtime of the current file. It works by sending an If-Modified-Since HTTP header, so you'll need some cooperation from the server for this to work reliably.
If using GNU make, you can use another neat trick: Remake the makefile itself. If you have a rule whose target is Makefile, it will be executed before anything else happens, and make will re-read the updated Makefile before proceeding:
.PHONY: Makefile
Makefile:
curl https://... --time-cond Makefile -o Makefile
Note that this will lead to infinite loops if for whatever reason the --time-cond leads to an unconditional update, so it wouldn't hurt to guard against that:
.PHONY: Makefile
Makefile:
[[ ! -v MAKE_RESTARTS ]] && \
curl https://... --time-cond Makefile -o Makefile
If I start (GNU) make with the -C option, it first cd into that directory and
$(CURDIR) will be set to that. I wonder if there is any mechanism to figure out in the makefile where make was originally invoked from?
That is, if I have a makefile in /some/location:
all:
#echo $(CURDIR)
#echo $(SOME_MAGIC_VARIABLE_I_AM_LOOKING_FOR)
and then I do:
$ cd /other/place
$ make -C /some/location
I would get
/some/location
/other/place
printed.
Thanks!
There is no such variable in make.
However, POSIX shells maintain and export a PWD variable to commands that they invoke. All environment variables are imported by make as make variables.
So, if you look in the make variable $(PWD) you will very likely find the original path that the shell that invoked make was using.
I'm working on some bare-metal embedded code that runs on ARM, and thus has to deal with the whole ARM vs. THUMB mode distinction. The current build system uses static pattern rules to determine whether to compile files in ARM or THUMB mode.
$(ACOBJS) : %.o : %.c
#echo
$(CC) -c $(CFLAGS) $(AOPT) -I . $(IINCDIR) $< -o $#
$(TCOBJS) : %.o : %.c
#echo
$(CC) -c $(CFLAGS) $(TOPT) -I . $(IINCDIR) $< -o $#
Where ACOBJS is a list of output objects that should be in ARM mode and the same for TCOBJS and Thumb mode. These lists are created from the list of sources in the usual manner of
ACOBJS = $(ACSRC:.c=.o)
TCOBJS = $(TCSRC:.c=.o)
Currently this results in the object files from the build being strewn about the source tree, which I don't particularly desire. I've been trying to set this up for out of tree builds but haven't been able to get this to work. I don't necessarily need to get full out of tree builds working, but I would like to at least be able to use an output directory under which all the intermediate files end up going. What is the best strategy to achieve this under these constraints?
One option I'm considering is using either automake or the whole autotools toolchain to build a makefile. This would seem to support creating the type of makefile I want, but seems like overkill. It also seems like there would be an inherent impedance mismatch between autotools, which is designed for portable builds, and bare-metal embedded systems, where things like host tuple are dictated by the target micro.
This is a bit old but I was just trying to do the same thing this was the first google hit. I thought it was worth sharing another approach since neither answer is convenient if you're not using autotools and want to be able to build in any directory with a single command and later just blow away that directory.
Here's an example of a Makefile that refers to files relative to the directory containing the Makefile.
MAKEFILE_DIR := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
MFD := $(MAKEFILE_DIR)
CXX=g++
CXXFLAGS=-std=c++14 -Wall -Wextra -pedantic -c
test: test.o adjacency_pointers_graph.o
$(CXX) $^ -o $#
%.o: $(MFD)/%.cpp $(MFD)/adjacency_pointers_graph.h
$(CXX) $(CXXFLAGS) $< -o $#
Then to do an sort of source build:
mkdir build
cd build
make -f ../Makefile
Considering/assuming you don't care about portability and are using GNU make, you can use the VPATH feature:
Create the directory where you want to do your build.
Create a 'Makefile' in that directory with (approximately) the following contents:
path_to_source = ..
VPATH = $(path_to_source)
include $(path_to_source)/Makefile
Change the path_to_source variable to point to the root of your source tree.
Additionally you probably need to tweak your original Makefile to make sure that it supports the out of source build. For example, you can't reference to prerequisites from your build rules and instead must use $^ and $<. (See GNU make - Writing Recipes with Directory Search) You might also need to modify the vpath-makefile. For example: adding CFLAGS+=-I$(path_to_source) might be useful.
Also note that if a file is in both your source and build directory, make will use the file in your build directory.
On automake
If you use automake, you're pretty much using the entire autotools. automake cannot work without autoconf.
The Makefiles generated by automake support out-of-source builds and cross-compilation, so you should be able to create subdirectories arm/ and thumb/ and run ../configure --host=arm-host-prefix in arm/ and run ../configure --host=thumb-host-prefix in thumb/. (I don't know the actual host tuples that you'd use for each compiler.)
Using GNU make
Since you're using GNUMake, you could do something like this:
ACOBJS := $(addprefix arm/,$(ACSRC:.c=.o))
TCOBJS := $(addprefix thumb/,$(TCSRC:.c=.o))
Use something like this answer to ensure that the arm/ and thumb/ directories (and any subdirectories) exist.
My project needs temporary directories which are created during the build using mkdir -p similarly to this:
all: dirtree $(OBJFILES)
dirtree:
#mkdir -p $(BUILD)/temp_directory
But this approach cannot be used with the -j switch, because first of the OBJFILES get compiled before the mkdir target is made.
Is there a standard way to do this?
The problem with your makefile is that creation of your object files does not depend on creation of the relevant directories (only a phony "all" target does). This kind of dependency is necessary for -j option, and even without it your makefile works only by chance. There are two (right) ways to impose the dependency in question.
Directories as separate targets
You created the target for directory creation; what left is just put it as a prerequisite to object file rule:
$(BUILD)/temp_directory/%.o: %.c | dirtree
$(CC) $^ -o $#
The pipe symbol | means that dirtree is an "order only prerequisite". It is used when "dirtree" is a prerequisite but changes in the dirtree do not invalidate object files and do not affect the outcome of compilation command.
Use of "order-only" prerequisite is important here. The thing is that dirtree target would be remade at each Make invocation. That would cause everything that depends on it be remade as well, so it would rebuild all object files every time.
Create directories in shell commands
Another way is to ensure that the directory is created immediately before you invoke compilation
$(BUILD)/temp_directory/%.o: %.c
#mkdir -p $(#D)
$(CC) $^ -o $#
Note the usage of $(#D). This is expanded as "the directory for the target file". So it may be used uniformly in many places, and even with aid of a variable.
Mkdir=#mkdir -p $(#D)
$(BUILD)/temp_directory/%.o: %.c
$(Mkdir)
$(CC) $^ -o $#
$(INSTALL_DIR)/%: src_dir/%
$(Mkdir)
cp -p $^ $#
Both ways ensure that the directory is created before the compilation commands are invoked. Both ways require you to write some text (either | dirtree or $(Mkdir)) at each rule that needs it. Both ways are -j compatible, but the second solution requires mkdir -p to be thread-safe (as two such commands at once may try to create the same directory, and one of them would fail).
While most systems implement it in such a way that mkdir -p is more or less thread safe, on some systems (as in some Solaris systems, for example), they are less thread-safe than the others. However, even in GNU toolchain mkdir -p may fail if they simultaneously invoke the same mkdir(2) library call.
If you want to be very safe, you can work this around as well. What could be the problem? That two mkdir -p scripts try to create the same directory, and clash somewhere inside C library. Then, one of these mkdir-s will succeed, and the other will fail. However, if the mkdir you invoked failed, then it could be thread-unsafety-related failure only if the directory had been created by a concurrent mkdir. So it would be enough to just check that the target directory is created after mkdir invocation:
Mkdir=#mkdir -p $(#D) || test -d $(#D)
(This solution also has an issue with mode: mkdir may fail when directory exists, but doesn't conform to umask, so you might want to check that as well. But that's too much I guess.)
I'm not sure I fully understand your question. However, I can say this: if your build breaks when you add parallelism, then it's an indication that you haven't defined the dependencies correctly. Ask yourself, "Do the directories need to exist before the object files are generated?" If the answer is "yes", then the directories should be listed as prerequisites of the object files. In other words:
${OBJFILES}: dirtree
And yes, that is pretty much the standard way to do this :)
You could have the rules for building the object files call mkdir -p as their first action.
I am using cmake to build my project. For UNIX, I would like to type make from my project's root directory, and have cmake invoked to create the proper Makefiles (if they don't exist yet) and then build my project. I would like the cmake "internal" files (object files, cmake internal Makefiles, etc.) to be hidden (e.g. put in a .build directory) so it doesn't clutter my project directory.
My project has several sub-projects (in particular, a library, a user executable, and a unit test executable). I would like Makefiles (i.e. I type make and this happens) for each sub-project to execute cmake (as above) and build only that sub-project (with dependencies, so the library would be built from the executables' Makefiles, if needed). The resulting binary (.so library or executable) should be in the sub-project's directory.
I made a Makefile which does the main project bit somewhat well, though it feels somewhat hackish. I can't build specific targets using it, because my Makefile simply calls make in cmake's build directory.
Note that because the library is a sole dependency (and probably doesn't need to be build manually, and because I'm lazy) I omitted it in my Makefile.
BUILD_DIR := .build
.PHONY: all clean project-gui ${BUILD_DIR}/Makefile
all: project-gui project-test
clean:
#([ -d ${BUILD_DIR} ] && make -C ${BUILD_DIR} clean && rm -r ${BUILD_DIR}) || echo Nothing to clean
project-gui: ${BUILD_DIR}/Makefile
#make -C ${BUILD_DIR} project-gui
#cp ${BUILD_DIR}/project-gui/project-gui $#
project-test: ${BUILD_DIR}/Makefile
#make -C ${BUILD_DIR} project-test
#cp ${BUILD_DIR}/project-test/project-test $#
${BUILD_DIR}/Makefile:
#[ -d ${BUILD_DIR} ] || mkdir -p ${BUILD_DIR}
#[ -f ${BUILD_DIR}/Makefile ] || (cd ${BUILD_DIR} && cmake ${CMAKE_OPTS} ..)
If it helps, here's my project structure (if this is "wrong" please tell me -- I'm still learning cmake):
project/
project/CMakeLists.txt
project/common.cmake
project/Makefile -- see Makefile above for this; should be replaced with something better, building libproject, project-gui, and project-test
project/libproject/
project/libproject/CMakeLists.txt
project/libproject/libproject.so -- after build
project/libproject/Makefile -- doesn't exist yet; should build libproject only
project/libproject/source/
project/libproject/include/
project/project-gui/
project/project-gui/CMakeLists.txt
project/project-gui/Makefile -- doesn't exist yet; should build libproject then project-gui
project/project-gui/source/
project/project-gui/include/
project/project-test/
project/project-test/CMakeLists.txt
project/project-test/Makefile -- doesn't exist yet; should build libproject then project-test
project/project-test/source/
project/project-test/include/
If you haven't caught on yet, I'm basically looking for a way to build the project and sub-projects as if cmake wasn't there: as if my project consisted of only Makefiles. Can this be done? Is the solution elegant, or messy? Should I be trying to do something else instead?
Thanks!
If cmake is generating the makefiles, you can simply include the generated makefile in the master makefile, eg
# makefile
all: # Default
include $GENERATED
$GENERATED:$CMAKEFILE
# Generate the makefile here`
The included files are generated then make is restarted with the new included files. The included files should detail the targets, etc.
You should be able to change the location of used files using the vpath directive, see e.g. the Gnu make manual,
vpath %.o project/.build
else the tedious way is to rewrite the rules making note of the necessary directory.
Ed:
Perhaps we shouldn't use a flat makefile.
Try something like:
# makefile
all: gui test
clean:
$(MAKE) -f $(GUI-MAKE) clean
$(MAKE) -f $(TEST-MAKE) clean
gui:$(GUI-MAKE)
$(MAKE) -f $(GUI-MAKE) all
$(GUI-MAKE):$(GUI-CMAKE)
# Generate
# Same for test
This should work if the $(MAKE) -f $(GUI-MAKE) all command works on the command line, and we've hidden cmake in the generating target. You would have to copy any other targets to the master makefile as well, and take care running make in parallel.
Propagating object files through should involve something like
%.o:$(GUI-MAKE)
$(MAKE) -f $(GUI-MAKE) $#
although you'll probably get errors trying to make test objects