How to create a directory in a makefile when mkdir -p is not available? - gnu-make

I have a makefile which does the usual directory creation:
$(Release_target_OBJDIR)/%.o: %.cpp
mkdir -p $(dir $#)
$(COMPILE.cpp) $< $(CFLAGS) $(INCLUDES) -o $#
Unfortunately when I run this under scratchbox2 the mkdir -p command always fails silently.
I attempted the following kludge which doesn't work:
$(Release_target_OBJDIR)/%.o: %.cpp
mkdir $(dir $(dir $(dir $#)))
mkdir $(dir $(dir $#))
mkdir $(dir $#)
$(COMPILE.cpp) $< $(CFLAGS) $(INCLUDES) -o $#
This outputs:
mkdir -p /home/foo/projects/htc/arm/obj/cbar/release/
mkdir -p /home/foo/projects/htc/arm/obj/cbar/release/
mkdir -p /home/foo/projects/htc/arm/obj/cbar/release/
... the trailing slash prevents the dir function from stripping the last directory in the way I wanted.
Short of writing a script or small C app to replicate the "-p" functionality, does anyone have any ideas for creating the subdirectories within the makefile?
Without the -p option mkdir will give an error when the makefile tries to create a directory which already exists. I can do mkdir blah 2> /dev/null but then I risk losing other error messages.
Does anyone have any thoughts as to why mkdir -p doesn't work under scratchbox2?
EDIT
Based on suggestions by bobbogo I put this together. It looks fairly convoluted, but seems to work, even under scratchbox2.
# Generic variables for use in functions
comma:= ,
empty:=
space:= $(empty) $(empty)
# Make directory function
forlooprange = $(wordlist 1,$(words $1),1 2 3 4 5 6 7 8 9 10)
forloop = $(foreach n,$(call forlooprange,$1),$(call $2,$n,$3))
mkdirfunc0 = test -d $1 || mkdir $1;
mkdirfunc1 = $(call mkdirfunc0,/$(subst $(space),/,$(foreach n,$(wordlist 1,$1,$2),$n)))
mkdirfunc2 = $(call forloop,$1,mkdirfunc1,$1)
mkdirmain = $(call mkdirfunc2,$(subst /, ,$1))
.PRECIOUS: %/.sentinel
%/.sentinel:
$(call mkdirmain,$*)
touch $#

You can replace your forest of mkdirs with this:
$(Release_target_OBJDIR)/%.o: %.cpp
$(foreach d,$(subst /, ,${#D}),mkdir $d && cd $d && ):
∶
This will create a shell command somethng like this:
mkdir projects && cd projects && mkdir htc && cd htc && mkdir arm && cd arm && :
This runs for every compile. Not very elegant. You could optimise this by using some sort of sentinel file. For instance:
$(Release_target_OBJDIR)/%.o: %.cpp ${Release_target_OBJDIR}/.sentinel
∶
%/.sentinel:
$(foreach d,$(subst /, ,$*),mkdir $d && cd $d && ):
touch $#
.sentinel gets created once before all objects, and is make -j friendly. In fact you should do it this way even if mkdir -p works for you (in which case you would use mkdir -p rather than the $(foreach) hacksolution).

You can tell make to ignore any failure return code from a command using -:
$(Release_target_OBJDIR)/%.o: %.cpp
-mkdir $(dir $(dir $(dir $#)))
-mkdir $(dir $(dir $#))
-mkdir $(dir $#)
$(COMPILE.cpp) $< $(CFLAGS) $(INCLUDES) -o $#
(Note that this doesn't address the trailing slash problem.)

Related

Makefile | How to check only once whether an output dir already exists?

Currently the shell script checks for the existence of intended object directory right before each compiler call. How do I modify my Makefile so that the code checks only once before it moves on to compiling all the prerequisites?
Here is my Makefile:
#########################################################
## BUILD TASKS ##
#########################################################
HDIR := hdr
SDIR := src
ODIR := obj
EXET := a
OBJS := main.o mainhdr.o testcode.o
OSRC := $(addprefix $(ODIR)/, $(OBJS))
CXX := g++
CXXFLAGS := -I$(HDIR) -g -Wall -std=c++17
## BUILD DIRECTIVE:
all: $(EXET)
$(EXET): $(OSRC)
$(CXX) $(CXXFLAGS) $^ -o $#
$(ODIR)/%.o: $(SDIR)/%.cpp
if [ ! -d "$(ODIR)" ]; then mkdir $(ODIR); fi
$(CXX) -c $(CXXFLAGS) $^ -o $#
## foo.bak: foo.bar
## if [ ! -d "$(ODIR)" ]; then mkdir $(ODIR); fi
#########################################################
## CLEAN TASK ##
#########################################################
.PHONY: clean
clean:
rm -r $(EXET) $(ODIR)
I have tried putting:
foo.bak: foo.bar
if [ ! -d "$(ODIR)" ]; then mkdir $(ODIR); fi
as suggested here right before the clean task, but that doesn't seem to work. I could be understanding it wrong, but isn't that the Makefile is executed recursively so by putting that block of code at the end it should be at the tip of recursion and thus executed before everything else?
On newer gnu-make, you can use 'order-only-prerequisites`. This eliminate the timestamp checking, and only required that the prerequisite will exists. This works well to ensure that directories will be created before files are stored into them, and can significantly speedup build jobs
$(ODIR):
mkdir $(ODIR)
# Note pipe '|' to separate order only prereq.
$(ODIR)/%.o: $(SDIR)/%.cpp | $(ODIR)
$(CXX) -c $(CXXFLAGS) $^ -o $#
See: https://www.gnu.org/software/make/manual/make.html#Prerequisite-Types

How can I avoid repetitive commands, such as cd in a target rule?

In my makefile I have many cd $(d) commands, one for each simple command of a target.
Is there any way to reduce the number of cd $(d) commands?
all:
cd $(d); command1
cd $(d); command2
cd $(d); command3
cd $(d); command4
cd $(d); command5
Use a shell script that CDs to $d first as make's ${SHELL}.
.PHONY: all
all: SHELL := ./cd-to-d
all: .SHELLFLAGS :=
all:
command1
command2
command3
command4
We use target specific variables so as not to disturb other commands in the makefile.
It gets a bit messy if we add the rule to create cd-to-d though.
cd-to-d: SHELL := ${SHELL}
cd-to-d: .SHELLFLAGS := ${.SHELLFLAGS}
cd-to-d:
echo '#!/bin/bash' >$#-tmp
echo 'cd "$d"' >>$#-tmp
echo 'exec "$$#"' >>$#-tmp
chmod a+x $#-tmp
mv $#-tmp $#
.PHONY: all
all: SHELL := ./cd-to-d
all: .SHELLFLAGS :=
all: cd-to-d
all:
command1
command2
command3
command4
Why all the $#-tmp noise? If we get an error while creating cd-to-d, we don't want to leave an old one lying around. You could use .DELETE_ON_ERROR: instead (I always do).
Why the SHELL := ${SHELL} noise? When make builds cd-to-d as a dependency of all, it will inherit all's target specific defines. We need to cancel those concerned with changing the recipe shell.
Yuk.
You could use line continuation and pass the shell just one line where commands are separated by semicolons:
all:
cd $(d); \
command1; \
command2; \
command3; \
command4; \
command5
Or even
all:
cd $(d); command1; command2; command3; command4; command5
Recursive make?
.PHONY: all
all: ; ${MAKE} -C $d all-in-d
PHONY: all-in-d
all-in-d:
command1
command2
command3
command4

How to create directories for dist files?

Here's my Makefile:
dist/%.js: src/%.js node_modules
$(NM)/babel $< -o $#
build: $(patsubst src/%,dist/%,$(wildcard src/**/*.js))
It runs a command like this:
node_modules/.bin/babel src/deep/foo.js -o dist/deep/foo.js
The problem is that if dist/deep doesn't exist, it errors:
Error: ENOENT: no such file or directory, open 'dist/deep/foo.js'
So what I want to do is add an extra dependency on the directory, which I was hoping I could do with something like this:
dist/%.js: src/%.js $(dir dist/%) node_modules
$(NM)/babel $< -o $#
dist/%/:
mkdir -p $#
build: $(patsubst src/%,dist/%,$(wildcard src/**/*.js))
But it doesn't work. $(dir dist/%) isn't filling in that % like I hoped. Running make --trace yields:
Makefile:10: update target 'dist/deep/foo.js' due to: src/deep/foo.js dist/ node_modules
i.e., you can see it has a dependency on dist/, but I was hoping it'd depend on dist/deep/ so that I could mkdir -p it.
Is there a way to achieve what I want?
First a subsidiary snag. Judging from:
$(wildcard src/**/*.js)
it seems you want this function to perform recursive globbing,
returning all *.js files that exist in src or any subdirectory
thereof.
I don't know what shell you've got, but they don't all do that by
default. The linux bash shell doesn't, though as of bash 4.0
it will do it if the shell option globstar is set.
And anyway, $(wildcard ...) won't do it (unless, possibly, the
operative shell does it by default, which I'm not in a position to
check out). So you can't dependably use $(wildcard ...) for that
purpose. You need make to be invoking a shell in which recursive
** globbing is enabled, and then call:
$(shell ls src/**/*.js)
So that's what I'll do now in showing how to solve your problem with
a simple example. I've got:
src/
one.js
a/
two.js
c/
four.js
b/
three.js
and I just want to each *.js file copied from beneath src to the
same relative name under dist, ensuring that dist and all
necessary subdirectories exist when required to. (Of course, this
could all be done at once with cp). Here is a makefile:
SHELL := /bin/bash
.SHELLFLAGS := -O globstar -c
SRCS := $(shell ls src/**/*.js)
DISTS := $(patsubst src/%,dist/%,$(SRCS))
DESTDIRS := $(dir $(DISTS))
.PHONY: all clean
all: $(DISTS)
dist/%.js: src/%.js | $(DESTDIRS)
cp $< $#
$(DESTDIRS):
mkdir -p $#
clean:
rm -fr dist
which runs like:
$ make
mkdir -p dist/a/c/
mkdir -p dist/b/
cp src/a/c/four.js dist/a/c/four.js
cp src/a/two.js dist/a/two.js
cp src/b/three.js dist/b/three.js
cp src/one.js dist/one.js
In that makefile,
| $(DESTDIRS)
makes each of the $(DESTDIRS) an order-only prerequisite
of any target dist/%.js. An order-only prequisite is not considered in determining whether its
target shall be made, but if it is determined that the target shall be made, then the
order-only prequisite will be made first.

Expanding directories in variables with make

I have a makefile (below) for a project where I've been given a folder of "Raw Data" - a set of files from a colleague, and I've made an R script that does an analysis on some of those files. What I want to do with a the makefile then is assign the directory to a variable RAWDIR, and then use that variable in specifying the make dependencies of the R script, and as a command line argument for the script. Usually in the shell, directories with spaces are expanded when using double quotes and curly braces, but I do not know if this is also correct for make files, as with the following makefile I get the message make: *** No rule to make target""../Raw', needed by pulls'. Stop. So I do not think my file path assigned to RAWDIR is being expanded properly.
Thanks.
RAWDIR="../Raw Data/Fc Project Raw Data"
.PHONY: dirs
pulls: dirs "${RAWDIR}/pm_fc_dnds_cleandata.csv" "${RAWDIR}/fc1_seqs.fasta" "${RAWDIR}/fc2_seqs.fasta" "${RAWDIR}/pm1_seqs.fasta" "${RAWDIR}/pm2_seqs.fasta"
Rscript Allele_Pulling.R "${RAWDIR}/" "${RAWDIR}/pm_fc_dnds_cleandata.csv"
dirs:
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/PM
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/Both
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC1PM1
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC1PM2
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC2PM1
mkdir -p -v Pulled_Allelic_Pairs/Unaligned/FC2PM2
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC
mkdir -p -v Pulled_Allelic_Pairs/Aligned/PM
mkdir -p -v Pulled_Allelic_Pairs/Aligned/Both
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC1PM1
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC1PM2
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC2PM1
mkdir -p -v Pulled_Allelic_Pairs/Aligned/FC2PM2
In general spaces in pathnames are not well supported by make. At least some functions in GNU make could handle spaces that are escaped by \.
The following should work in your use case:
RAWDIR="../Raw\ Data/Fc\ Project\ Raw\ Data"

how to specify obj and bin directories in GNUmake

I have a following directory structure,
I also have this make file that works fine, but it needs all files in the same directory and also it creates *.o and bin files in the same directory. Can someone please show me how to improve this code so that i can move *.h files into /h, *.c files into /src. Also *.o files would be created in /obj and the binary file will be created in /bin?
I was thinking of something like this. This part only creates *.o files, no binary files. However, this is giving me an error right now.
Step 1: h/
Add a variable, make a small change to the %.o rule, and add a vpath directive, so that the %.o rule will know where to look:
INC_DIR = h
%.o: %.c
$(cc) -I$(INC_DIR) -c $<
vpath %.h $(INC_DIR)
Step 2: src/
Add another variable, change the assignment of objs, add another vpath:
SRC_DIR := src
objs:=$(patsubst $(SRC_DIR)/%.c,%.o,$(wildcard $(SRC_DIR)/*.c))
vpath %.c $(SRC_DIR)
Step 3: obj/
Add a variable, change objs and the %.o rule again, and the clean rule:
OBJ_DIR = obj
objs:=$(patsubst $(SRC_DIR)/%.c,$(OBJ_DIR)/%.o,$(wildcard $(SRC_DIR)/*.c))
$(OBJ_DIR)/%.o: %.c
$(cc) -Ih -c $< -o $#
clean:
rm -f *.d $(OBJ_DIR)/*.o $(prog)
Step 4: bin/
Add another variable, and change the assignment of prog:
BIN_DIR := bin
prog:=$(BIN_DIR)/$(notdir $(PWD))
EDIT:
What you are now asking for is a bad design. But here it is:
obj/makefile:
SRC_DIR := ../src
objs:=$(patsubst $(SRC_DIR)/%.c,%.o,$(wildcard $(SRC_DIR)/*.c))
cc:=gcc
.PHONY: ALL_OBJS
ALL_OBJS: $(objs)
INC_DIR := ../h
%.o: %.c
$(cc) -I$(INC_DIR) -c $<
vpath %.c $(SRC_DIR)
.PHONY: clean test
clean:
rm -f *.[od]
-include *.d
bin/makefile:
P:= $(PWD)
P:= $(dir $(P))
prog:= $(notdir $(P:/=))
OBJ_DIR := ../obj
objs:=$(notdir $(wildcard $(OBJ_DIR)/*.o))
cc:=gcc
ccflags:=-lcurses -lgdbm -lgdbm_compat
$(prog): $(objs)
$(cc) $(ccflags) -o $# $^
vpath %.o $(OBJ_DIR)
.PHONY: clean test
clean:
rm -f *.d $(prog)
test: $(prog)
$(test)
-include *.d

Resources