The problem
Consider the following recipe:
a: b
<create a>
Maybe it's correct. But if b imports another program c,
then it's not. It should instead say this:
a: b c
<create a>
And if c depends on d, it should say this:
a: b c d
<create a>
And any time c changes what it imports,
any output that depends on b
needs its recipe updated in the makefile.
That's tedious. It's also dangerous,
because it's easy to miss a recipe that needed updating.
What would be better
is to only have to say once what other source files a given source file depends on,
rather than each time it appears in any recipe's prerequisites.
That is, the Makefile should be doing the recursion, not I.
(Even better would be if I had to say zero times what the source file depends on, instead generating that information automatically from the code. I wonder if such a solution exists?)
I have an awkward solution.
Here's a (extremely) minimal example:
https://github.com/JeffreyBenjaminBrown/makefile-import-aware/
Here's the Makefile (everything else in the repo is just empty files or documentation):
.no_younger_than/lib/a: \
code/lib/a
install -D /dev/null .no_younger_than/lib/a
.no_younger_than/b: \
code/b \
.no_younger_than/lib/a
install -D /dev/null .no_younger_than/b
c: .no_younger_than/b
echo "yes" > c
Those install directives create a filetree parallel to the code,
which contains an empty "evidence" file corresponding to each source file.
The makefile creates such evidence each time it runs.
If the timestamp on an evidence file is T,
that means the file it corresponds to was not modified any later than T.
This solution does everything that I want,
but I dread having to explain it to someone else.
I'm hoping there exists prior art that's already been documented,
so I can avoid said explanation.
Related
I want to save some git information in a textfile, which I want to put in the src folder of my project.
In case branch Master is checked out, I only want the date of the latest commit. If any other branch is checked out I want the date and the name of the branch like so:
date-branchname
This is my code:
src/version2.txt:
DATE=$(shell git log -1 --date=format:"%Y.%m.%d" --format="%ad")
BRANCH=$(shell git rev-parse --abbrev-ref HEAD)
ifeq ($(BRANCH), 'Master')
$(DATE) > $#
else
DATE+='-'
DATE+=$(BRANCH)
$(DATE) > $#
endif
I'm new to GNU make and quite confused with its syntax.
I assume my ifeq/else blocks are working fine, since I checked printing a dummy text to the version.txt file while having the Master or some other branch checked out.
I also saw my commands to get the date or the branch are ok, since I can put them to the textfile like so:
git log -1 --date=format:"%Y.%m.%d" --format="%ad" > $#
Only when I want to use variables, it seems the variable is empty, for example
$(DATE) > $#
seems to print an empty string to the textfile.
Also, I don't know if my way of creating the DATE-BRANCH output is correct at all.
I've spent ages trying and would really appreciate some help.
Thanks
You cannot create GNU make variables inside recipes and use them later in the GNU make file because GNU make first parses whole file and then runs recipes
You cannot use GNU make ifeqs inside recipes
% in command lines may cause problems in Windows, so I've added a workaround for it in the following code (skip it if your Makefile should work only for Unix-like systems)
' should not be used in ifeqs constants (GNU make reads it literaly)
src/version2.txt does not depend on anything and therefore will not be regenerated by GNU make (if it already exists), consider .PHONY for the target
Try this:
BRANCH:=$(shell git rev-parse --abbrev-ref HEAD)
ifeq ($(OS),Windows_NT)
P:=%%
else
P:=%
endif
.PHONY : src/version2.txt
ifeq ($(BRANCH),master)
src/version2.txt :
git log -1 --date=format:"$PY.$Pm.$Pd" --format="$Pad" >$#
else
src/version2.txt :
git log -1 --date=format:"$PY.$Pm.$Pd" --format="$Pad-$(BRANCH)" >$#
endif
Note that BRANCH will be computed each time you run make even if the src/version2.txt should not be regenerated.
I want to build the following dependency graph, but with pre and post being artifact free:
Before creating/updating any of a, b, or c the command pre should run once and afterwards post should run once. Both do not and preferably should not produce artifacts. And of course, these should only be run if any of a b c have changed. This should all be triggered by a phony all target, i.e. a is never run independently.
Using order-only prerequisites a: | pre does not help because these are always run. Making post depend on a b c won't work because then it is also run all the time because post does not create an artifact.
If this is impossible and artifacts are required after all, how would pre (the more interesting of the two) only run if any of the targets which depend on it have changed?
Note: a etc. are normal makefile targets (which could be called independently), e.g.:
a: a.in a.dependency
#echo Creating a
#mkabc a.in > a
There is only one way in make to force a command to execute before target X is built, but only if target X needs to be built, and that's put the command as the first thing in the recipe for target X. There's no way to manipulate the dependency graph in GNU make so that make determines if a target needs to be built and, if so, first builds some other target before the recipe runs.
So you will definitely have to use recursive make here, putting the command to build the pre target into the recipe of the other targets. However, of course that will cause it to be built multiple times which you don't want.
One way to get around that is to play a trick using eval. Try this (untested):
BUILD_PRE = $(shell $(MAKE) -j1 pre >/dev/null)
post: a b c
echo $#
pre:
echo $#
a b c:
$(BUILD_PRE)$(eval BUILD_PRE =)
touch $#
.PHONY: pre post
In the rule for a, b, and c we first expand the BUILD_PRE variable which results in a recursive make invocation via the shell call. Then the eval expansion will reset the value of BUILD_PRE so that it's now empty; this means in subsequent rules for b and c this first line will expand to the empty string and pre will not be run again.
You may ask, why do we need to use shell here? Can't we just use:
BUILD_PRE = $(MAKE) -j1 pre
so that the first recipe contained a recursive make? The problem with this is that it won't work with parallel make. Suppose the first target make attempts to build is a (it will always be of course). That recipe will contain a recursive make invocation and make will start it. But if you are using -j make does not wait for that recipe to complete: it will go try to start b and c. Since BUILD_PRE is now empty you only get one build of pre, but b and c are not waiting for pre to be completed.
By using a shell function the recursive invocation is forced to complete when the recipe is expanded, before any other recipe is started.
I should say, I suspect there may be a few odd things about this. In particular when make normally invokes a recursive build it does some setup etc. that won't happen when a recursive build is invoked through shell. But, it should work.
Edit: Final Makefile with '+' prefix to mark recursive make calls:
all: allabc
BUILD_PRE = $(shell $(MAKE) pre)
BUILD_POST =
pre:
#echo PRE abc >&2
post:
#echo POST abc >&2
allabc: a b c
#+$(BUILD_POST) > /dev/null
a:
+$(BUILD_PRE)$(eval BUILD_PRE = )
touch "$#"
$(eval BUILD_POST = $$(MAKE) post)
b:
+$(BUILD_PRE)$(eval BUILD_PRE = )
touch "$#"
$(eval BUILD_POST = $$(MAKE) post)
c:
+$(BUILD_PRE)$(eval BUILD_PRE = )
touch "$#"
$(eval BUILD_POST = $$(MAKE) post)
clean:
rm -f a b c
Not sure I understand all the details but assuming you want to build your 5 targets when invoking make all, with the dependencies you show (and maybe a, b and c in parallel), you can, for instance:
.PHONY: all pre post
all:
$(MAKE) pre
$(MAKE) a b c
$(MAKE) post
pre:
<pre-recipe>
post:
<post-recipe>
a:
<a-recipe>
...
Working with cleartool under UNIX, if I have a file in a snapshot view that is unmodified (e.g. still checked in, not checked out, not hijacked),
I can successfully get its predecessor version with this:
$ cleartool describe -predecessor -short file.c
/main/ABC_int/ABC_STAGING/user_ABC_STAGING_dev1/9
No problem in that case.
However, if the file happens to be hijacked (which happens very often with the way I work), cleartool describe refuses to give me its predecessor version:
a) The following two lines simulate an hijack:
$ chmod 666 file.c
$ touch file.c
b) Then at that point if I try the same command line again on the same file:
$ cleartool describe -predecessor -short file.c
cleartool: Error: -predecessor invalid for non file system objects: "file.c".
How can I get the changeset predecessor version path of the file when it is hijacked?
Is there a different way to do this than:
Take a backup of the hijacked file ("cp file.c file.c.backup"),
Run "cleartool update -overwrite <file>" to un-hijack it,
Re-run the cleartool describe command line on it to get the predecessor on the un-hijacked file ("cleartool describe -predecessor -short file.c"),
Then re-hijack it again ("chmod 666 file.c"),
And finally move the backup on top of it ("mv file.c.backup file.c")?
Thanks!
One simple solution is to have a dynamic view with the same config spec as your current snapshot view.
While the file state in the snapshot view might not allow to get its predecessor version, the same file referenced in the dynamic view will.
cleartool descr -pred -short /view/yourDynamicView/vobs/AVob/path/to/files.c
You would still work as usual in the snapshot view, but you would rely on the dynamic view to get the information you need.
The OP confirms in the comments:
it works!
Create temporary dynamic view with:
cleartool mkview -tag <dynview_tagname> -stream <mystream#/myPVOB> -stg -auto
Obtain predecessor of hijacked file with:
cleartool describe -predecessor -short /view/<dynview_tagname>/vobs/<path>/file.c
Destroy temporary dynamic view with:
cleartool rmview -tag <dynview_tagname>
I have a query regarding the execution of a complex command in the makefile of the current system.
I am currently using shell command in the makefile to execute the command. However my command fails as it is a combination of a many commands and execution collects a huge amount of data. The makefile content is something like this:
variable=$(shell ls -lart | grep name | cut -d/ -f2- )
However the make execution fails with execvp failure, since the file listing is huge and I need to parse all of them.
Please suggest me any ways to overcome this issue. Basically I would like to execute a complex command and assign that output to a makefile variable which I want to use later in the program.
(This may take a few iterations.)
This looks like a limitation of the architecture, not a Make limitation. There are several ways to address it, but you must show us how you use variable, otherwise even if you succeed in constructing it, you might not be able to use it as you intend. Please show us the exact operations you intend to perform on variable.
For now I suggest you do a couple of experiments and tell us the results. First, try the assignment with a short list of files (e.g. three) to verify that the assignment does what you intend. Second, in the directory with many files, try:
variable=$(shell ls -lart | grep name)
to see whether the problem is in grep or cut.
Rather than store the list of files in a variable you can easily use shell functionality to get the same result. It's a bit odd that you're flattening a recursive ls to only get the leaves, and then running mkdir -p which is really only useful if the parent directory doesn't exist, but if you know which depths you want to (for example the current directory and all subdirectories one level down) you can do something like this:
directories:
for path in ./*name* ./*/*name*; do \
mkdir "/some/path/$(basename "$path")" || exit 1; \
done
or even
find . -name '*name*' -exec mkdir "/some/path/$(basename {})" \;
Here's some pseudocode for what I want my makefile to do:
if (A doesn't exist) or (B is newer than A):
rm -rf A
create an empty A
parallel_for X in (a large set of files):
if (X is newer than A):
update A using the contents of X
In the above pseudocode, A is an SQLite database, B is a C header file, and each of the files in the "large set of files" is a C source file.
Basically, if I only modify one of the C source files, I just want the database to be quickly updated rather than rebuilding the entire database from scratch.
Is this type of problem solvable directly in GNU make, or am I going to have to resort to using a script?
Thanks in advance!
Something like this ought to work:
A:: B
-rm -f $#
create_A $#
A:: $(all_X)
update_A_from_Xes $# $?
$? expands to the subset of $(all_X) that are newer than A (see the "Automatic Variables section of the GNU Make manual for more details). Therefore, update_A_from_Xes must update its first argument with respect to all of the subsequent arguments; it will only be invoked once.
The double colons tell Make that the commands to run to update A are different when it's out of date with respect to B than when it's out of date with respect to the Xes. I am not sure whether both sets of commands will get run in the case that it is out of date with respect to both; if they do both get run, the A:: B rules will get run first.