When running a CMake generated Makefile with multiple processes (make -jN), the output often gets messed up like this:
[ 8%] [ 8%] [ 9%] Building CXX object App/CMakeFiles/App.dir/src/File1.cpp.o
Building CXX object App/CMakeFiles/App.dir/src/File2.cpp.o
Building CXX object App/CMakeFiles/App.dir/src/File3.cpp.o
I'm not sure, but I think this behavior is also there for Makefiles not generated by CMake. I'd say it happens when multiple processes write to stdout at the same time.
I know I'm probably being pedantic, but is there any (simple) fix to this? ;)
If you're using GNU make, you can do it by redefining SHELL such that commands are wrapped by a trivial utility that ensures atomicity of information printed to standard output. Here's a more detailed description, including sample source for the wrapper utility.
I tried to get the CMake people to fix this, but apparently they don't want to. See http://www.cmake.org/Bug/view.php?id=7062.
The specific CMake bug related to interleaved make output using -jN with N>1 is CMake bug 0012991: "Parallel build output mess". It is still open in the "backlog" state waiting to be fixed.
This bug is actually annoying enough that it's a strong reason to switch to Ninja instead of make. Plus the fact that Ninja is faster than make. Ninja also uses an appropriate number of parallel jobs based on the number of CPU cores present. Also cool is how Ninja is, by default, very quiet: all progress happens on a single line in the terminal unless the build process emits messages or a build step fails. If a build step fails, Ninja prints the full command line that invoked it and displays the output. It's really nice since it makes any warning or error messages stand out. Although currently there is no colored terminal output: that would be a nice improvement but for me the advantages of Ninja over make are tremendous.
Looks like it is already fixed. Add a -Oline parameter to the command line:
make -j 8 -Oline
Version of make:
GNU Make 4.3
Built for x86_64-pc-msys
Sun's (now Oracle's) dmake available on Linux and Solaris takes care of that.
See here and there.
Here is a simple working example of using a wrapper for Make. I'm not sure if I'd encourage it's use, but it's an idea.
# Makefile
SHELL = /tmp/test/wrapper
test: test1 test2
test1:
$(eval export TARGET=$#)
env
test2:
$(eval export TARGET=$#)
env
and this:
#!/usr/bin/env bash
# wrapper
bash $# | sed -e "s/^/${TARGET} /"
Related
Finally getting around to switching to zsh (from bash)... I'm trying to understand a bit more about the completion system and could use a quick pointer. I have been able to get other completions to work for command arguments, but I'm struggling with path completions.
I use a simple function (cdp) to jump to project directories. I've set up a very basic completion script, which almost works. I just can't seem to get the behavior that I'm hoping for.
Ideally, typing cdp in{tab} would expand to all projects starting with in, such as:
~/Projects/indigo ~/Projects/instant
Instead, I can only get cdp {tab} to get the ~/Projects path. From there, it will expand the first-level directory. I'd like to be able to just run standard completion for cd once the project directory is expanded.
Here is the completion script, save in _cdp and added to fpath:
#compdef cdp
basedir="$HOME/Projects"
# the function for jumping to directories...
cdp() {
if [ -z "$1" ] ; then
cd $basedir
else
cd "$1"
fi
}
# completion helper...
_alternative "directories:user directory:($basedir/*)"
It's pretty basic, I'm just stuck trying to sort out where to go next. Any thoughts or pointers would be great. Thanks!
UPDATE
I'm finding that cdpath works fine for most of what I need... It would still be interesting to know how to complete this simple function, but for now at least I have a working solution using cdpath and auto_cd.
Is it possible to have bjam run a script prior to building? If so, how? I would like apply a patch to the code before compiling, but do not want to change the normal workflow for developers. Thanks!
Here's one way I've found that is quick
Echo [ SHELL "./myscript" ] ;
or you could use generators
I would like to have a makefile like this:
cudaLib :
# Create shared library with nvcc
ocelotLib :
# Create shared library for gpuocelot
build-cuda : cudaLib
make build
build-ocelot : ocelotLib
make build
build :
# build and link with the shared library
I.e. the *Lib tasks create a library that runs cuda directly on the device, or on gpuocelot respectively.
For both build tasks I need to run the same build steps, only creating the library differs.
Is there an alternative to running make directly?
make build
Kind of a post-requisite?
Note: This answer focuses on the aspect of a robust recursive invocation of a different target in a given makefile.
To complement Jack Kelly's helpful answer, here's a GNU makefile snippet that demonstrates the use of $(MAKE) to robustly invoke a different target in the same makefile (ensuring that the same make binary is called, and that the same makefile is targeted):
# Determine this makefile's path.
# Be sure to place this BEFORE `include` directives, if any.
THIS_FILE := $(lastword $(MAKEFILE_LIST))
target:
#echo $# # print target name
#$(MAKE) -f $(THIS_FILE) other-target # invoke other target
other-target:
#echo $# # print target name
Output:
$ make target
target
other-target
Using $(lastword $(MAKEFILE_LIST)) and -f ... ensures that the $(MAKE) command uses the same makefile, even if that makefile was passed with an explicit path (-f ...) when make was originally invoked.
Note: While GNU make does have features for recursive invocations - for instance, variable $(MAKE) specifically exists to enable them - their focus is on invoking subordinate makefiles, not on calling a different target in the same makefile.
That said, even though the workaround above is somewhat cumbersome and obscure, it does use regular features and should be robust.
Here is the link to the manual section covering recursive invocations ("sub-makes"):
Recursive Use of make
Most versions of make set a variable $(MAKE) that you can use for recursive invocations.
As you have written it, the build target will need to do something different depending on whether you have just done an ocelot or cuda build. That's another way of saying you have to parameterise build in some way. I suggest separate build targets (much like you already have), with associated variables. Something like:
build-cuda: cudaLib
build-ocelot: ocelotLib
build-cuda build-ocelot:
shell commands
which invoke ${opts-$#}
On the command-line you type make build-cuda (say). Make first builds cudaLib, then it carries out the recipe for build-cuda. It expands the macros before calling the shell. $# in this case is build-cuda, thus ${opts-$#} is first expanded to ${opts-build-cuda}. Make now goes on to expand ${opts-build-cuda}. You will have defined opts-build-cuda (and of course its sister opts-build-ocelot) elsewhere in the makefile.
P.S. Since build-cuda et. al. are not real files, you had better tell make this (.PHONY: build-cuda).
Is there any standard way of providing list of program switches, so it would be possible for zsh to determine possible completions? Or must it provided directly to zsh developers and only they can add completions to zsh?
Your first stop should be man zshcompsys.
Then you could look at an example such as /usr/share/zsh/functions/Completion/Unix/_vim.
The Z-Shell doesn't automatically know what possible switches work with which binary files. As far as I'm aware, there's no standard way for a shell to determine this.
ZSH works by using completion functions, which are written for specific programs. For example, zsh ships with completion functions for ssh, cvs, git, ls, etc.
If you want to look at these completion functions, you can. If you're in a zsh shell, echo $fpath to see the function path that zsh uses to load completion functions. There's a directory called /usr/local/share/zsh/4.3.17/function (location may vary for distributions / zsh versions), which has a bunch of files beginning with _ - _ssh, _cvs, etc. Those are the completion functions.
One massive clue that these are not generated automatically comes from a comment in the _ssh completion function that ships with 4.3.17 (may or may not be in your specific version):
# Completions currently based on OpenSSH 5.9 (released on 2011-09-06).
#
# TODO: update ssh-keygen (not based on 5.9)
# TODO: sshd, ssh-keyscan, ssh-keysign
Providing completion for the Z-Shell: using fpath
You can write your own completion functions, and developers can write functions for their programs and submit to the zsh developers for inclusion. Z-Shell completion functions go somewhere on the fpath.
If the program, say foobar, follows GNU conventions for options, you can use:
compdef _gnu_generic foobar
Otherwise you can write your own functions. The easiest to use IMO is _describe.
Create a file _foobar with contents:
#compdef foobar
cmds=(
'--one:option one'
'--four:option four'
'no-slashes:options do not need to start with a slash'
)
_describe 'foobar' cmds
Place the file somewhere in your $fpath
Add compdef _foobar foobar
If You are using ruby with the optparse package there is a hidden flag --*-completion-zsh=NAME that will output all that is needed for the completion for that ruby program. Store it in a file named _NAME somewhere in your $fpath and it will work. NAME should be exactly what your program/script is called.
I use a folder in my $HOME for that and added the path to $fpath but that required an additional line in my .zshrc:
autoload -U ~/.completion/*(:t)
How do I set the LD_LIBRARY_PATH env variable from a Makefile?
I have some source code that links to a shared library that in turn links to a different shared library (more than 1). The Makefile for building the application only knows about the first shared library.
If I want to build this, I have to specify:
#export LD_LIBRARY_PATH=/path/to/the/shared/libs (for bash)
and that works fine.
However, I would like to do this from the Makefile itself.
Yes, "export" is the correct directive to use. It is documented in detail here. This is the same mechanism as make itself uses to propagate variables to sub-makes. The drawback is that you cannot selectively pass down the variable to some commands and not to others.
There are two other options I can think of:
Using .EXPORT_ALL_VARIABLES (specify as a target somewhere), causes all variables to be exported to the environment of sub-commands.
Specify on the command line:
foo:
EXPORTEDVAR=somevalue gcc $< -o $#
If you don't want to export the LD_LIBRARY_PATH variable within the makefile (e.g. because you have recursive Makefiles which all add to the variable), you can keep it bound to all calls to your compiler and linker.
Either you add it directly to all gcc and ld calls within your target rules, e.g.
my_target: my_target.o
LD_LIBRARY_PATH=/my/library/path gcc -o my_target my_target.o
or you set the global make variables that define the compilers include the path, e.g.:
CC=LD_LIBRARY_PATH=/my/library/path gcc
CPP=LD_LIBRARY_PATH=/my/library/path gcc
CXX=LD_LIBRARY_PATH=/my/library/path gcc
I chose gcc as compiler but of course you can use any compiler you like.
I had the same problem, I had to export LD_LIBRARY_PATH as you did:
export LD_LIBRARY_PATH=/path/to/the/shared/libs ; my_command
My friend showed me an alternative when LD_LIBRARY_PATH only applies to one command, notice no semicolon below.
LD_LIBRARY_PATH=/path/to/the/shared/libs my_command
This article explains more.
I had tried adding:
export LD_LIBRARY_PATH=/path/to/the/shared/libs
which apparently works fine.
I was getting errors because my /path/to/the/shared/libs was incorrect.
Would still be good to know what others do for this and/if there is a better way.
If you want to set LD_LIBRARY_PATH for a particular make , try this LD_LIBRARY_PATH=/path/to/the/shared/libs make.
Adding this to the top of the Makefile worked for me:
export LD_LIBRARY_PATH := $(HOME)/lib
Pay attention not to add any extra spaces to the end of the line. I wasted some time until I realized that was the problem.