cmake: Working with multiple output configurations - build-process

I'm busy porting my build process from msbuild to cmake, to better be able to deal with the gcc toolchain (which generates much faster code for some of the numeric stuff I'm doing).
Now, I'd like cmake to generate several versions of the output, stuff like one version with sse2, another with x64, and so on. However, cmake seems to work most naturally if you simply have a bunch of flags (say, "sse2_enable", and "platform") and then generate one output based on those platforms.
What's the best way to work with multiple output configurations like this? Intuitively, I'd like to iterate over a large number of flag combinations and rerun the same CMakeLists.txt files for each combination - but of course, you can't express that within the CMakeLists.txt files (AFAIK).

The recommended way to do this is to simply have multiple build directories. From each one you simply call cmake with the required settings.
For example you could do, starting in the base source directory (using Linux shell syntax but the idea is the same):
mkdir build-sse2 && cd build-sse2
cmake .. -DENABLE_SSE2 # or whatever to enable it in your CMakeLists.txt
make
cd ..
mkdir build-x64 && cd build-x64
cmake .. -DENABLE_X64 # or whatever again...
make
This way, each build directory is completely separated from each other.
This allows you to have one directory for Debug, another for Release and another for cross-compiling.

There hasn't been much activity here, so I've come up with a workable solution myself. It's probably not ideal, so if you have a better idea, please do add it!
Now, it's hard to iterate over build configs in cmake because cmake's crucial variables don't live in function scope - so, for instance, that means if you do include_directories(X) the X directory will remain in the include list even after the function exits.
Directories do have scope - and while normally each input directory corresponds to one output directory, you can have multiple output directories.
So, my solution looks like this:
project(FooAllConfigs)
set(FooVar 2)
set(FooAnotherVar b)
add_subdirectory("project_dir" "out-2b")
set(FooVar 5)
set(FooAnotherVar c)
add_subdirectory("project_dir" "out-5c")
set(FooVar 3)
set(FooAnotherVar b)
add_subdirectory("project_dir" "out-3b")
set(FooVar 3)
set(FooAnotherVar c)
add_subdirectory("project_dir" "out-3c")
The normal project dir then contains a CMakeLists.txt file with code to set up the appropriate includes and compiler options given the global variables set in the FooAllConfigs project, and it also determines a build suffix that's appended to all build outputs - any even indirectly included output (e.g. as generated by add_executable) must have a unique name.
This works fine for me.

Related

Scons positively refuses to build into a variant_dir

I have been porting a project that used Make to SCons. Generally, I am pleased by how easy it is to use SCons, relatively to make. However, there is one thing that has resisted several hours of attempts.
The files in my projects are contained into a tree which starts at
ProjectHome. The srouces are in several subdirectories contained in ProjectHome/src/
I have a SConstruct file in ProjectHome which defines the build enviroment and then calls a SConscript
(in ProjectHome) which builds the object files, which are then put into a library in ProjectHome/lib
by SConstruct.
Everything works fine, except that I would like to separate where the .o files are kept from
where the source files are.
So here's what I have
#SConstruct.py
...
# The environment is defined above, no issues
cppobj, chfobj=SConscript('./SConscript.py', 'env', variant_dir='build', src_dir='.', duplicate=False)
env.Install('lib/'+str(Dim)+'D', env.SharedLibrary(target='Grade'+str(n), source=cppobj+chfobj))
and this is for the SConscript.py
#SConscript.py
import platform
import os
import sys
def getSubdirs(abs_path_dir) :
""" returns a sorted list with the subdirectoris in abs_path_dir"""
lst=[x[0] for x in os.walk(abs_path_dir)]
lst.sort()
return lst
Dirs=getSubdirs(os.getcwd()+'/src') # gives me list of the directories in src
CppNodes=[]
ChFNodes=[]
Import('env')
for directory in Dirs[2:3]:
CppNodes+=Glob(directory+'/*.cpp')
ChFNodes+=Glob(directory+'/*.ChF')
# env.Object can work on lists
ChFobj=env.SharedObject(ChFNodes)
# This builder likes to work one at a time
# this build an internal representation of _F.H headers
# so that when an #include in encountered, scons look
# at this list too, and not just what specified by the IncDirs
if len(ChFNodes)==1: # this is ridiculous but having only one ChF file causes troubles
os.system('touch dummyF.ChF')
ChFNodes.append('dummyF.ChF')
ChFHeader=[]
for file in ChFNodes:
ChFHeader+=env._H(source=file)
Cppobj=env.SharedObject(CppNodes)
Return('Cppobj ChFobj')
However, for the life of me, build is ignored completely. I have tried different combinations,
even placing SConscript.py in the build dir, cally SConscript('build/SCoscript.py', 'env',...) you name it: Scons stubbornly refuses to do anything with build. Any help is appreciated. To be clear, it works in creating the libraries. It just that it places the intermediate object files in the src dirs.

(ASDF 3) Is it possible to recursively load systems in subdirectories?

I know about using :modules, but what about when systems get nested? Suppose I have the following structure, relative to some unknown user directory:
foo/
-foo.asd
-bar/
--bar.asd
This could arise, for example, when using Git submodules. How shall I configure the (defsystem) call in foo.asd to load bar as a dependency, without modifying a config file outside of foo/ or demanding particular placement for the foo/ tree itself? Feels like it should be simple.
3 Feb. 2020: From #Svante's answer, it sounds like my question is really 'How do I dynamically ensure that foo/ and bar/ both get into the *source-registry*?' The ASDF manual makes me think this should do the trick:
(asdf:initialize-source-registry
'(:source-registry
(:tree "«absolute-path-to-foo»/")
:inherit-configuration))
though I have not seen an example of that usage.
26 Mar. 2020: The technique above seems to work fine, so I'm closing this question. ASDF 3 is excellent.
ASDF doesn't care about relative locations of .asd files. ASDF systems and their dependencies are completely orthogonal to file/directory structure and oblivious to any source version control.
It just looks in several locations for .asd files. Each such file then may contain definitions for systems. It will generally recurse into the configured folders, so any .asd file in a git submodule would usually also be found.
The definitions, e. g. of components, inside of an .asd file then work relatively from the location of that file.
In your example, if you give a :depends-on ("bar") option to the "foo" system, it would just work, no matter where bar.asd resides (as long as it is somewhere where ASDF finds it).
A bit more awareness would be required if you have several versions of a library. This might happen if you work on "foo" and "bar" at the same time, while a stable version of "bar" is also available, e. g. in a quicklisp dist. Then the lookup order comes into play, but usually your “personal” directories have precedence over “system” directories, so again, it would just work. For more control, you might want to look into qlot.

Randomize Make goals for a target

I have a C++ library and it has a few of C++ static objects. The library could suffer from C++ static initialization fiasco. I'm trying to vet unforeseen translation unit dependencies by randomizing the order of the *.o files during a build.
I visited 2.3 How make Processes a Makefile in the GNU manual and it tells me:
Goals are the targets that make strives ultimately to update. You can override this behavior using the command line (see Arguments to Specify the Goals) ...
I also followed to 9.2 Arguments to Specify the Goals, but a treatment was not provided. It did not surprise me.
Is it possible to have Make randomize its goals? If so, then how do I do it?
If not, are there any alternatives? This is in a test environment, so I have more tools available to me than just GNUmake.
Thanks in advance.
This is really implementation-defined, but GNU Make will process targets from left to right.
Say you have an OBJS variable with the objects you want to randomize, you could write something like (using e.g. shuf):
RAND_OBJS := $(shell shuf -e -- $(OBJS))
random_build: $(RAND_OBJS)
This holds as long as you're not using parallel make (-j option). If you are the order will still be randomized, but it will also depend on number of jobs, system load, current phase of the moon, etc.
Next release of GNU make will have --shuffle mode. It will allow you to execute prerequisites in random order to shake out missing dependencies by running $ make --shuffle.
The feature was recently added in https://savannah.gnu.org/bugs/index.php?62100 and so far is available only in GNU make's git tree.

Qt internationalization and CMake: how to update *.ts and don't lose them

I'm having this CMakeLists.txt in directory with translation files (*.ts):
SET(TRANSLATIONS
lang_de.ts
lang_en.ts
)
FIND_PACKAGE(Qt5LinguistTools)
QT5_ADD_TRANSLATION(QM_FILES ${TRANSLATIONS})
SET(QM_FILES ${QM_FILES} PARENT_SCOPE)
ADD_CUSTOM_TARGET (translations ALL DEPENDS ${QM_FILES})
It builds *.qm files from specified *.ts.
But I want to improve this and get two custom targets, which won't built automatically.
One for appending new strings from sources into ts files, and one for refreshing ts. The last one would update ts from sources and remove obsolete strings from ts.
I've tried to add this after lines above:
ADD_CUSTOM_TARGET (
ts_append
COMMAND QT5_CREATE_TRANSLATION(QM_FILES ${CMAKE_SOURCE_DIR}/src/app ${TRANSLATIONS} OPTIONS -I ${CMAKE_SOURCE_DIR}/src)
)
ADD_CUSTOM_TARGET (
ts_refresh
COMMAND QT5_CREATE_TRANSLATION(QM_FILES ${CMAKE_SOURCE_DIR}/src/app ${TRANSLATIONS} OPTIONS -no-obsolete -I ${CMAKE_SOURCE_DIR}/src)
)
but it seems I can't use QT5_CREATE_TRANSLATION macro inside custom target, isn't it?
Maybe I'm on wrong way, how would you solve this problem: easy updating of ts and don't lose them after make clean?
To solve the make clean problem, add a sub directory (ADD_SUBDIRECTORY(translations)) and add SET_DIRECTORY_PROPERTIES(PROPERTIES CLEAN_NO_CUSTOM 1) to the contained CMakeLists.txt.
See here for an example of that.
For the second part of your question there are two possible ways to do it. Either use FILE(WRITE <filename> "QT5_CREATE_TRANSLATION(QM_FILES ${SOURCE_DIR}/src/app ${TRANSLATIONS} OPTIONS -I ${SOURCE_DIR}/src)") and then use COMMAND ${CMAKE_COMMAND} -DSOURCE_DIR=${CMAKE_SOURCE_DIR} -DTRANSLATIONS=${TRANSLATIONS} <filename> in add_custom_target. I doubt there's a good way of retrieving the contents of QM_FILES though.
The second option is creating two additional sub directories, each with a QT5_CREATE_TRANSLATIONS and a ADD_CUSTOM_TARGET call.

Generating multi-mode executables with a single CMakeLists.txt

Is there a way to create separate executables within one CMakeLists.txt file for the same classes, but for a different intention? This is somewhat like the DEBUG/RELEASE switch, but I need to do a decision at source code level.
Pseudo-CMakeLists.txt:
SET_INTENTION(app1 1)
ADD_EXECUTABLE(app1 main.cxx)
SET_INTENTION(app2 2)
ADD_EXECUTABLE(app2 main.cxx)
Pseudo-Code main.cxx:
if (intention == 1)
//do something different to intention == 2
I tried ADD_DEFINITIONS as preprocessor definitions and #ifdef in source, but CMake seems to interpret the whole file and got all definitions, no matter at what position the definition was added. Splitting the ADD_EXECUTABLES() into two CMakeLists.txt (in subfolders) is not really straight forward and leads to problems when using QT4_WRAP_CPP and QT4_WRAP_UI. I appreciate any ideas / workarounds.
The COMPILE_DEFINTIONS property looks promising:
add_executable(app1 main1.cxx)
get_target_property(APP1_COMPILE_DEFS app1 COMPILE_DEFINITIONS)
set_target_property(app1 PROPERTIES COMPILE_DEFINITIONS "${APP1_COMPILE_DEFS};INTENTION=1")
add_executable(app2 main2.cxx)
get_target_property(APP2_COMPILE_DEFS app2 COMPILE_DEFINITIONS)
set_target_property(app2 PROPERTIES COMPILE_DEFINITIONS "${APP2_COMPILE_DEFS};INTENTION=2")
Then use preprocessor #if INTENTION=1 and so forth in your source files. If you're not using a recent CMake, you may need to mess with the COMPILE_FLAGS property instead.

Resources