So allegedly, the configuration tool for Qt went through some changes, necessary to be able to make more streamlined Qt builds, a.k.a "Qt Lite". However, there doesn't seem to be any documentation about how to use that feature, or at least I don't find any, and looking at the comments from the release announcement, others can't neither.
What's more, the changes are definitely in there, judging by the fact that the configuration that I've been using for the last couple of years fails in a bunch of ways. I am not sure how up-to-date the built in help is, since the last time I tried using it for guidance, it turned out it was largely outdated and contained options that were no longer supported.
So it would be nice if someone could shed some light on what has changed and how, and how to configure for "Lite" builds. And especially on module and feature dependencies, because I think we'd all like to avoid wasting time on builds that will start building despite an improper configuration that omits necessary dependencies just to have it inevitably fail and result in nothing but a waste of time.
Per to the changelog:
The configuration system has been rewritten almost from scratch. This improved the consistency between builds on Unix and Windows, but some subtle unintended behavior changes are also possible. Also, some obsolete options have been entirely removed and will now cause errors.
It is not permissible any more to manually #define QT_NO_
anywhere. Instead, configure's -no-feature-* options must be used.
Note that this does not apply to defines which modify behavior rather
than entirely removing features.
The -no-feature-* option family was integrated with the rest of the
configuration system. Numerous existing features were made optional,
and build problems in various reduced configurations were fixed.
This is an ongoing effort known as "Qt Lite".
Features for -no-feature-* lists are in qtbase\src\corelib\global\qfeatures.txt.
All features are enabled by default.
More information can be found in the Qt Lite Overview Presentation and its slides.
You can also use the new UI Tool which is known as Qt Configuration Tool and which is a part of Qt for Embedded Devices package - see its documentation. The configuration tool is available for commercial Qt customers only at the moment (Qt 5.8).
The changes that are behind my failed configuration:
there is no longer the option to specify whether sql support is built-in or plug-in, so the format is now just -sql-<driver>, the documentation is still not updated and lists the old format - -<option>-sql-<driver>.
the -l option to add a specific library has been removed, which is turning out to be problematic in multiple areas.
Edit: Also, this blog entry just posted on doing lite builds might be useful.
Everything that describes what the new configuration system understands is given in the configure.json files scattered around Qt modules. The configure tool uses these files to build a list of command line arguments it understands.
Without the use of other tools, to learn about Qt features you need to inspect these json files and choose the features/options you wish turned on or off.
Sub Configurations
These act as includes, and refer to the configure.json file in a given folder. E.g. qtbase/configure.json includes qtbase/src/corelib/configure.json, qtbase/src/network/configure.json etc.:
"subconfigs": [
"src/corelib",
"src/network",
[...]
],
Explicit Command Line Options
The commandline/options value lists the configure options a given Qt module understands. These options are separate from the feature system, although they may be used for convenience to provide shorthand aliases that control features. For example, in qtbase/configure.json, we have:
{ "commandline": { "options": { "accessibility": "boolean", [...] }
This command line option controls the identically named accessibility feature. It is more convenient to use than dealing with the feature system's option [-no]-feature-accessibility. The following pairs have identical effects:
-accessibility or -feature-accessibility
-no-accessibility or -no-feature-accessibility
Values:
boolean options are given to configure as -option and -no-option, meaning true and false, respectively.
all other options are given as -option value.
Feature Options
The features value lists the features available in a given module. The features are effectively booleans. They are all enabled by default, subject to passing configuration tests that enable them.
To control a feature foo:
-no-feature-foo disables the feature. E.g. to disable the iconv feature, you'd do configure -no-feature-iconv [...].
-feature-foo enables the feature and ensures that it is available. This will cause an error if a configuration test for the feature fails. It's useful in build systems that build a particularly configured Qt along with your application: it ensures that the features your code depends on will be available.
Failing Builds
Generally speaking, no matter what combination of feature selections you provide, if configure doesn't fail, the build is supposed to succeed.
we'd all like to avoid wasting time on builds that will start building despite an improper configuration
The configure tool will detect any invalid configurations. If configure succeeds yet the build fails, it's a Qt bug and you should report it.
Related
I just started to work on a large project that uses SBT for building. I got a new computer with Java 8 installed, but the rest of the team still uses Java 7. That's not a problem as far as the code goes because we're all set to generate v7 byte-code.
The problems arise when attempting to publishing the project using the publishLocal action. Please keep in mind though that I am very new to SBT and some things I say/assume may not be accurate.
We use sbt 0.13. When I run the command sbt publishLocal, it runs the doc action, which in turn runs javadoc to generate the documentation. Since I have java 1.8 installed, it uses the corresponding version of javadoc, which let's be honest is a real pain in the neck, complaining about every single missing #return or #param, self-closing elements (e.g. "<p/>") and such, and returning a non-zero value because of this, thus making the publication fail. However, as I mentioned, the project is fairly big and, although it would be better to complete the javadoc documentation, it's not feasible at the moment.
Luckily, javadoc 8 provides an option do disable the pedantry: -Xdoclint:none will make it quiet about pretty much anything, allowing me to run the publish action by adding it to the javacOptions.
However, as I said, the other team members are still using java 7, and, unfortunately, javadoc 7 does not support that option, so if I push the build.sbt file with this option it will fail on other machines.
So now I'm wondering what I can do. The way I see it, there's a number of options, none of which seems "simple enough" to fix that stupid problem:
downgrade locally to java 7 (not a big fan of having two concurrent versions lying around)
have all other team members upgrade to java 8 (pain for them)
fix all javadoc problems in the whole project (pain for everyone)
Hopefully, there's another option I'm missing that would allow me, for instance, to set the javadoc options based on the java version? Or anything else that doesn't require touching anything else than the build.sbt file...
Thanks!
David
Yes, you can set the Javadoc options based on the Java version:
javacOptions in Compile ++=
sys.props("java.version").split('.') match {
case Array("1", n, _*) if n.toInt <= 7 =>
Seq()
case _ =>
Seq("-Xdoclint:none")
}
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
What is a best practices to write large software projects in OCaml?
How do you structure your projects?
What features of OCaml should and should not be used to simplify code management? Exceptions? First-class modules? GADTs? Object types?
Build system? Testing framework? Library stack?
I found great recommendations for haskell and I think it would be good to have something similar for OCaml.
I am going to answer for a medium-sized project in the conditions that I am familiar with, that is between 100K and 1M lines of source code and up to 10 developers. This is what we are using now, for a project started two months ago in August 2013.
Build system and code organization:
one source-able shell script defines PATH and other variables for our project
one .ocamlinit file at the root of our project loads a bunch of libraries when starting a toplevel session
omake, which is fast (with -j option for parallel builds); but we avoid making crazy custom omake plugins
one root Makefile contains all the essential targets (setup, build, test, clean, and more)
one level of subdirectories, not two
most subdirectories build into an OCaml library
some subdirectories contain other things (setup, scripts, etc.)
OCAMLPATH contains the root of the project; each library subdirectory produces a META file, making all OCaml parts of the projects accessible from the toplevel using #require.
only one OCaml executable is built for the whole project (saves a lot of linking time; still not sure why)
libraries are installed via a setup script using opam
local opam packages are made for software that it not in the official opam repository
we use an opam switch which is an alias named after our project, avoiding conflicts with other projects on the same machine
Source-code editing:
emacs with opam packages ocp-indent and ocp-index
Source control and management:
we use git and github
all new code is peer-reviewed via github pull requests
tarballs for non-opam non-github libraries are stored in a separate git repository (that can be blown away if history gets too big)
bleeding-edge libraries existing on github are forked into our github account and installed via our own local opam package
Use of OCaml:
OCaml will not compensate for bad programming practices; teaching good taste is beyond the scope of this answer. http://ocaml.org/learn/tutorials/guidelines.html is a good starting point.
OCaml 4.01.0 makes it much easier than before to reuse record field labels and variant constructors (i.e. type t1 = {x:int} type t2 = {x:int;y:int} let t1_of_t2 ({x}:t2) : t1 = {x} now works)
we try to not use camlp4 syntax extensions in our own code
we do not use classes and objects unless mandated by some external library
in theory since OCaml 4.01.0 we should prefer classic variants over polymorphic variants
we use exceptions to indicate errors and let them go through happily until our main server loop catches them and interprets them as "internal error" (default), "bad request", or something else
exceptions such as Exit or Not_found can be used locally when it makes sense, but in module interfaces we prefer to use options.
Libraries, protocols, frameworks:
we use Batteries for all commodity functions that are missing from OCaml's standard library; for the rest we have a "util" library
we use Lwt for asynchronous programming, without the syntax extensions, and the bind operator (>>=) is the only operator that we use (if you have to know, we do reluctantly use camlp4 preprocessing for better exception tracking on bind points).
we use HTTP and JSON to communicate with 3rd-party software and we expect every modern service to provide such APIs
for serving HTTP, we run our own SCGI server (ocaml-scgi) behind nginx
as an HTTP client we use Cohttp
for JSON serialization we use atdgen
"Cloud" services:
we use quite a lot of them as they are usually cheap, easy to interact with, and solve scalability and maintenance problems for us.
Testing:
we have one make/omake target for fast tests and one for slow tests
fast tests are unit tests; each module may provide a "test" function; a test.ml file runs the list of tests
slow tests are those that involve running multiple services; these are crafted specifically for our project, but they cover as much as possible as a production service. Everything runs locally either on Linux or MacOS, except for cloud services for which we find ways to not interfere with production.
Setting this all up is quite a bit of work, especially for someone not familiar with OCaml. There is no framework taking care of all that yet, but at least you get the choice of the tools.
OASIS
To add to Pavel answer:
Disclaimer: I am the author of OASIS.
OASIS also has oasis2opam that can help to create OPAM package quickly and oasis2debian to create Debian packages. This is extremly useful if you want to create a 'release' target that automate most of the tasks to upload a package.
OASIS is also shipped with a script called oasis-dist.ml that creates automatically tarball for upload.
Look all this in https://github.com/ocaml.org.
Testing
I use OUnit to do all my tests. This is simple and pretty efficient if you are used to xUnit testing.
Source control/management
Disclaimer: I am the owner/maintainer of forge.ocamlcore.org (aka forge.o.o)
If you want to use git, I recommend to use github. This is really efficient for review.
If you use darcs or subversion, you can create an account on forge.o.o.
In both case having a public mailing list where you send all commit notification is a must have, so that everyone can see them and review them. You can use either Google groups or a mailing list on forge.o.o.
I recommend to have a nice web (github or forge.o.o) page with OCamldoc documentation build everytime you commit. If you have a huge code base this will help you to use the OCamldoc generated documentation right from the beginning (and fix it quickly).
I recommend to create tarballs when you reach a stable stage. Don't just rely on checking out the latest git/svn version. This tip has saved me hours of work in the past. As said by Martin, store all your tarballs in a central place (a git repository is a good idea for that).
This one probably doesn't answer your question completely, but here is my experience regarding build environment:
I really appreciate OASIS. It has a nice set of features, helping not only to build the project, but also to write documentation and support test environment.
Build system
OASIS generates setup.ml file from the specification (_oasis file), which works basically as a building script. It accepts -configure, -build, -test, -distclean flags. I quite used to them while working with different GNU and other projects that usually use Makefiles and I find it convenient that it is possible to use all of them automatically here.
Makefiles. Instead of generating setup.ml, it is also possible to generate Makefile with all options described above available.
Structure
Usually my project that is built by OASIS has at least three directories: src, _build, scripts and tests.
In the former directory all source files are stored in one directory: source (.ml) and interface (.mli) files are stored together. May be if the project is too large, it is worth introducing more subdirectories.
The _build directory is under the influence of OASIS build system. It stores both source and object files there and I like that build files are not interfered with source files, so I can easily delete it in case something goes wrong.
I store multiple shell scripts in the scripts directory. Some of them are for test execution and interface file generation.
All input and output files for tests I store in a separate directory.
Interfaces/Documentation
The use of interface files (.mli) has both advantages and drawbacks for me. It really helps to find type errors, but if you have them, you have to edit them as well when making changes or improvements in your code. Sometimes forgetting this causes nasty errors.
But the main reason why I like interface files is documentation. I use ocamldoc to generate (OASIS supports this feature with -doc flag) html pages with documentation automatically. In my opinion it is enough to write comments describing each function in the interface and not to insert comments in the middle of code. In OCaml functions are usually short and concise and if there is a necessity to insert extra comments there, may be it is better to split the function.
Also be aware of -i flag for ocamlc. The compiler can automatically generate interface file for a module.
Tests
I didn't find a reasonable solution for supporting tests (I would like to have some ocamltest application), that's why I am using my own scripts for executing and verifying use cases. Fortunately, OASIS supports executing custom commands when setup.ml is run with -test flag.
I don't use OASIS for a long time and if anyone knows any other cool features, I would like also to know about them.
Also, it you are not aware of OPAM, it is definitely worth looking at. Without it installing and managing new packages is a nightmare.
I am looking for a way to create a minimal static build of Qt 5.1. By minimal, I mean a build with only the basic set of widget classes available (including 2D graphics acceleration through an OpenGL canvas/widget). I am aiming for a static build on Windows 7 32-bit using MinGW-builds GCC 4.8.1 as well as their bundle of MSYS and tools.
I have downloaded the entire source tree, and have met some problems with compiling due to the dependency of packages such as OpenSSL and ICU (the latest OpenSSL did not compile). I wanted to see, if I could avoid dependencies of these packages and, at the same time, achieve a minimal static build of Qt 5.1 so my final executable is not gigantic in size.
I took a look at this list:
http://download.qt-project.org/official_releases/qt/5.1/5.1.0/submodules/.
Q1: Which of these modules, do I need to select to have a basic Qt build without all the fancy stuff?
Q2: Are these modules interdependent on each other in some way (I assume they are all dependent on "qtbase")?
Q3: Do I use the names from that list, (for example: "qtxmlpatterns", "qtx11extras", "qtwebkit", etc.), to deselect them in the configuration of my Qt build?
It would also be nice, with an explanation of the various submodules of Qt 5? If someone could point to any links or docs with illuminating words on this subject, that would also be great.
I shared this same goal. I wanted to "minify" Qt, building only a small subset of the libraries/DLL(s) and omitting the rest.
I did succeed, at least on Mac OS X. I assume that this approach will work on other platforms, too.
I thought that the trick would be to find some kind of exclusion flags for the "configure" script, but that turned out not to be the case.
Here is what I did:
Download the source distribution (in my case, qt-everywhere-opensource-src-5.1.1). of course, extract it all.
Edit the following three "pro" files. these files are in a qt-specific format (but plain text). They are platform-independent project files that Qt (via qmake) uses to generate makefiles.
qtbase/src/src.pro
qtbase/src/plugins/plugins.pro
qtbase/examples/examples.pro
All the changes that I made to the "pro" files were deletions. I deleted references to: dbus, ipc, network, qtconcurrent, sql, xml, sqldrivers, qdbusxml2cpp, qdbuscpp2xml, src_dbus, src_concurrent, src_sql, src_network.
That last part may sound scary. It really wasn't.
The pro files contain a list of module names, plugin names, and directory names. It seemed fairly straightforward how to delete (for example) all SQL-related lines.
Step 3.
The usual configure/make/install. However, note the "module-qtbase" argument passed to make:
./configure -prefix ~/my_test_dir/
make -j2 module-qtbase
make -j2 module-qtbase-install_subtargets
Success. This built and installed only the essentials (QtCore, QtGui, QtOpenGL, QtWidgets) and skipped all the supporting dylibs like QtConcurrent, QtMultimedia, QtSql, and QtWebKit.
ICU can link statically, and you can customize it (see docs) to reduce size. By default the data is also large, I don't know how much data you need. You could include only enough data to run in one language.
Only part of an answer.
There's no need to disable Qt modules. Each application that uses Qt decides what modules it uses and links with. A basic console-only application may use the core module only. That's what you'd use when creating, say, a web service in Qt.
I've been asked to customise the layout of the GNOME 3 desktop. Apparently the way to do that is by writing an "extension".
I've managed to do some of the things I wanted to do, but I feel utterly starved of information. I cannot find any useful documentation anywhere. I've wasted entire days of my life frantically googling every imaginable search term in a desperate attempt to find useful information.
The GNOME website has hundreds of extensions for download. These are not trivial 3-liners; they're sophisticated pieces of code. It defies belief that anybody could write these without documentation explaining how to do it.
Please, can somebody tell me where the actual documentation is? So far, the best I've managed to do is take apart existing extensions trying to track down the magic command that does the specific bit I'm interested in. (Not an easy task!)
Command names, object paths, example programs, anything would be helpful!
I have recently dug into it myself. The documentation is usually sparse or outdated. Here are some sources which helped me to get started (and through development):
Basic Stuff
Step-by-step tutorial (Gnome 3.4)
Unofficial documentation for the JavaScript bindings of many libraries
The sources of the gnome-shell's JavaScript bindings
Explanation of the St (Shell Toolkit) Ui-Toolkit components.
Some unofficial guidelines to get your extension on extensions.gnome.org
Since the documentation is nearly unavailable (or up to date), you'll need to do a lot of source-reading. I linked the gnome-shell sources above (the JavaScript part) which is a good start when diving into parts that are not covered by the In-official documentation (which is the most complete thing you'll find).
What's also particular helpful is checking extensions.gnome.org for extensions which do similar things to what you want to create, and look at their sources (most of them are open-source on GitHub or Bitbucket. You can also install them and find the sources under ~/.local/share/gnome-shell/extensions/).
When searching for something to use or more documentation on a particular function, you can also consult manuals for bindings in different languages (thought the parameters and return-values might not match).
Last but not least, here is some debugging advice:
LookingGlass is not particularly helpful. It only shows one line of an exception (the description) and only if they occur at startup time (when your extension is first started).
For full StackTraces and runtime-exceptions, consult the ~/.xsession-errors-file. It might be very long and bloated. I use this handy script to read it:
# Grabs the last session-errors from the current X11 session.
# This includes full Stack-Trace of gnome-shell-extension errors.
# See https://live.gnome.org/GnomeShell/Extensions/StepByStepTutorial#lookingGlass
tail -n100 ~/.cache/gdm/session.log | less
Note that since Gnome 3.6, if you are using gdm as display manager, the current session log is the file ~/.cache/gdm/session.log.
On some newer distros using systemd, you can get the error logs with:
journalctl -f /usr/bin/gnome-session
For debugging the prefs-part of your extension, you can launch the preferences by using the gnome-shell-extension-prefs-tool from a terminal, to see any exception-output on the console (you can also call the tool like gnome-shell-extension-prefs [uuid], to directly show your extensions preferences).
Since there is currently no real way of debugging with breakpoints (there is, but it's tricky), you can log on the console for quick checking, use the print()-function. You will see the output as mentioned above (either in the sessions-error file or on the terminal when starting gnome-shell-extension-prefs-tool).
Although it might be a little hard to get into it, the extension framework is quite powerful. Have fun!
I wrote a Blog-Post with somewhat greater detail, which can be found here: Making Gnome-Shell Extensions
An extensive list of references can be found on the Gnome Developer - API Reference page.
I used the following for my extension, but your use may vary:
GTK+ 3
GTK+ is the primary library used to construct user interfaces in GNOME applications. It provides user interface controls and signal callbacks to control user interfaces.
GDK 3
GDK is an intermediate layer which isolates GTK+ from the details of the windowing system.
Clutter
Clutter is a GObject based library for creating fast, visually rich, graphical user interfaces.
GObject Introspection
GObject Introspection is striving to provide a middleware layer between (GObject based) C libraries and language bindings.
Shell
Shell Reference Manual
St
St - Shell Toolkit - is the GNOME Shell's custom Clutter-based toolkit that defines useful actors. Some of these actors, such as StBoxLayout and StBin implement various layout options.
Icon Theme Specification
This freedesktop.org specification describes a common way to store icon themes.
NOTE: These last two are very helpful in finding visual element parameters!
PyGTK
PyGTK is GTK+ for Python. This reference contains a chapter for each Python PyGTK module (that corresponds to the underlying GTK+ library) containing the class descriptions.
PyGObject
PyGObject is a Python extension module that gives clean and consistent access to the entire GNOME software platform through the use of GObject Introspection. Specifically speaking, it is Python Bindings for GLib, GObject, GIO and GTK+.
This reference contains a chapter for each PyGObject module containing the class descriptions.
The documentation is on:
https://gjs.guide/extensions/
For the documentation of libraries:
https://gjs-docs.gnome.org/
More details on https://gjs.guide/extensions/overview/architecture.html
The other stuff you might want to check are
https://gitlab.gnome.org/GNOME/gnome-shell/blob/main/js/ui/popupMenu.js
https://gitlab.gnome.org/GNOME/gnome-shell/blob/main/js/ui/dialog.js
https://gitlab.gnome.org/GNOME/gnome-shell/blob/main/js/ui/modalDialog.js
https://gitlab.gnome.org/GNOME/gnome-shell/blob/main/js/ui/panelMenu.js
https://gitlab.gnome.org/GNOME/gnome-shell/tree/main/js
https://gitlab.gnome.org/GNOME/mutter
You can browse under js/ for more code to be reused.
You might also want to check https://gi.readthedocs.io/en/latest/index.html
Question:
I could not find anything under https://gjs-docs.gnome.org/ except some CSS and Javascript documentation ?!?!
Answer:
You have to first enable the docs to use them. Here, you will be mainly looking for:
clutter
meta
shell
st
Create a file like:
echo '{"docs":"clutter9~9_api/clutterx118~8_api/gobject20~2.66p/meta9~9_api/shell01~0.1_api/st10~1.0_api","hideIntro":"1"}' > devdocs.json
Import this file to https://gjs-docs.gnome.org/settings
Now you will be able to visit:
https://gjs-docs.gnome.org/shell01~0.1_api-global/
https://gjs-docs.gnome.org/shell01~0.1_api/
https://gjs-docs.gnome.org/meta9~9_api/
https://gjs-docs.gnome.org/st10~1.0_api/
https://gjs-docs.gnome.org/clutter9~9_api/
https://gjs-docs.gnome.org/clutter9~9_api-actor/
Warning: The version on the devdocs.json file is hardcoded. It will be outdated in no time, so you might want to check the version. The point is - you can not access docs until you enable them.
P.S. I know, this is a mess. This is how they did it.
Doxygen is a bit slow - it takes about a couple of minutes to process my whole project, so for small incremental changes this is longer than actually building the rest of my code. There are thousands of files without any documentation so I guess it is spending most of its time processing them. Is there any way to get it to skip files without any documentation?
What about getting it to only process changed files?
From Doxygen documentation:
How can I exclude all test directories
from my directory tree?
Simply put an exclude pattern like
this in the configuration file:
EXCLUDE_PATTERNS = /test/
So, you should be using patterns to exclude files. It's been a long time since I've used Doxygen, but i don't remember any option to process only changed files.
I found that turning off the option SEARCH_INCLUDES made a big difference. It was looking through the whole platform SDK and include paths for the compiler which were not documented anyway and would not appear in the generated documentation.
There is a DOT_NUM_THREADS options which may increase the performance on multicore machines. Unfortunately doxygen itself is just single threaded.
Another approach would be to organize your code into modules run for each module a separate doxygen instance and link the resulting tags together: http://www.doxygen.nl/manual/external.html
Doxygen is good at finding connections between files, either changed or not. But Doxygen does not remember informations about unchanged files, so it must process the whole codebase each time.
May be a solution would be to organize the project such that never changed files belong to one module which is excluded from Doxygen scope and whose documentation is already available. Then it would be possible to tell Doxygen to link newly built documentation to this existing module documentation.
Going further, it would also be possible to make Doxygen running module by module, processing only changed modules and a top level documentation which links to all module documentations.
I don't think having Doxygen run on a normal dev cycle is a good idea. Our Doxygen build runs as part of our Continuous Integration server's responsibilities.
That said, there are some benefits of running doxygen every build to catch missing docs.
So I would trim the doxygen config for dev builds removing diagrams, and even stop apple importing it into xcode.