We have an autotools project that has a mixture of unit and integration tests, all of which run via 'make check'. This isn't ideal, as some of the integration tests take a while, and have all sorts of dependencies (database, etc.)
I'd like to separate the integration tests and assign them their own make target. That way, unit tests can still be run often (via make check), and the integration tests can be run as needed in a similar fashion.
Is there a straightforward (or otherwise) way to add an additional make target?
NOTE: I should probably also add that this is a large project, so editing/maintaining every makefile by hand is not desirable. I'd like to do it the 'autotools way' if possible.
-- UPDATE 1 --
I've attempted Jon's solution, and it's a step closer, but not quite there. I still have a couple of issues:
1) Recursion - I'm OK with modifying the makefile.am in the root of the build tree, as well as any directory that contains the tests, but it seems like there should be a way to do this where I don't have to change every Makefile.am in the hierarchy. (the check target works this way, after all)
2) .PHONY - I keep getting messages about .PHONY being redefined. Which is understandable, because it's being set by another package (specifically, doxygen). How do I make the two play nice together?
In your am files, all make syntax is passed into the generated Makefile. So if you want a new target just create it like you would in a Makefile and it will appear in the auto generated Makefile. Put the following at the bottom of your am files.
integration-tests: prerequisites....
commands to run test
.PHONY: integration-tests
Since there haven't been any more responses, I'm going to answer with my solution.
I solved the recursion issue by eliminating the recursion altogether. Using this page
as a guide, I switched the entire project from recursive make to non-recursive make. I then cloned the non-recursive check-related targets (check, check-am, check-TESTS, etc.) into a new set of targets for the integration tests. So far, this works extremely well.
Note: you may be wondering why I didn't just clone the recursive targets instead. Quite frankly, I couldn't find them. Either I didn't know where to look (the rules weren't in the generated Makefile) or something is happening implicitly, and I don't understand autotools well enough to follow it.
As for the issue with .PHONY being redefined, I still haven't found a solution, other than to conditionally exclude the other definition when I'm doing the integration tests.
Related
I stumbled upon an issue when I recently switched to VSCode as editor.
I have several projects that have a full (medium-complex+) autotool
setup and they all work fine. However I discovered that the makefile plugin for VSCode in order to initialize itself (and finding all dependencies and targets) starts by running
make --dry-run --always-make
as first time initialization. This throws the makefile (or actually the
re-config) into an endless loop re-running "configure" (since the targets are never resolved to disk).
I have also confirmed this behavior with the smallest possible autoconf/automake
setup. I can also kind-of understand why this happens (and it seems make
have an internal way to discover this exact situation with the special
variable MAKE_RESTARTS that could possible be used to detect a cyclic
behavior)
Is there a known best-practice workaround ? or is it even a reasonable expectations that these two options in combination should work? (Good to have a second opinion before I go down the rabbit-hole of reminding myself of all the details I forgot about the magical land of autotools)?
I am currently writing a text with R bookdown and asked two friends to read my text and give comments, corrections and general feedback. My source files for the text are stored on GitHub and I would like my collaborators to make changes in the files (one for each chapter) with the help of git. However, none of us are really experts on git. This makes it hard to figure out what a suitable workflow is.
For now, we decided that each one of them creates himself a branch so that he does not directly push into the master branch. After I have read their changes I would like to decide what I merge into the master branch and what not. So far, it looks like each change needs to be in a separate commit because I am not able to merge single lines from a specific commit (not sure if that is at all possible). However, this seems like a lot of annoying and unnecessary commits to create. So, I guess I am looking for a way to avoid that and/or general pointers towards a good workflow for such kind of projects.
A useful command will be git cherry-pick, it allows you to select specific commits from a branch.
A general good practice is that commits should be self contained (if applied alone they make sense) and they target a specific feature (in the use case mentioned, that could be a paragraph or a section or a chapter).
In the end, if you would like to apply only specific changes of a commit, that would have to happen manually, someone has to decide which parts to apply and which not. A commit can be edited using git rebase -i <branch name> before being merged. This question might also be useful.
I finally found what worked for me in here. Basically, on my master branch I had to use
git merge --no-commit --no-ff branch-to-merge
This will merge all changes into my master branch but does not immediatly commit the changes so that they can still be staged/unstaged. Then, I can decide what line change to include by staging the line changes I want to keep and discard all other line changes. Finally, I commit all staged line changes et voilà, that's what I wanted to get.
Sidenote: I am using gitkraken and as a beginner with git I enjoy using the GUI but the merge part with the options "no-commit" and "no-fast-forwarding" had to be done via the git console (at least I could not find a way to to that using the GUI). Choosing which lines to stage and which to discard is then an easy task via the GUI.
I am unfamiliar with good coding practices in Julia. Working in Jupyter notebook in python I typically put all import statements in one cell at the top of the file, which helps me easily see what the dependencies are.
Is it advisable to do the same with 'using' statements in Julia (I'm also working in Jupyter notebook for now)?
Yes you should. There is only one case I know where you might not want to do so, and that is if there is a large module that is only used under rare conditions -- for example, if you run a program that can produce a plot, or do similar optional functions with slowly loading modules, in some rare usage scenarios, but normally will never use that slow-to-load module.
Even then, you can easily wind up with errors at run time due to redefinitions of functions that have been already been run before the newly loaded module redefines them.
we have a larger dataset and have several preprocessing scripts.
These scripts alter data in place.
It seems when I try to register it with dvc run it complains about cyclic dependencies (input is the same as output).
I would assume this is a very common use case.
What is the best practice here ?
Tried to google around but i did not see any solution to this (besides creating another folder for the output).
Usually, we split input and output into separate files rather than modify everything in place, not only for the separation of concerns principles but also to make it fit with tools like DVC.
Hope you can try this way instead.
I have a rather big library with a significant set of APIs that I need to expose. In fact, I'd like to expose the whole thing. There is a lot of namespacing going on, like:
FooLibrary.Bar
FooLibrary.Qux.Rumps
FooLibrary.Qux.Scrooge
..
Basically, what I would like to do is make sure that the user can access that whole namespace. I have had a whole bunch of trouble with this, and I'm totally new to closure, so I thought I'd ask for some input.
First, I need closurebuilder.py to send the full list of files to the closure compiler. This doesn't seem supported: --namespace Foo does not include Foo.Bar. --input only allows a single file, not a directory. Nor can I simply send my list of files to the closure compiler directly, because my code is also requiring things like "goog.assers", so I do need the resolver.
In fact, the only solution I can see is having a FooLibrary.ExposeAPI JS file that #require's everything. Surely that can't be right?
This is my main issue.
However, later the closure compiler, with ADVANCED_OPTIMIZATIONS on, will optimize all these names away. Now I can fix that by adding "#export" all over the place, which I am not happy about, but should work. I suppose it would also be valid to use an extern here. Or I could simply disable advanced optimizations.
What I can't do, apparently, is say "export FooLibrary.*". Wouldn't that make sense?
Finally, for working in source mode, I need to do goog.require() for every namespace I am using. This is merely an inconvenience, though I am mentioning because it sort of related to my trouble above. I would prefer to be able to do:
goog.requireRecursively('FooLibrary')
in order to pull all the child namespaces as well; thus, recreating with a single command the environment that I have when I am using the compiled version of my library.
I feel like I am possibly misunderstanding some things, or how Closure is supposed to be used. I'd be interested in looking at other Closure-based libraries to see how they solve this.
You are discovering that Closure-compiler is built more for the end consumer and not as much for the library author.
If you are exporting basically everything, then you would be better off with SIMPLE_OPTIMIZATIONS. I would still highly encourage you to maintain compatibility of your library with ADVANCED_OPTIMIZATIONS so that users can compile the library source with their project.
First, I need closurebuilder.py to send the full list of files to the closure compiler. ...
In fact, the only solution I can see is having a FooLibrary.ExposeAPI JS file that #require's everything. Surely that can't be right?
You would need to specify an --root of your source folder and specify the namespaces of the leaf nodes of your file dependency tree. You may have better luck with the now deprecated CalcDeps.py script. I still use it for some projects.
What I can't do, apparently, is say "export FooLibrary.*". Wouldn't that make sense?
You can't do that because it only makes sense based on the final usage. You as the library writer wish to export everything, but perhaps a consumer of your library wishes to include the source (uncompiled) version and have more dead code elimination. Library authors are stuck in a kind of middle ground between SIMPLE and ADVANCED optimization levels.
What I have done for this case is maintain a separate exports file for my namespace that exports everything. When compiling a standalone version of my library for distribution, the exports file is included in the compilation. However I can still include the library source (without the exports) into a project and get full dead code elimination. The work/payoff balance of this though must be weighed against just using SIMPLE_OPTIMIZATIONS for the standalone library.
My GeolocationMarker library has an example of this strategy.