A question about composing sbt projects:
I forked a library project (https://github.com/allenai/pipeline) and wish to hack on it vis-a-vis my project that uses it. If I put them in the same multi-project build ― really my preferred option over publishing locally and consuming ― sbt will not read the project dir of the nested fork (where .scala files and plugin definitions originally reside).
E.g. I try, in build.sbt:
lazy val pipeline = (project in file("pipeline"))
lazy val githubCruncher = (project in file("."))
.dependsOn(pipeline)
And I get errors indicating directory pipeline/project is being ignored; the build.sbt itself of the nested project is being read, but sbt does not find what is defined alongside to it in its sibling project directory (and hence fails to load build definition).
So, must I squash the two projects into a single build.sbt or can I somehow elegantly compose the two without that? I am asking because this has also been a problem on other projects I work on where I ended up squashing each and every build definition minutia into one big monolith ― even though I'd prefer distributing some of the sub-projects as independent projects aside my own dev configuration.
According to my experience looks like everything inside the project directory of a project being pulled into a multi-project build ― needs to move to project under the root of the multi-project build, whereas the build.sbt of the pulled-in project does not have to be squashed ― a reference to it from from the top level build.sbt will work (unless that reference is followed by definitions for it right there in the top level build.sbt).
Sbt plugins coming from the pulled-in project then become applied to the unified multi-project build, and need to be disabled with disablePlugins for other sub-projects, if or where appropriate or desireable.
Related
I have a Qt-application. It has a .pro file with TEMPLATE = app. One of the project's subfolders is a git-submodule to another Qt project: a collection of libraries, which has it's own .pro file with TEMPLATE = subdirs.
Graphically it looks like:
project/
app.pro (TEMPLATE = app)
stuff/
libs/ <-- git-submodule
libs.pro (TEMPLATE = subdirs)
lib1/
lib1.pro (TEMPLATE = lib)
lib2/
lib2.pro (TEMPLATE = lib)
libs as a standalone project compiles well and produces .lib files.
But in this case I want somehow include libs.pro to a project as a subdir although app.pro's TEMPLATE is not subdirs but app. Maybe that is why my attempts to write something like SUBDIRS += askelib to app.pro had no effect.
All in all my aim is to get .lib files inside build folder of app.pro.
I emphasize that libs is a git-submodule because nothing should be changed inside libs project to achieve my goal.
I know that it probably should work if I change app.pro's TEMPLATE to subdirs. But that's not what I really want to do because it will make things more difficult since project hierarchy then will achieve another nesting level:
subdirs_proj/
app/
libs/
instead of
app/
libs/
EDIT:
To clerify my reasons:
My aim is to make a project tree as clearer as it can be. Like you clone a project from github, enter into it's directory and see app.pro at the top level. And everything is clear, easy and beautiful. And not like you see a strange subdirs.pro at the top but the actual project is in app subdirectory and you also have posibility to confuse main application subfolder with a library subfolder in case if their names are not so obvious as app and libs but something like torpedo and helios. Hope my thought is clear :)
You already have your answer: make the top-level project a subdir project.
I do not understand why you want to avoid that and why you find it confusing.
IMHO, having an app project that has subdirs is more confusing than having a subdir project that has subdirs.
And I don't think removing a folder level compensate for having subdirs in an app .pro. Think about a new developer coming on the project, if he sees TEMPLATE=app he will assume you only build a single project, but it is not. Meaning that your project is not "clear, easy and beautiful" and completely violates the principle of least astonishment.
I regularly have projects that have the following architecture:
project-a/project-a.pro (subdirs)
/cli-app/cli-app.pro (app)
/gui-app/gui-app.pro (app)
/core-lib/core-lib.pro (lib)
/3rd-party/3rd-party.pro (subdirs)
/3rd-party/somelib/somelib.pro (lib)
I find it clearer than messing with the project type to remove a folder level. And if you are afraid developers will not know what each sub folder is, maybe you should throw some README files explaining what is what.
Also you can take a look at the Qt project itself, it has many .pro files and not once you have an app project that contains subdirs. And I think it is rather clear, in particular for such a big project.
We have a multi module project consisting of two modules, modA and modB.
modA depends on modB.
modB in turn depends on a list of libraries (libA and libB) where we also have the source code. This sources have already been adapted by us.
At last, libB and libC are independend from each other, but depend on a third library, libC.
What I want to have is a setup, where the three libraries (which are in principle also a multi-module SBT project) can just be "included" in the top level project.
The point here is also that these libraries can be re-used for other projects, too, so the changed sources should not belong to this super project only.
Currently I tried to solve it by including the library as GIT submodule.
Unfortunately SBT does not (seem to) support hierarchical sub modules, so I cannot really just have a second, also multi-module SBT file for all libraries which just gets included in the "super-super" project.
This current setup is clearly not the SBT way.
What is the intended method of solving this?
Just adapting the library separately and re-using it just as JAR file in the super project is possible, but clumsy, because the using project(s) are the main reason to hack the library, so it would be nice if this works in a smooth way.
My main project needs to generate a sub-project with its own build.sbt and then compile and use the sub-project. If the sub-project was pre-generated, I could reference it from the main build.sbt with RootProject. But if the sub-project is not yet generated, any attempt to use a value of lazy val sub = RootProject(subBaseDir) fails.
Is it possible to load the sub-project that does not exist at the moment of sbt start, so some tasks of the main project depended of the sub-project?
Yes, it should be possible in SBT 0.13.13 using its new feature called "Synthetic subprojects". You won't have a build.sbt file for such synthetic projects, since they're, well, completely synthetic. But otherwise they're completely functional projects, and you should be able to set up the dependencies between tasks of current project and its derived projects.
Disclaimer: I haven't tried this new feature myself yet.
I have a multi-project build that makes extensive use of sub-projects in order to make dependencies between them explicit (by means of dependsOn - the sub-projects act as layers).
If it comes to publishing I want to roll up all the sub-project artifacts into the artifact of the main project (jar), and collect all their libraryDependencies. In other words, I want to publish as if there would be no sub-projects, just a single root project containing all my code (i.e. no ueber-jar). The sub-projects are only there to break up the compilation into smaller scopes with enforced dependency structure, they are not supposed to be distributed separately.
What is the best way to achieve that?
I just started working with ant a few days ago. Right now I have a general buildall.xml which should call each project's build.xml. Because some projects depend on each other, I need to rebuild some other projects which depend on it. This isn't a problem--I'm just setting the depends property of the target. However, ant is always building the dependencies, even when the files haven't changed.
Let's say project1 has no dependencies; project2 depends on project1; project3 depends on project1, 2; project4 depends on project1, 2, and 3; and so on.
I could hack a solution which looks at project K, and checks if project 1 .. project K have updated files using uptodate. If so, then run the target. This is messy and appears unnecessary.
What is the cleanest way to implement this?
EDIT: So I decided to just hack in a bunch of targets, "check_projectK" where it does the uptodate checks on all of its source files, its build file, and the build files of the 1 .. K-1 projects. Due to dependencies, this is always handled correctly. However, this is still a large amount of copy and paste for a large workspace. I will leave this open.
Short answer, ANT can't do it, not unless you have some kind of way to connect to your version control system and check if anything has changed (you are using source control right?). Ant doesn't know about when what the last time a file changed and then see if it matches with what was built; it doesn't have the concept of a dependency repository. The whole purpose of Ant is that it just builds.
The solution to your problem isn't Ant, it's Maven. Maven HAS a dependency repository. There's also a very nifty plugin for Maven used specifically with Flex appropriately called FlexMojos. By using this, Maven can know when something was last built because it's uploaded to the repository. Then your other projects can add it's dependencies and download the SWC needed.
On top of that, it mixes great with a continuous integration engine like Hudson, Bamboo and Teamcity, which builds a project every time a file has been committed to your source control system, and then updates all dependent projects automatically!