How do you do develop an SBT project, itself? - sbt

Background: I've got a Play 2.0 project, and I am trying to add something to do aspectj weaving using aspects in a jar on some of my classes (Java). (sbt-aspectj doesn't seem to do it, or I can't see how). So I need to add a custom task, and have it depend on compile. I've sort of figured out the dependency part. However, because I don't know exactly what I'm doing, yet, I want to develop this using the IDE (I'm using Scala-IDE). Since sbt projects (and therefore Play projects) are recursively defined, I assumed I could:
Add the eclipse plugin to the myplay/project/project/plugins.sbt
Add the sbt main jar (and aspectj jar) to myplay/project/project/build.sbt:
libraryDependencies ++= Seq(
"org.scala-sbt" % "main" % "0.12.2",
"aspectj" % "aspectj-tools" % "1.0.6"
)
Drop into the myplay/project
Run sbt, run the eclipse task, then import the project into eclipse as a separate project.
I can do this, though the build.scala (and other scala files) aren't initially considered source, and I have to fiddle with the build path a bit. However, even though I've got the sbt main defined for the project, both eclipse IDE and the compile task give errors:
> compile
[error] .../myplay/project/Build.scala:2: not found: object keys
[error] import keys.Keys._
[error] ^
[error] .../myplay/project/SbtAspectJ.scala:2: object Configurations is not a member of package sbt
[error] import sbt.Configurations.Compile
[error] ^
[error] .../myplay/project/SbtAspectJ.scala:3: object Keys is not a member of package sbt
[error] import sbt.Keys._
[error] ^
[error] three errors found
The eclipse project shows neither main nor aspectj-tools in its referenced-libraries. However, if I give it a bogus version (e.g. 0.12.4), reload fails, so it appears to be using
the dependency.
So,...
First: Is this the proper way to do this?
Second: If so, why aren't the libs getting added.
(Third: please don't let this be something dumb that I missed.)

If you are getting the object Keys is not a member of package sbt error, then you should check that you are running sbt from the base directory, and not the /project directory.

sbt-aspectj
sbt-aspectj doesn't seem to do it, or I can't see how.
I think this is the real issue. There's a plugin already that does the work, so try making it work instead of fiddling with the build. Using plugins from build.scala is a bit tricky.
Luckily there are sample projects on github:
import sbt._
import sbt.Keys._
import com.typesafe.sbt.SbtAspectj.{ Aspectj, aspectjSettings, compiledClasses }
import com.typesafe.sbt.SbtAspectj.AspectjKeys.{ binaries, compileOnly, inputs, lintProperties }
object SampleBuild extends Build {
....
// precompiled aspects
lazy val tracer = Project(
"tracer",
file("tracer"),
settings = buildSettings ++ aspectjSettings ++ Seq(
// stop after compiling the aspects (no weaving)
compileOnly in Aspectj := true,
// ignore warnings (we don't have the sample classes)
lintProperties in Aspectj += "invalidAbsoluteTypeName = ignore",
// replace regular products with compiled aspects
products in Compile <<= products in Aspectj
)
)
}
How do you do develop an SBT project, itself?
If you're interested in hacking on the build still the first place to go is the Getting Started guide. Specifically, your question should be answered in .scala Build Definition page.
I think you want your build to utilize "aspectj" % "aspectj-tools" % "1.0.6". If so it should be included in myplay/project/plugins.sbt, and your code should go into myplay/project/common.scala or something. If you want to use IDE, you have have better luck with building it as a sbt plugin. That way your code would go into src/main/scala. Check out sbt/sbt-aspectj or sbt/sbt-assembly on example of sbt plugin structure.

Related

Execute sbt task without prior compiling (for generating database classes with JOOQ)

I have a PlayFramework 2.7 application which is build by sbt.
For accessing the database, I am using JOOQ. As you know, JOOQ reads my database schema and generates the Java source classes, which then are used in my application.
The application only compiles, if the database classes are present.
I am generating the classes with this custom sbt task:
// https://mvnrepository.com/artifact/org.jooq/jooq-meta
libraryDependencies += "org.jooq" % "jooq-meta" % "3.12.1"
lazy val generateJOOQ = taskKey[Seq[File]]("Generate JooQ classes")
generateJOOQ := {
(runner in Compile).value.run("org.jooq.codegen.GenerationTool",
(fullClasspath in Compile).value.files,
Array("conf/db.conf.xml"),
streams.value.log).failed foreach (sys error _.getMessage)
((sourceManaged.value / "main/generated") ** "*.java").get
}
I googled around and found the script above and modified it a little bit according to my needs, but I do not really understand it, since sbt/scala are new to me.
The problem now is, when I start the generateJOOQ, sbt tries to compile the project first, which fails, because the database classes are missing. So what I have to do is, comment all code out which uses the generated classes, execute the task which compiles my project, generates the database classes and then enable the commented out code again. This is a pain!
I guess the problem is the command (runner in Compile) but I did not find any possibility to execute my custom task WITHOUT compiling first.
Please help! Thank you!
Usually, when you want to generate sources, you should use a source generator, see https://www.scala-sbt.org/1.x/docs/Howto-Generating-Files.html
sourceGenerators in Compile += generateJOOQ
Doing that automatically causes SBT to execute your task first before trying to compile the Scala/Java sources.
Then, you should not really use the runner task, since that is used to run your project, which depends on the compile task, which needs to execute first of course.
You should add the jooq-meta library as a dependeny of the build, not of your sources. That means you should add the libraryDependencies += "org.jooq" % "jooq-meta" % "3.12.1" line e.g. to project/jooq.sbt.
Then, you can simply call the GenerationTool of jooq just as usually in your task:
// build.sbt
generateJOOQ := {
org.jooq.codegen.GenerationTool.main(Array("conf/db.conf.xml"))
((sourceManaged.value / "main/generated") ** "*.java").get
}

SBT support for custom ivy module status?

In my SBT build I have a dependency on an ivy artifact that makes use of a custom module status. This causes the following error in SBT:
[error] (*:update) sbt.ResolveException: unresolved dependency: my-org#myapp-core_2.11;1.0: java.text.ParseException: inconsistent module descriptor file found in 'http://artifacts.myorg.com/libs-snapshots-local/myapp-core_2.11/1.0/myapp-ivy.xml': bad status: 'snapshot';
I can work around this by telling SBT to use an external ivy settings, like so:
externalIvySettings(baseDirectory(_ / "ivySettings.xml"))
And then create an ivySettings.xml containing the following:
<statuses default="release">
<status name="release" integration="false"/>
<status name="snapshot" integration="false"/>
</statuses>
But surely there must be a better way? The problem with this work around is that now all my settings (such as resolvers) have to be in the ivy file too, because (IFAIK) it's all or nothing when you use externalIvySettings.
Is there a way to specify a set of custom statuses within my build.sbt? Or alternatively is there a way to tell sbt to combine external ivy settings with the ones it generates from the build.sbt.
Since specifying custom module statuses is a valid thing to do in ivy, this should really be supported in sbt too.
This is because for some repositories, they use non-standard status which fails the consistency check. We addressed this by constructing the customized resolver which doesn't do consistency check. You can also construct resolver with the custom status using the same approach. The following is the working snippet.
resolvers += {
val resolver = new org.apache.ivy.plugins.resolver.IBiblioResolver
resolver.setName("Custom Ivy Snapshots")
resolver.setRoot("http://Custom/snapshots/")
val settings = new org.apache.ivy.core.settings.IvySettings()
settings.setVariable("ivy.local.default.ivy.pattern", Pattern)
settings.setVariable("ivy.local.default.artifact.pattern", Pattern)
resolver.setSettings(settings)
resolver.setM2compatible(true)
resolver.setCheckconsistency(false)
new RawRepository(resolver)
}

sbt cross configuration dependencies

what is the reason SBT won't allow me to have dependencies between different configurations of different projects in a multi-project build?
consider the following setup in the main build.sbt file:
lazy val domain: Project = project in file("domain") dependsOn(testUtils % "test->test")
lazy val testUtils: Project = project in file("testUtils") dependsOn(domain % "compile->test")
...
I would want to write all my test helpers in testUtils, and have each of the other projects' test code to be clean test logic without the (sometimes duplicated among different projects) boilerplate of the aiding methods.
SBT is forcing me to put the : Project type, since it complains the value is "recursive". and upon reloading, I get:
...
at $281429c805669a7befa4$.domain(build.sbt:142)
at $281429c805669a7befa4$.testUtils$lzycompute(build.sbt:144)
at $281429c805669a7befa4$.testUtils(build.sbt:144)
at $281429c805669a7befa4$.domain$lzycompute(build.sbt:142)
at $281429c805669a7befa4$.domain(build.sbt:142)
[error] java.lang.StackOverflowError
[error] Use 'last' for the full log.
is there a way around this? or should I write test-related logic in each module test, even at the cost of getting the code less organize, many "test->test" dependencies, etc'...

How to configure Typesafe Activator not to use user home?

I'd like to configure Typesafe Activator and it's bundled tooling not to use my user home directory - I mean ~/.activator (configuration?), ~/.sbt (sbt configuration?) and especially ~/.ivy2, which I'd like to share between my two OSes.
Typesafe "documentation" is of little help.
Need help for both Windows and Linux, please.
From Command Line Options in the official documentation of sbt:
sbt.global.base - The directory containing global settings and plugins (default: ~/.sbt/0.13)
sbt.ivy.home - The directory containing the local Ivy repository and artifact cache (default: ~/.ivy2)
It appears that ~/.activator is set and used in the startup scripts and that's where I'd change the value.
It also appears (in sbt/sbt.boot.properties in activator-launch-1.2.1.jar) that the value of ivy-home is ${user.home}/.ivy2:
[ivy]
ivy-home: ${user.home}/.ivy2
checksums: ${sbt.checksums-sha1,md5}
override-build-repos: ${sbt.override.build.repos-false}
repository-config: ${sbt.repository.config-${sbt.global.base-${user.home}/.sbt}/repositories}
It means that without some development it's only possible to change sbt.global.base.
➜ minimal-scala activator -Dsbt.global.base=./sbt -Dsbt.ivy.home=./ivy2 about
[info] Loading project definition from /Users/jacek/sandbox/sbt-launcher/minimal-scala/project
[info] Set current project to minimal-scala (in build file:/Users/jacek/sandbox/sbt-launcher/minimal-scala/)
[info] This is sbt 0.13.5
[info] The current project is {file:/Users/jacek/sandbox/sbt-launcher/minimal-scala/}minimal-scala 1.0
[info] The current project is built against Scala 2.11.1
[info] Available Plugins: sbt.plugins.IvyPlugin, sbt.plugins.JvmPlugin, sbt.plugins.CorePlugin, sbt.plugins.JUnitXmlReportPlugin
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.4
If you want to see under the hood, you could query for the current values of the home directories for sbt and Ivy with consoleProject command (it assumes you started activator with activator -Dsbt.global.base=./sbt -Dsbt.ivy.home=./ivy2):
> consoleProject
[info] Starting scala interpreter...
[info]
import sbt._
import Keys._
import _root_.sbt.plugins.IvyPlugin
import _root_.sbt.plugins.JvmPlugin
import _root_.sbt.plugins.CorePlugin
import _root_.sbt.plugins.JUnitXmlReportPlugin
import currentState._
import extracted._
import cpHelpers._
Welcome to Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_60).
Type in expressions to have them evaluated.
Type :help for more information.
scala> appConfiguration.eval.provider.scalaProvider.launcher.bootDirectory
res0: java.io.File = /Users/jacek/sandbox/sbt-launcher/minimal-scala/sbt/boot
scala> appConfiguration.eval.provider.scalaProvider.launcher.ivyHome
res1: java.io.File = /Users/jacek/.ivy2
Iff you're really into convincing Activator to use sbt.ivy.home, you have to change sbt/sbt.boot.properties in activator-launch-1.2.2.jar. Just follow the steps:
Unpack sbt/sbt.boot.properties out of activator-launch-1.2.2.jar.
jar -xvf activator-launch-1.2.2.jar sbt/sbt.boot.properties
Edit sbt/sbt.boot.properties and replace ivy-home under [ivy].
ivy-home: ${sbt.ivy.home-${user.home}/.ivy2}
Add the changed sbt/sbt.boot.properties to activator-launch-1.2.2.jar.
jar -uvf activator-launch-1.2.2.jar sbt/sbt.boot.properties
With the change, -Dsbt.ivy.home=./ivy2 works fine.
scala> appConfiguration.eval.provider.scalaProvider.launcher.bootDirectory
res0: java.io.File = /Users/jacek/sandbox/sbt-launcher/minimal-scala/sbt/boot
scala> appConfiguration.eval.provider.scalaProvider.launcher.ivyHome
res1: java.io.File = /Users/jacek/sandbox/sbt-launcher/minimal-scala/ivy2
I was experimenting with this today. After a while, it seems to me like this could be the best thing to do:
Windows:
setx _JAVA_OPTIONS "-Duser.home=C:/my/preferred/home/"
Linux:
export _JAVA_OPTIONS='-Duser.home=/local/home/me'
Then you should be good to go for any Java Program that wants to store data in your home directory.
As an addition to Jacek's answer, another way that worked for me to set the .ivy2 directory was to use the sbt ivyConfiguration task. It returns configuration settings related to ivy, including the path to the ivy home (the one which defaults to ~/.ivy2).
Simply add these few lines to the build.sbt file in your project :
ivyConfiguration ~= { originalIvyConfiguration =>
val config = originalIvyConfiguration.asInstanceOf[InlineIvyConfiguration]
val ivyHome = file("./.ivy2")
val ivyPaths = new IvyPaths(config.paths.baseDirectory, Some(ivyHome))
new InlineIvyConfiguration(ivyPaths, config.resolvers, config.otherResolvers,
config.moduleConfigurations, config.localOnly, config.lock,
config.checksums, config.resolutionCacheDir, config.log)
}
It returns a new ivy configuration identical to the original one, but with the right path to the ivy home directory (here ./.ivy2, so it'll be located just next to the build.sbt file). This way, when sbt uses the ivyConfiguration task to get the ivy configuration, the path to the .ivy2 directory will be the one set above.
It worked for me using sbt 0.13.5 and 0.13.8.
Note: for sbt versions 0.13.6 and above, the construction of the InlineIvyConfiguration needs an additional parameter to avoid being flagged as deprecated, so you might want to change the last line into :
new InlineIvyConfiguration(ivyPaths, config.resolvers, config.otherResolvers,
config.moduleConfigurations, config.localOnly, config.lock,
config.checksums, config.resolutionCacheDir, config.updateOptions, config.log)
(note the additional config.updateOptions)
I had that same problem on Mac. I erased the directory in user home directory named .activator and all related folder. After that run activator run command on terminal. Activator download all of the folder which I deleted. Than problem solved.

Defining plugin dependency between subprojects in SBT?

EDIT:
Since I put up the bounty, I thought I should restate the question
How can a SBT project P, with two sub-projects A and B, set up B to have a plugin dependency on A, which is a SBT plugin?
Giving P a plugin dependency on A does not work, since A depends on other things in P, which results in a circular dependency graph
It has to be a plugin dependency, for A is a plugin needed to run Bs test suite.
dependsOn doesn't work, because, well, it has to be a plugin dependency
I'd like to know either of
How to do this, or
Why this is impossible, and what the next best alternatives are.
EDIT: clarified that it's a plugin-dependency, since build-dependency is ambiguous
When you have a multi-project build configuration with "project P and two sub-projects A and B" it boils down to the following configuration:
build.sbt
lazy val A, B = project
As per design, "If a project is not defined for the root directory in the build, sbt creates a default one that aggregates all other projects in the build." It means that you will have an implicit root project, say P (but the name is arbitrary):
[plugin-project-and-another]> projects
[info] In file:/Users/jacek/sandbox/so/plugin-project-and-another/
[info] A
[info] B
[info] * plugin-project-and-another
That gives us the expected project structure. On to defining plugin dependency between B and A.
The only way to define a plugin in a SBT project is to use project directory that's the plugins project's build definition - "A plugin definition is a project in <main-project>/project/." It means that the only way to define a plugin dependency on the project A is to use the following:
project/plugins.sbt
addSbtPlugin("org.example" % "example-plugin" % "1.0")
lazy val plugins = project in file(".") dependsOn(file("../A"))
In this build configuration, the plugins project depends on another SBT project that happens to be our A that's in turn a plugin project.
A/build.sbt
// http://www.scala-sbt.org/release/docs/Extending/Plugins.html#example-plugin
sbtPlugin := true
name := "example-plugin"
organization := "org.example"
version := "1.0"
A/MyPlugin.scala
import sbt._
object MyPlugin extends Plugin
{
// configuration points, like the built in `version`, `libraryDependencies`, or `compile`
// by implementing Plugin, these are automatically imported in a user's `build.sbt`
val newTask = taskKey[Unit]("A new task.")
val newSetting = settingKey[String]("A new setting.")
// a group of settings ready to be added to a Project
// to automatically add them, do
val newSettings = Seq(
newSetting := "Hello from plugin",
newTask := println(newSetting.value)
)
// alternatively, by overriding `settings`, they could be automatically added to a Project
// override val settings = Seq(...)
}
The two files - build.sbt and MyPlugin.scala in the directory A - make up the plugin project.
The only missing piece is to define the plugin A's settings for the project B.
B/build.sbt
MyPlugin.newSettings
That's pretty much it what you can do in SBT. If you want to have multi-project build configuration and have a plugin dependency between (sub)projects, you don't have much choice other than what described above.
With that said, let's see if the plugin from the project A is accessible.
[plugin-project-and-another]> newTask
Hello from plugin
[success] Total time: 0 s, completed Feb 13, 2014 2:29:31 AM
[plugin-project-and-another]> B/newTask
Hello from plugin
[success] Total time: 0 s, completed Feb 13, 2014 2:29:36 AM
[plugin-project-and-another]> A/newTask
[error] No such setting/task
[error] A/newTask
[error] ^
As you may have noticed, newTask (that comes from the plugin from the project A) is available in the (default) root project and the project B, but not in A.
As Jacek said, it cannot be done as I would like, as a subproject cannot have a SBT plugin that the root project does not. On the other hand, this discussion on the mailing list contains several alternatives, and would no doubt be useful to anyone who comes across this question in the future.
EDIT: Well, in the end the alternatives mentioned (sbt scripted, etc) were hard and clunky to use. My final solution was to just have a separate project (not subproject) inside the repo that depends on the original project via it's ivy coordinates, and using bash to publishLocal the first project, going into the second project and running its tests
sbt publishLocal; cd test; sbt test; cd ..
I always thought the point of something like SBT was to avoid doing this kind of bash gymnastics, but desperate times call for desperate measures...
This answer may include the solution https://stackoverflow.com/a/12754868/3189923 .
From that link, in short, set exportJars := true and to obtain jar file paths for a (sub)project exportedProducts in Compile.
Leaving the facts about plugins by side, you have a parent project P with sub-projects A and B. And then you state that A depends on P. But P is a aggregate of A and B and hence depends on A. So you already have a circular dependency between A and P. This can never work.
You have to split P in two parts: The part where A depends on (let's call this part A') and the rest (let's call this P_rest). Then you throw away P and make a new project P_rest consisting of A', A and B. And A depends on A'.

Resources