I'm unable to understand the difference between the following expressions. The expressions a reduced -- the real world case has more settings distributed in seperate objects.
specified directly as sequence
Seq(libraryDependencies +=
"org.openjdk.jmh" % "jmh-core" % "1.6.2" % "compile")
wrapped in inConfig
inConfig(Compile)(libraryDependencies +=
"org.openjdk.jmh" % "jmh-core" % "1.6.2" % "compile")
In both cases show compile:libraryDependencies shows the same
[info] List(org.scala-lang:scala-library:2.10.4, org.openjdk.jmh:jmh-core:1.6.2:compile)
but for show compile:managedClasspath the dependency towards JMH is only shown in the first case. As a result the normal compiler run fails due to unresolvable classes.
Please explain or point the logical difference between both cases.
TLDR: Use % "compile" not Compile when declaring libraryDependencies
What you're seeing is a gap, possibly a bug, in how sbt tries to use Ivy's module configurations (such as compile) as one of its setting scopes.
For reference:
Official docs about Scopes
Blog post about sbt "dimensions"
The problem lies that, currently, you can declare the configuration at both the value level:
"org.openjdk.jmh" % "jmh-core" % "1.6.2" % "compile"
and the key level:
inConfig(Compile)(libraryDependencies += xyz)
or alternatively:
libraryDependencies in Compile += xyz
As you say in both examples show compile:libraryDependencies shows the same sequence, but show libraryDependencies demonstrates that you only added jmh-core in the Compile axes of libraryDependencies:
show libraryDependencies
[info] List(org.scala-lang:scala-library:2.10.4, org.openjdk.jmh:jmh-core:1.6.2:compile)
show libraryDependencies
[info] List(org.scala-lang:scala-library:2.10.4)
This then leads as to why show compile:managedClasspath is different.
Have a look at what inspect actual compile:managedClasspath outputs:
[info] Task: scala.collection.Seq[sbt.Attributed[java.io.File]]
[info] Description:
[info] The classpath consisting of external, managed library dependencies.
[info] Provided by:
[info] {file:/Users/dnw/Desktop/t-2015-04-08.0540/}t-2015-04-08-0540/compile:managedClasspath
[info] Defined at:
[info] (sbt.Classpaths) Defaults.scala:991
[info] Dependencies:
[info] *:update
[info] */*:classpathTypes
[info] compile:classpathConfiguration
[info] compile:managedClasspath::streams
[info] Reverse dependencies:
[info] compile:externalDependencyClasspath
[info] Delegates:
[info] compile:managedClasspath
[info] *:managedClasspath
[info] {.}/compile:managedClasspath
[info] {.}/*:managedClasspath
[info] */compile:managedClasspath
[info] */*:managedClasspath
[info] Related:
[info] test:managedClasspath
[info] runtime:managedClasspath
The thing to note is its dependency on *:update, which isn't scoped to compile. From there it eventually leads to *:libraryDependencies which in your second example doesn't include jmh-core.
Related
I found this in the root build.sbt
skip in test := true
What does it do? I cannot find any reference regarding this in sbt documents. I'm using sbt 1.3.8 in my Scala project.
If set to true, the test task won't run and no test will be executed.
In your context, no test of the root project will be executed.
One way to know what the task is doing and how it used by other tasks is to run inspect task.
sbt "inspect test:skip"
Produces:
[info] Task: Boolean
[info] Description:
[info] For tasks that support it (currently only 'compile', 'update', and 'publish'), setting skip to true will force the task to not to do its work. This exact semantics may vary by task.
[info] Provided by:
[info] Global / skip
[info] Defined at:
[info] (sbt.Defaults.globalSbtCore) Defaults.scala:294
[info] Delegates:
[info] Test / skip
[info] Runtime / skip
[info] Compile / skip
[info] skip
[info] ThisBuild / Test / skip
[info] ThisBuild / Runtime / skip
[info] ThisBuild / Compile / skip
[info] ThisBuild / skip
[info] Zero / Test / skip
[info] Zero / Runtime / skip
[info] Zero / Compile / skip
[info] Global / skip
[info] Related:
[info] Global / skip
With description
For tasks that support it (currently only 'compile', 'update', and 'publish'), setting skip to true will force the task to not to do its work. This exact semantics may vary by task.
I've created an Autoplugin for an SBT project to launch middleware inside Docker containers for integration tests (Zookeeper and Kafka).
My first version without Autoplugin was to add manually in the projects settings such as :
(test in Test) <<= (test in Test) dependsOn zkStart
That was working very well.
Now with an Autoplugin, I've the following code
override def projectSettings: Seq[Def.Setting[_]] = Seq(
(test in Test) <<= (test in Test) dependsOn ZookeeperPlugin.zkStart
)
but Zookeeper is no longer start before tests.
when I do
[core_akka_cluster] $ inspect test
[info] Task: Unit
[info] Description:
[info] Executes all tests.
[info] Provided by:
[info] {file:/Users/xx/Projects/../../}core_akka_cluster/test:test
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:394
We can see that the setting test:test is provided by the default SBT values.
When I manually add the previous settings in the build definition of my project, this works once more and we have the following analysis
[core_akka_cluster] $ inspect test
[info] Task: Unit
[info] Description:
[info] Executes all tests.
[info] Provided by:
[info] [info] {file:/Users/xx/Projects/../../}core_akka_cluster/test:test
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:394
[info] (com.ingenico.msh.sbt.KafkaPluginSettings) KafkaPlugin.scala:36
Any idea about precedence in this case?
Thanks
Are you making the auto plugin a triggered plugin?
Since test is also added by an auto plugin (JvmPlugin) by sbt, you should require JvmPlugin.
I am trying to understand what SBT scope axes are and how they influence scoped keys in the map of the build.
There are three scope axes : project, configuration and task.
Is the following tuple the scoped key for the map of the build?
(value of project axis, value of configuration axis, value of task axis, value of unscoped key)
Or is a scoped key (conceptually) a pair having one of the following three forms ?
(value of project axis, value of unscoped key)
(value of configuration axis, value of unscoped key)
(value of task axis, value of unscoped key)
Or is the scoped key something else, if yes how does it look like (conceptually) ?
So the question could be what is (conceptually) a scoped key?
Is it a four tuple?
Is it a pair?
Is it something else, if yes, then how does it looks like (conceptually)?
From Custom settings and tasks:
Keys have one of three types: SettingKey, TaskKey and InputKey.
An example of a key can be:
val scalaVersion = settingKey[String]("The version of Scala used for building.")
It's available by default in sbt and to query for a value you execute show:
> show scalaVersion
[info] 2.10.4
Execute inspect to learn about the details of a key (it's not exactly about a key but a setting as you soon learn):
> inspect scalaVersion
[info] Setting: java.lang.String = 2.10.4
[info] Description:
[info] The version of Scala used for building.
[info] Provided by:
[info] */*:scalaVersion
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:236
[info] Reverse dependencies:
[info] *:libraryDependencies
[info] *:scalaInstance
[info] *:evicted
[info] *:dependencyUpdatesData
[info] *:update
[info] *:allDependencies
[info] Delegates:
[info] *:scalaVersion
[info] {.}/*:scalaVersion
[info] */*:scalaVersion
[info] Related:
[info] */*:scalaVersion
You can define your own key with the macro settingKey:
lazy val abc: sbt.SettingKey[String] = settingKey[String]("My own setting key")
According to the scaladoc of sbt.SettingKey:
Identifies a setting. It consists of three parts: the scope, the name, and the type of a value associated with this key. The scope is represented by a value of type Scope. The name and the type are represented by a value of type AttributeKey[T]. Instances are constructed using the companion object.
As you can see in the scaladoc, any SettingKey has the required def scope: Scope that describes the scope of the setting. Soon to be explained.
You can't see the key in the build until you assign it a value (via a computation). You assign an Initialize[T] or just T to a SettingKey[T] via :=, += and ++= (more often used and recommended) or <++=, <+= and <<= operators (that are Scala methods described in the scaladoc of sbt.SettingKey).
abc := "my value"
The above line is equivalent to the following line (they're interchangeable):
abc.:=("my value")
As you can see in the definition of :=:
final def :=(v: T): Def.Setting[T]
the result of it is Setting[T].
So, the pair (SettingKey[T], Initialize[T]) or simply (SettingKey[T], T) is a Setting[T].
A setting of type Setting[T] where T is the type of the value that is a result of the setting's evaluation describes a transformation to the map of all the settings in a build. It's a recipe for a value that's going to be available under a key after the build's loaded.
Now, the last part, the scope.
As you may've noticed, a setting is always in a scope. You can change the scope of a setting using final def in(scope: Scope): SettingKey[T].
Use inspect to learn about a setting.
> inspect abc
[info] Setting: java.lang.String = my value
[info] Description:
[info] My new setting key
[info] Provided by:
[info] {file:/Users/jacek/sandbox/sbt-theory/}sbt-theory/*:abc
[info] Defined at:
[info] /Users/jacek/sandbox/sbt-theory/build.sbt:3
[info] Delegates:
[info] *:abc
[info] {.}/*:abc
[info] */*:abc
You can see that the setting delegates (section Delegates) to *:abc, {.}/*:abc and */*:abc. The other places that I'm going to describe as A/B:K are A - the project axis, B - the configuration axis and K the key or the task scope (since sbt allows for attibutes that appear after ::).
Guess what the value of *:abc is. inspect it.
> inspect *:abc
[info] Setting: java.lang.String = my value
[info] Description:
[info] My new setting key
[info] Provided by:
[info] {file:/Users/jacek/sandbox/sbt-theory/}sbt-theory/*:abc
[info] Defined at:
[info] /Users/jacek/sandbox/sbt-theory/build.sbt:3
[info] Delegates:
[info] *:abc
[info] {.}/*:abc
[info] */*:abc
Guess what the value of {.}/*:abc that corresponds to this build denoted as {.} in the global configuration *. inspect it.
> inspect {.}/*:abc
[info] No entry for key.
[info] Description:
[info] My new setting key
[info] Delegates:
[info] {.}/*:abc
[info] */*:abc
[info] Related:
[info] *:abc
There's no value for abc setting in ThisBuild (aka thisBuild) scope. Set it in build.sbt using:
abc in ThisBuild := "abc in ThisBuild"
inspect it (after reload to reload the build's changes):
> inspect {.}/*:abc
[info] Setting: java.lang.String = abc in ThisBuild
[info] Description:
[info] My new setting key
[info] Provided by:
[info] {file:/Users/jacek/sandbox/sbt-theory/}/*:abc
[info] Defined at:
[info] /Users/jacek/sandbox/sbt-theory/build.sbt:5
[info] Delegates:
[info] {.}/*:abc
[info] */*:abc
[info] Related:
[info] *:abc
With the change build.sbt looks as follows:
lazy val abc: sbt.SettingKey[String] = settingKey[String]("My new setting key")
abc := "my value"
abc in ThisBuild := "abc in ThisBuild"
that says that there's abc key that has different values per scope - ThisBuild and Global. Clear? Ask away!
How to set up sbt-assembly to include jars with runtime scope? What's the rationale for not doing it by default?
This is now being discussed under sbt/sbt-assembly#120.
sbt-assembly is intended to wire full classpath from Runtime configuration. And it's coded as follows:
fullClasspath in assembly <<= fullClasspath or (fullClasspath in Runtime),
For Spark, for whatever reason fullClasspath is wired, so that's likely interfering. The reason sbt-assembly included fullClasspath I think is now historical, so I might get rid of "fullClasspath or" part. As noted in the issue, the current workaround is likely:
fullClasspath in assembly := (fullClasspath in Runtime).value
FYI for those who learn sbt the above sbt-0.13+ format with := corresponds to the older pre-sbt-0.13 format with := as follows:
fullClasspath in assembly <<= (fullClasspath in Runtime)
My tests show that the plugin does include dependencies with runtime scope as evidenced with the following example.
project/plugins.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
build.sbt
libraryDependencies += "ch.qos.logback" % "logback-classic" % "1.1.2" % "runtime"
assemblySettings
This is a sample session:
> about
[info] This is sbt 0.13.6-SNAPSHOT
[info] The current project is {file:/C:/dev/sandbox/runtime-assembly/}runtime-assembly 0.1-SNAPSHOT
[info] The current project is built against Scala 2.10.4
[info] Available Plugins: sbt.plugins.IvyPlugin, sbt.plugins.JvmPlugin, sbt.plugins.CorePlugin, sbt.plugins.JUnitXmlReportPlugin, org.sbtidea.SbtIdeaPlugin, de.johoop.jacoco4sbt.JacocoPlugin, com.timushev.sbt.updates.UpdatesPlugin, sbtassembly.Plugin
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.4
> assembly
[info] Updating {file:/C:/dev/sandbox/runtime-assembly/}runtime-assembly...
[info] Resolving org.jacoco#org.jacoco.agent;0.6.4.201312101107 ...
[info] downloading http://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.1.2/logback-classic-1.1.2.jar ...
[info] [SUCCESSFUL ] ch.qos.logback#logback-classic;1.1.2!logback-classic.jar (1122ms)
[info] downloading http://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.1.2/logback-core-1.1.2.jar ...
[info] [SUCCESSFUL ] ch.qos.logback#logback-core;1.1.2!logback-core.jar (1313ms)
[info] Done updating.
[info] Including: slf4j-api-1.7.6.jar
[info] Including: logback-classic-1.1.2.jar
[info] Including: logback-core-1.1.2.jar
[info] Including: scala-library.jar
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'META-INF\MANIFEST.MF' with strategy 'discard'
[warn] Strategy 'discard' was applied to a file
[info] SHA-1: 0f520cdfc30de8ced8408a3c25b326b3ff5afda8
[info] Packaging C:\dev\sandbox\runtime-assembly\target\scala-2.10\runtime-assembly-assembly-0.1-SNAPSHOT.jar ...
[info] Done packaging.
Jar verification:
/cygdrive/c/dev/sandbox/runtime-assembly
$ jar -tf target/scala-2.10/runtime-assembly-assembly-0.1-SNAPSHOT.jar | grep logback | wc -l
592
I have a project where I need to disable assertions when creating the binaries. Now I could just do:
scalacOptions += "-Xdisable-assertions"
But then also the unit tests would be run without assertions. Is there a (hopefully) simple way to achieve what I need?
How do you create the binaries? What task/command do you use?
Use the task as the scope for scalacOptions to have different values for them. See Scoping by configuration axis:
By default, all the keys associated with compiling, packaging, and
running are scoped to a configuration and therefore may work
differently in each configuration. The most obvious examples are the
task keys compile, package, and run; but all the keys which affect
those keys (such as sourceDirectories or scalacOptions or
fullClasspath) are also scoped to the configuration.
Use inspect when in doubt.
> inspect scalacOptions
[info] Task: scala.collection.Seq[java.lang.String]
[info] Description:
[info] Options for the Scala compiler.
[info] Provided by:
[info] {file:/C:/dev/sandbox/task-dependsOn/}task-dependson/compile:scalacOptions
[info] Defined at:
[info] (sbt.Classpaths) Defaults.scala:1424
[info] Dependencies:
[info] task-dependson/compile:autoCompilerPlugins
[info] task-dependson/compile:settingsData
[info] task-dependson/compile:update
[info] task-dependson/compile:buildDependencies
[info] task-dependson/compile:thisProjectRef
[info] Delegates:
[info] task-dependson/compile:scalacOptions
[info] task-dependson/*:scalacOptions
[info] {.}/compile:scalacOptions
[info] {.}/*:scalacOptions
[info] */compile:scalacOptions
[info] */*:scalacOptions
[info] Related:
[info] b/compile:scalacOptions
[info] b/test:scalacOptions
[info] task-dependson/test:scalacOptions
[info] task-dependson/jacoco:scalacOptions
[info] a/jacoco:scalacOptions
[info] */*:scalacOptions
[info] a/test:scalacOptions
[info] a/compile:scalacOptions
[info] b/jacoco:scalacOptions
The Compile configuration scope is the default one (see show defaultConfiguration for a project) so scalacOptions += "-Xdisable-assertions" is in fact scalacOptions in Compile += "-Xdisable-assertions". Use different configuration, say Test, and you'll get different results.
There's however a hitch in SBT (I missed the very first time I responded) - settings are chained and when a setting is not defined in a scope, it gets its value from a more general scope. When I said, scalacOptions +=... is in fact scalacOptions in Compile I missed the important feature of settings - scalacOptions is global while scalacOptions in Compile is Compile-scoped.