How to set up sbt-assembly to include jars with runtime scope? What's the rationale for not doing it by default?
This is now being discussed under sbt/sbt-assembly#120.
sbt-assembly is intended to wire full classpath from Runtime configuration. And it's coded as follows:
fullClasspath in assembly <<= fullClasspath or (fullClasspath in Runtime),
For Spark, for whatever reason fullClasspath is wired, so that's likely interfering. The reason sbt-assembly included fullClasspath I think is now historical, so I might get rid of "fullClasspath or" part. As noted in the issue, the current workaround is likely:
fullClasspath in assembly := (fullClasspath in Runtime).value
FYI for those who learn sbt the above sbt-0.13+ format with := corresponds to the older pre-sbt-0.13 format with := as follows:
fullClasspath in assembly <<= (fullClasspath in Runtime)
My tests show that the plugin does include dependencies with runtime scope as evidenced with the following example.
project/plugins.sbt
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")
build.sbt
libraryDependencies += "ch.qos.logback" % "logback-classic" % "1.1.2" % "runtime"
assemblySettings
This is a sample session:
> about
[info] This is sbt 0.13.6-SNAPSHOT
[info] The current project is {file:/C:/dev/sandbox/runtime-assembly/}runtime-assembly 0.1-SNAPSHOT
[info] The current project is built against Scala 2.10.4
[info] Available Plugins: sbt.plugins.IvyPlugin, sbt.plugins.JvmPlugin, sbt.plugins.CorePlugin, sbt.plugins.JUnitXmlReportPlugin, org.sbtidea.SbtIdeaPlugin, de.johoop.jacoco4sbt.JacocoPlugin, com.timushev.sbt.updates.UpdatesPlugin, sbtassembly.Plugin
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.4
> assembly
[info] Updating {file:/C:/dev/sandbox/runtime-assembly/}runtime-assembly...
[info] Resolving org.jacoco#org.jacoco.agent;0.6.4.201312101107 ...
[info] downloading http://repo1.maven.org/maven2/ch/qos/logback/logback-classic/1.1.2/logback-classic-1.1.2.jar ...
[info] [SUCCESSFUL ] ch.qos.logback#logback-classic;1.1.2!logback-classic.jar (1122ms)
[info] downloading http://repo1.maven.org/maven2/ch/qos/logback/logback-core/1.1.2/logback-core-1.1.2.jar ...
[info] [SUCCESSFUL ] ch.qos.logback#logback-core;1.1.2!logback-core.jar (1313ms)
[info] Done updating.
[info] Including: slf4j-api-1.7.6.jar
[info] Including: logback-classic-1.1.2.jar
[info] Including: logback-core-1.1.2.jar
[info] Including: scala-library.jar
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'META-INF\MANIFEST.MF' with strategy 'discard'
[warn] Strategy 'discard' was applied to a file
[info] SHA-1: 0f520cdfc30de8ced8408a3c25b326b3ff5afda8
[info] Packaging C:\dev\sandbox\runtime-assembly\target\scala-2.10\runtime-assembly-assembly-0.1-SNAPSHOT.jar ...
[info] Done packaging.
Jar verification:
/cygdrive/c/dev/sandbox/runtime-assembly
$ jar -tf target/scala-2.10/runtime-assembly-assembly-0.1-SNAPSHOT.jar | grep logback | wc -l
592
Related
I'm unable to understand the difference between the following expressions. The expressions a reduced -- the real world case has more settings distributed in seperate objects.
specified directly as sequence
Seq(libraryDependencies +=
"org.openjdk.jmh" % "jmh-core" % "1.6.2" % "compile")
wrapped in inConfig
inConfig(Compile)(libraryDependencies +=
"org.openjdk.jmh" % "jmh-core" % "1.6.2" % "compile")
In both cases show compile:libraryDependencies shows the same
[info] List(org.scala-lang:scala-library:2.10.4, org.openjdk.jmh:jmh-core:1.6.2:compile)
but for show compile:managedClasspath the dependency towards JMH is only shown in the first case. As a result the normal compiler run fails due to unresolvable classes.
Please explain or point the logical difference between both cases.
TLDR: Use % "compile" not Compile when declaring libraryDependencies
What you're seeing is a gap, possibly a bug, in how sbt tries to use Ivy's module configurations (such as compile) as one of its setting scopes.
For reference:
Official docs about Scopes
Blog post about sbt "dimensions"
The problem lies that, currently, you can declare the configuration at both the value level:
"org.openjdk.jmh" % "jmh-core" % "1.6.2" % "compile"
and the key level:
inConfig(Compile)(libraryDependencies += xyz)
or alternatively:
libraryDependencies in Compile += xyz
As you say in both examples show compile:libraryDependencies shows the same sequence, but show libraryDependencies demonstrates that you only added jmh-core in the Compile axes of libraryDependencies:
show libraryDependencies
[info] List(org.scala-lang:scala-library:2.10.4, org.openjdk.jmh:jmh-core:1.6.2:compile)
show libraryDependencies
[info] List(org.scala-lang:scala-library:2.10.4)
This then leads as to why show compile:managedClasspath is different.
Have a look at what inspect actual compile:managedClasspath outputs:
[info] Task: scala.collection.Seq[sbt.Attributed[java.io.File]]
[info] Description:
[info] The classpath consisting of external, managed library dependencies.
[info] Provided by:
[info] {file:/Users/dnw/Desktop/t-2015-04-08.0540/}t-2015-04-08-0540/compile:managedClasspath
[info] Defined at:
[info] (sbt.Classpaths) Defaults.scala:991
[info] Dependencies:
[info] *:update
[info] */*:classpathTypes
[info] compile:classpathConfiguration
[info] compile:managedClasspath::streams
[info] Reverse dependencies:
[info] compile:externalDependencyClasspath
[info] Delegates:
[info] compile:managedClasspath
[info] *:managedClasspath
[info] {.}/compile:managedClasspath
[info] {.}/*:managedClasspath
[info] */compile:managedClasspath
[info] */*:managedClasspath
[info] Related:
[info] test:managedClasspath
[info] runtime:managedClasspath
The thing to note is its dependency on *:update, which isn't scoped to compile. From there it eventually leads to *:libraryDependencies which in your second example doesn't include jmh-core.
Since Play Framework 2.3 assets are packaged into one jar archive file. I would like to publish this jar automatically with the project, i.e. upon publish or publishLocal I want the assets jar to be published as well.
How to achieve that?
After inspect tree dist I managed to find the task playPackageAssets that generates the assets file:
[play-publish-webjar] $ inspect playPackageAssets
[info] Task: java.io.File
[info] Description:
[info]
[info] Provided by:
[info] {file:/Users/jacek/sandbox/play-publish-webjar/}root/*:playPackageAssets
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:641
[info] Dependencies:
[info] *:playPackageAssets::packageConfiguration
[info] *:playPackageAssets::streams
[info] Reverse dependencies:
[info] *:scriptClasspath
[info] universal:mappings
[info] Delegates:
[info] *:playPackageAssets
[info] {.}/*:playPackageAssets
[info] */*:playPackageAssets
A naive solution might be to attach the assets webjar as is generated by playPackageAssets to publishLocal task's artifacts. Add the following to build.sbt (the types are to show what you work with):
import play.PlayImport.PlayKeys._
packagedArtifacts in publishLocal := {
val artifacts: Map[sbt.Artifact, java.io.File] = (packagedArtifacts in publishLocal).value
val assets: java.io.File = (playPackageAssets in Compile).value
artifacts + (Artifact(moduleName.value, "asset", "jar", "assets") -> assets)
}
Repeat it for the other tasks you want to exhibit similar behaviour.
I'm however quite doubtful it's the best solution.
I have a project where I need to disable assertions when creating the binaries. Now I could just do:
scalacOptions += "-Xdisable-assertions"
But then also the unit tests would be run without assertions. Is there a (hopefully) simple way to achieve what I need?
How do you create the binaries? What task/command do you use?
Use the task as the scope for scalacOptions to have different values for them. See Scoping by configuration axis:
By default, all the keys associated with compiling, packaging, and
running are scoped to a configuration and therefore may work
differently in each configuration. The most obvious examples are the
task keys compile, package, and run; but all the keys which affect
those keys (such as sourceDirectories or scalacOptions or
fullClasspath) are also scoped to the configuration.
Use inspect when in doubt.
> inspect scalacOptions
[info] Task: scala.collection.Seq[java.lang.String]
[info] Description:
[info] Options for the Scala compiler.
[info] Provided by:
[info] {file:/C:/dev/sandbox/task-dependsOn/}task-dependson/compile:scalacOptions
[info] Defined at:
[info] (sbt.Classpaths) Defaults.scala:1424
[info] Dependencies:
[info] task-dependson/compile:autoCompilerPlugins
[info] task-dependson/compile:settingsData
[info] task-dependson/compile:update
[info] task-dependson/compile:buildDependencies
[info] task-dependson/compile:thisProjectRef
[info] Delegates:
[info] task-dependson/compile:scalacOptions
[info] task-dependson/*:scalacOptions
[info] {.}/compile:scalacOptions
[info] {.}/*:scalacOptions
[info] */compile:scalacOptions
[info] */*:scalacOptions
[info] Related:
[info] b/compile:scalacOptions
[info] b/test:scalacOptions
[info] task-dependson/test:scalacOptions
[info] task-dependson/jacoco:scalacOptions
[info] a/jacoco:scalacOptions
[info] */*:scalacOptions
[info] a/test:scalacOptions
[info] a/compile:scalacOptions
[info] b/jacoco:scalacOptions
The Compile configuration scope is the default one (see show defaultConfiguration for a project) so scalacOptions += "-Xdisable-assertions" is in fact scalacOptions in Compile += "-Xdisable-assertions". Use different configuration, say Test, and you'll get different results.
There's however a hitch in SBT (I missed the very first time I responded) - settings are chained and when a setting is not defined in a scope, it gets its value from a more general scope. When I said, scalacOptions +=... is in fact scalacOptions in Compile I missed the important feature of settings - scalacOptions is global while scalacOptions in Compile is Compile-scoped.
I'm using sbt 0.13.0 and Scala 2.10.3
I wanted to publish an artifact to my local maven repo so I added the following to build.sbt:
publishMavenStyle := true
publishTo := Some(Resolver.file("file", new File(Path.userHome.absolutePath+"/.m2/repository")))
artifactName := {
(sv: ScalaVersion, module: ModuleID, artifact: Artifact) =>
artifact.name + "-" + module.revision + "." + artifact.extension
}
When I use publish cmd jars are created in my home .m2 directory but there are of the same content meaning they only contain scaladocs - only html, css and js files.
This is my second time I'm publishing my artifact in this project. Last time it worked perfectly. Most of the classes changed their packages, can this be the reason, and how to fix this?
Remove artifactName and it should work fine again.
Why do you redefine it (as it now breaks publish)? What's the initial goal?
[sbt-0-13-1]> about
[info] This is sbt 0.13.1
[info] The current project is {file:/Users/jacek/sandbox/so/sbt-0.13.1/}sbt-0-13-1 0.1-SNAPSHOT
[info] The current project is built against Scala 2.10.3
[info] Available Plugins: com.typesafe.sbt.SbtGit, com.typesafe.sbt.SbtProguard, growl.GrowlingTests, np.Plugin, com.timushev.sbt.updates.UpdatesPlugin
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.3
[sbt-0-13-1]> help artifactName
Function that produces the artifact name from its definition.
[sbt-0-13-1]> inspect artifactName
[info] Setting: scala.Function3[sbt.ScalaVersion, sbt.ModuleID, sbt.Artifact, java.lang.String] = <function3>
[info] Description:
[info] Function that produces the artifact name from its definition.
[info] Provided by:
[info] */*:artifactName
[info] Defined at:
[info] (sbt.Defaults) Defaults.scala:533
[info] Reverse dependencies:
[info] *:makePom::artifactPath
[info] Delegates:
[info] *:artifactName
[info] {.}/*:artifactName
[info] */*:artifactName
[info] Related:
[info] */*:artifactName
See how the default implementation is defined in sbt.Artifact.
Normally, when not using Git, you can just write:
addCompilerPlugin("something" % "blah" ...)
scalacOptions += "-P:blah:..."
This addCompilerPlugin takes a ModuleID. But here... I've tried adding:
lazy val root = project in file(".") dependsOn
uri("git://github.com/puffnfresh/wartremover.git#master")
to the project/plugins.sbt as well as:
lazy val wartRemover = RootProject(
uri("git://github.com/puffnfresh/wartremover.git#master"))
lazy val root = Project(...).settings(
...
scalacOptions += "-P:wartremover:...",
...
) depends on wartRemover
Both result in:
[error] (root/*:update) sbt.ResolveException: unresolved dependency:
org.brianmckenna#wartremover_2.10.3;0.5-SNAPSHOT: not found
tl;dr The project wartremover has not been published for Scala version 2.10.3. Downgrade yours with the following in build.sbt among the other necessary settings:
scalaVersion := "2.10.2"
Detailed procedure focusing on Scala 2.10.3
The following in build.sbt
addCompilerPlugin("org.brianmckenna" % "wartremover" % "0.5" cross CrossVersion.full)
adds org.brianmckenna:wartremover:0.5:plugin->default(compile) to libraryDependencies.
[sbt-0-13-2]> show libraryDependencies
[info] List(org.scala-lang:scala-library:2.10.3, org.brianmckenna:wartremover:0.5:plugin->default(compile))
So to use a RootProject that points at the project wartremover at GitHub I had to use the following in build.sbt (this is the complete file):
scalacOptions in root += "-P:wartremover:traverser:org.brianmckenna.wartremover.warts.Unsafe"
lazy val root = project in file(".") dependsOn wartRemover % "plugin->default(compile)"
lazy val wartRemover = RootProject(
uri("git://github.com/puffnfresh/wartremover.git#master"))
Since the project wartremover is not published for 2.10.3 I followed the steps below:
Show available projects
[root]> projects
[info] In file:/Users/jacek/sandbox/so/sbt-0.13.2/
[info] * root
[info] In git://github.com/puffnfresh/wartremover.git#master
[info] wartremover
Switch to wartremover and publishLocal it for scalaVersion set to 2.10.3.
[wartremover]> set scalaVersion := "2.10.3"
[info] Defining wartremover/*:scalaVersion
[info] The new value will be used by wartremover/*:allDependencies, wartremover/*:assemblyPackageScala::assemblyDefaultJarName and 12 others.
[info] Run `last` for details.
[info] Reapplying settings...
[info] Set current project to wartremover (in build git://github.com/puffnfresh/wartremover.git#master)
[wartremover]> publishLocal
[info] Packaging /Users/jacek/.sbt/0.13/staging/d6dd3d2e3d818e69943a/wartremover/target/scala-2.10/wartremover_2.10.3-0.6-SNAPSHOT-sources.jar ...
[info] Updating {git://github.com/puffnfresh/wartremover.git#master}wartremover...
...
[info] published ivy to /Users/jacek/.ivy2/local/org.brianmckenna/wartremover_2.10.3/0.6-SNAPSHOT/ivys/ivy.xml
[success] Total time: 7 s, completed Jan 18, 2014 11:34:07 PM
Switch to the project root and do update. It should now work fine.
[wartremover]> project {file:/Users/jacek/sandbox/so/sbt-0.13.2/}
[info] Set current project to root (in build file:/Users/jacek/sandbox/so/sbt-0.13.2/)
[root]> update
[info] Updating {file:/Users/jacek/sandbox/so/sbt-0.13.2/}root...
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] Done updating.
[success] Total time: 0 s, completed Jan 18, 2014 11:36:24 PM
console should work fine now, too.
[root]> console
[info] Starting scala interpreter...
[info]
<console>:5: error: var is disabled
var value: scala.tools.nsc.interpreter.IMain = _
^
Welcome to Scala version 2.10.3 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_45).
Type in expressions to have them evaluated.
Type :help for more information.
scala>
The error message worries me, though. I don't know the plugin and neither do I know how to get rid of it. It also happens when I follow the steps described in Compiler plugin when scalaVersion := "2.10.2" is set in build.sbt (so the compiler plugin's available in the repo).