sbt 0.13.2-M3 to 0.13.5-RC3 issue - sbt

I'm having a problem moving from sbt version 0.13.2-M3 to 0.13.5-RC3 where transitive dependencies that 13.2-M3 successfully resolves fail to be resolved by 0.13.5-RC3.
I get unresolved dependency errors where the version is "working#".
It's happening when I have a multi-project build with two sub-projects, one of which depends on the other. They both have dependencies whose maven poms specify a common parent (though I'm not sure if that's a red herring or not).
It only happens when the dependencies aren't already in the local ivy cache.
A minimal repro Build is:
import sbt._
import Keys._
object BarBuild extends Build {
val buildSettings = Seq(scalaVersion := "2.10.3")
lazy val root = Project(
id = "bar",
base = file(".")
) aggregate(withSolrCore, withSolrClient)
lazy val withSolrCore = Project(
id = "withSolrCore",
base = file("solrCore"),
settings = buildSettings ++ Seq(
libraryDependencies ++= Seq("org.apache.solr" % "solr-core" % "4.7.1")
)
) dependsOn (withSolrClient)
lazy val withSolrClient = Project(
id = "withSolrClient",
base = file("solrClient"),
settings = buildSettings ++ Seq(
libraryDependencies ++= Seq("org.apache.solr" % "solr-solrj" % "4.7.1")
)
)
}
With build.properties's sbt.version=0.13.5-RC3 I see lots of errors like
[warn] module not found: org.apache.lucene#lucene-analyzers-kuromoji;working#heraclitus.local
and
[error] unresolved dependency: org.apache.lucene#lucene-core;working#heraclitus.local: not found
but with sbt.version=0.13.2-M3 everything's peachy.
I'm not sure if I'm doing something wrong or something's up with sbt, but at this point I suspect the latter.
Thanks.

This is a known ivy issue. The workaround is to override the versions of all the dependencies in the full transitive closure that's breaking with the "real" versions to use. (I derived the real ones by running update on a stub project with only the problem dependency with an older version of sbt, 0.13.2, which is pre-ivy-bug), like,
dependencyOverrides ++= Set(
"com.google.guava" % "guava" % "14.0.1",
"com.google.protobuf" % "protobuf-java" % "2.5.0",
"com.googlecode.concurrentlinkedhashmap" % "concurrentlinkedhashmap-lru" % "1.2",
"com.spatial4j" % "spatial4j" % "0.4.1",
...
)

Related

When using a Scala compiler plugin in sbt, how do you set a library dependency for the plugin?

I'm using a compiler plugin I wrote that depends on the Kyro serialization library. When attempting to use my plugin I set this up in build.sbt (top-level) like this:
lazy val dependencies =
new {
val munit = "org.scalameta" %% "munit" % "0.7.12" % Test
val kyro = "com.esotericsoftware" % "kryo" % "5.0.0-RC9"
}
lazy val commonDependencies = Seq(
dependencies.kyro,
dependencies.munit
)
lazy val root = (project in file("."))
.settings(
libraryDependencies ++= commonDependencies,
Test / parallelExecution := false
)
addCompilerPlugin("co.blocke" %% "dotty-reflection" % reflectionLibVersion)
But when I compile my target project, I get a java.lang.NoClassDefFoundError that it can't find Kyro. I've added kyro to my dependencies, but since this is for the compiler, not my app, it's not picking that up.
How can I properly tell sbt about a dependency my plugin needs?

scalaFX standalone execute jar file

Good day! Help me, please. I startup this example
sbt> run
It's okey all play, after
sbt> package
Will build jar file, after double click messge:
Error: A JNI error has occured, please check your installation and try again.
Scala version: 2.12.4. JVM:1.8.0_152. ScalaFX:8.0.102-R11
hello.scala: `
package hello
import scalafx.Includes._
import scalafx.application.JFXApp
import scalafx.application.JFXApp.PrimaryStage
import scalafx.scene.Scene
import scalafx.scene.paint.Color._
import scalafx.scene.shape.Rectangle
object HelloStage extends JFXApp {
stage = new JFXApp.PrimaryStage {
title.value = "Hello Stage"
width = 600
height = 450
scene = new Scene {
fill = LightGreen
content = new Rectangle {
x = 25
y = 40
width = 100
height = 100
fill <== when(hover) choose Green otherwise Red
}
}
}
}
build.sbt:
name := "Scala"
organization := "scalafx.org"
version := "1.0.5"
scalaVersion := "2.12.4"
scalacOptions ++= Seq("-unchecked", "-deprecation", "-Xcheckinit", "-encoding", "utf8")
resourceDirectory in Compile := (scalaSource in Compile).value
libraryDependencies ++= Seq(
"org.scalafx" %% "scalafx" % "8.0.102-R11",)
addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full)
fork := true
This is a Java classpath issue. When you try to execute the resulting JAR file, it cannot find the jar files that it needs to run.
Try the following:
Firstly, copy & paste the following to project/plugins.sbt:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.5")
This loads the sbt-assembly plugin, which will create a fat JAR file, containing all of the dependencies.
Secondly, change your build.sbt file to the following:
name := "Scala"
organization := "scalafx.org"
version := "1.0.5"
scalaVersion := "2.12.4"
scalacOptions ++= Seq("-unchecked", "-deprecation", "-Xcheckinit", "-encoding", "utf8")
libraryDependencies += "org.scalafx" %% "scalafx" % "8.0.102-R11"
fork := true
mainClass in assembly := Some("hello.HelloStage")
This simplifies what you originally had. The macro paradise compiler plugin is not required, and I also removed the slightly odd resourceDirectory setting.
To create the fat JAR, run the command:
sbt
sbt> assembly
The JAR file you're looking for is most likely located at target/scala-2.12/Scala-assembly-1.0.5.jar. You should now be good to go...
Alternatively, you can add all the necessary JAR files to your classpath. Another plugin that can help with that (you probably shouldn't use it with sbt-assembly) - is sbt-native-packager, which creates installers for you. You can then install your app and run it like a regular application.

SBT one-jar Plugin Usage

I'm using the one-jar plugin to generate a fat jar file. Here is how my Build.scala looks like:
import com.github.retronym.SbtOneJar
import sbt._
import Keys._
object build extends Build {
def standardSettings = Seq(
exportJars := true
) ++ Defaults.defaultSettings
lazy val metricsProducer = Project("metricsProducer",
file("beta"),
settings = standardSettings ++ SbtOneJar.oneJarSettings
)
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "1.6.0",
"org.apache.kafka" %% "kafka" % "0.9.0.0"%,
"joda-time" % "joda-time" % "2.7" %,
"io.spray" %% "spray-json" % "1.3.2" %
)
}
When I tried to run this using:
sbt run one-jar
unresolved dependency: org.scala-sbt.plugins#sbt-onejar;0.8: not found
I have the dependency plug in added in the plugins.sbt. Any clues?
Not sure on sbt one-jar if it is still supported. I managed to get this working using the sbt assembly plugin.
https://github.com/sbt/sbt-assembly

SBT: How to set transitive dependencies of a dependency to "provided" later?

I have something like this in my build.sbt:
lazy val someDeps = Seq(
libraryDependencies += "com.example" %% "foo" % "1.3.37",
// more
)
lazy val some_library = project.in(file("libs/somelibrary")).
settings(commonSettings).
settings(
// project-specific settings
libraryDependencies ++= someDeps
)
lazy val something_with_deps_provided = project.in(file("swdp")).
settings(commonSettings).
settings(
// project-specific settings
libraryDependencies ++= someDeps.map(d => d % "provided")
).dependsOn(some_library)
When I now use the sbt-assembly-plugin to create the assembly of something_with_deps_provided, it still puts the dependencies into the resulting jar, ignoring the provided. Is it possible to set a transitive dependency to provided later and if yes, how is it done?
In cases like this, excludeDependencies can be used as described in
SBT manual here:
Exclude Transitive Dependencies.
With your example:
lazy val something_with_deps_provided = project.in(file("swdp"))
.settings(commonSettings)
.dependsOn(some_library)
.settings(
// project-specific settings
excludeDependencies ++= someDeps.map { d =>
ExclusionRule(
organization = d.organization,
name = d.name
)
}
)
The dependencies from someDeps will no longer be included in the
assembly JAR for something_with_deps_provided project.

Using sbt-aether-deploy with sbt-native-packager

Has anyone published an sbt-native-packager produced artifact (tgz in my case) using sbt-aether-deploy to a nexus repo? (I need this for the timestamped snapshots, specifically the "correct" version tag in nexus' artifact-resolution REST resource).
I can do one or the other but can't figure out how to add the packagedArtifacts in Universal to the artifacts that sbt-aether-deploy deploys to do both.
I suspect the path to pursue would be to the addArtifact() the packagedArtifacts in Universal or creating another AetherArtifact and then to override/replace the deployTask to use that AetherArtifact?
Any help much appreciated.
I am the author of the sbt-aether-deploy plugin, and I just came over this post.
import aether.AetherKeys._
crossPaths := false //needed if you want to remove the scala version from the artifact name
enablePlugins(JavaAppPackaging)
aetherArtifact := {
val artifact = aetherArtifact.value
artifact.attach((packageBin in Universal).value, "dist", "zip")
}
This will also publish the other main artifact.
If you want to disable publishing of the main artifact, then you will need to rewrite the artifact coordinates. Maven requires a main artifact.
I have added a way to replace the main artifact for this purpose, but I can now see that way is kind of flawed. It will still assume that the artifact is published as a jar file. The main artifact type is locked down to that, since the POM packaging is set to jar by default by SBT.
If this is an app, then that limitation is probably OK, since Maven will never resolve that into an artifact.
The "proper" way in Maven terms is to add a classifier to the artifact and change the "packaging" in the POM file to "pom". We will see if I get around to changing that particular part.
Ok, I think I got it amazingly enough. If there's a better way to do it I'd love to hear. Not loving that blind Option.get there..
val tgzCoordinates = SettingKey[MavenCoordinates]("the maven coordinates for the tgz")
lazy val myPackagerSettings = packageArchetype.java_application ++ deploymentSettings ++ Seq(
publish <<= publish.dependsOn(publish in Universal),
publishLocal <<= publishLocal.dependsOn(publishLocal in Universal)
)
lazy val defaultSettings = buildSettings ++ Publish.settings ++ Seq(
scalacOptions in Compile ++= Seq("-encoding", "UTF-8", "-target:jvm-1.7", "-deprecation", "-feature", "-unchecked", "-Xlog-reflective-calls"),
testOptions in Test += Tests.Argument("-oDF")
)
lazy val myAetherSettings = aetherSettings ++ aetherPublishBothSettings
lazy val toastyphoenixProject = Project(
id = "toastyphoenix",
base = file("."),
settings = defaultSettings ++ myPackagerSettings ++ myAetherSettings ++ Seq(
name in Universal := name.value + "_" + scalaBinaryVersion.value,
packagedArtifacts in Universal ~= { _.filterNot { case (artifact, file) => artifact.`type`.contains("zip")}},
libraryDependencies ++= Dependencies.phoenix,
tgzCoordinates := MavenCoordinates(organization.value + ":" + (name in Universal).value + ":tgz:" + version.value).get,
aetherArtifact <<= (tgzCoordinates, packageZipTarball in Universal, makePom in Compile, packagedArtifacts in Universal) map {
(coords: MavenCoordinates, mainArtifact: File, pom: File, artifacts: Map[Artifact, File]) =>
createArtifact(artifacts, pom, coords, mainArtifact)
}
)
)
I took Peter's solution and reworked it slightly, avoiding the naked Option.get by creating the MavenCoordinates directly:
import aether.MavenCoordinates
import aether.Aether.createArtifact
name := "mrb-test"
organization := "me.mbarton"
version := "1.0"
crossPaths := false
packageArchetype.java_application
publish <<= (publish) dependsOn (publish in Universal)
publishLocal <<= (publishLocal) dependsOn (publishLocal in Universal)
aetherPublishBothSettings
aetherArtifact <<= (organization, name in Universal, version, packageBin in Universal, makePom in Compile, packagedArtifacts in Universal) map {
(organization, name, version, binary, pom, artifacts) =>
val nameWithoutVersion = name.replace(s"-$version", "")
createArtifact(artifacts, pom, MavenCoordinates(organization, nameWithoutVersion, version, None, "zip"), binary)
}
The nameWithoutVersion replace works around SBT native packager including the version in the artifact name:
Before: me/mbarton/mrb-test-1.0/1.0/mrb-test-1.0.zip
After: me/mbarton/mrb-test/1.0/mrb-test-1.0.zip
crossPaths avoids the Scala postfix on the version.

Resources