how to define multi-project build with sbt-native-packager that creates an RPM for each sub-project - sbt

In all of the examples I've seen regarding multi-module builds and sbt-native-packager, they all aggregate the sub-projects into a single package. I have sub-projects that each provide a micro-service. I believe that each of these should have it's own native package, but I don't see how to do that and have a one command build for all of the sub-projects.

This turns out be straightforward. Simply provide native-packager settings for each of the sub-projects that you want to package and don't provide any on the aggregating project.
I tested by modifying https://github.com/muuki88/sbt-native-packager-examples/tree/master/multi-module-build accordingly:
import NativePackagerKeys._
name := "mukis-fullstack"
// used like the groupId in maven
organization in ThisBuild := "de.mukis"
// all sub projects have the same version
version in ThisBuild := "1.0"
scalaVersion in ThisBuild := "2.11.2"
// common dependencies
libraryDependencies in ThisBuild ++= Seq(
"com.typesafe" % "config" % "1.2.0"
)
// this is the root project, aggregating all sub projects
lazy val root = Project(
id = "root",
base = file("."),
// configure your native packaging settings here
// settings = packageArchetype.java_server++ Seq(
// maintainer := "John Smith <john.smith#example.com>",
// packageDescription := "Fullstack Application",
// packageSummary := "Fullstack Application",
// entrypoint
// mainClass in Compile := Some("de.mukis.frontend.ProductionServer")
// ),
// always run all commands on each sub project
aggregate = Seq(frontend, backend, api)
) dependsOn(frontend, backend, api) // this does the actual aggregation
// --------- Project Frontend ------------------
lazy val frontend = Project(
id = "frontend",
base = file("frontend"),
settings = packageArchetype.java_server++ Seq(
maintainer := "John Smith <john.smith#example.com>",
packageDescription := "Frontend appplication",
mainClass in Compile := Some("de.mukis.frontend.ProductionServer")
)
) dependsOn(api)
// --------- Project Backend ----------------
lazy val backend = Project(
id = "backend",
base = file("backend"),
settings = packageArchetype.java_server++ Seq(
maintainer := "John Smith <john.smith#example.com>",
packageDescription := "Fullstack Application",
packageSummary := "Fullstack Application",
// entrypoint
mainClass in Compile := Some("de.mukis.frontend.ProductionServer")
)
) dependsOn(api)
// --------- Project API ------------------
lazy val api = Project(
id = "api",
base = file("api")
Results:
debian:packageBin
...misc messages elided...
[info] dpkg-deb: building package `frontend' in `../frontend_1.0_all.deb'.
[info] dpkg-deb: building package `backend' in `../backend_1.0_all.deb'.

For whom just ended up here, a more up-to-date answer could look like:
lazy val root = (project in file("."))
.aggregate(common, frontend, backend)
lazy val common = (project in file("common"))
lazy val frontend = (project in file("frontend"))
.enablePlugins(JavaServerAppPackaging)
lazy val backend = (project in file("backend"))
.dependsOn(common)
.enablePlugins(JavaAppPackaging)
.settings(javaPackagingSettings)
lazy val javaPackagingSettings = Seq(
// follow sbt-native-packager to identify settings you need
)
Description
Here is the scenario supporting the above configuration
Project root is the parent and we don't want to package it. It aggregates other subprojects.
Project common is a sort of library and also we don't want to package it
Project backend depends on common for the libraries.
Project frontend is a standalone project packaged as a Java server app with default configuration

Related

When using a Scala compiler plugin in sbt, how do you set a library dependency for the plugin?

I'm using a compiler plugin I wrote that depends on the Kyro serialization library. When attempting to use my plugin I set this up in build.sbt (top-level) like this:
lazy val dependencies =
new {
val munit = "org.scalameta" %% "munit" % "0.7.12" % Test
val kyro = "com.esotericsoftware" % "kryo" % "5.0.0-RC9"
}
lazy val commonDependencies = Seq(
dependencies.kyro,
dependencies.munit
)
lazy val root = (project in file("."))
.settings(
libraryDependencies ++= commonDependencies,
Test / parallelExecution := false
)
addCompilerPlugin("co.blocke" %% "dotty-reflection" % reflectionLibVersion)
But when I compile my target project, I get a java.lang.NoClassDefFoundError that it can't find Kyro. I've added kyro to my dependencies, but since this is for the compiler, not my app, it's not picking that up.
How can I properly tell sbt about a dependency my plugin needs?

How to use sbt packageBin with multiple modules in order

I have a sbt project with multiple sub modules which look like this:
--\ root
-- module 1
-- module 2
Using packageBin, I can get two zip files: module1.zip and module2.zip.
This is my build.sbt:
import Dependencies._
import NativePackagerHelper._
lazy val commonSettings = Seq(
organization := "com.zhyea.sbt",
version := "0.1-SNAPSHOT",
scalaVersion := "2.11.12",
exportJars := true,
artifactName := {
(sv: ScalaVersion, module: ModuleID, artifact: Artifact) => artifact.name + "." + artifact.extension
}
)
lazy val module2 = project.settings(commonSettings).settings()
.enablePlugins(JavaAppPackaging, UniversalPlugin)
.settings(libraryDependencies ++= module2Dependencies)
lazy val module1 = project.settings(commonSettings)
.enablePlugins(JavaAppPackaging, UniversalPlugin)
.settings(libraryDependencies ++= module1Dependencies)
lazy val root = project.in(file("."))
.settings(commonSettings)
.aggregate(module2, module1)
.enablePlugins(JavaAppPackaging, UniversalPlugin)
.dependsOn(module2, module1).configs()
mappings in Universal ++= directory("module2/target/universal")
The mappings in Universal ++= directory("module1/target/universal")
Now I want to execute the packageBin task at root and add sub module zips into root.zip.
The problem is that when the root module executes packageBin task, the sub modules' packageBin tasks haven't finished, and the root cannot get module1.zip and mudule2.zip.
How can I tell sbt to execute the packageBin task in order?
i just pack all sub modules' files into one zip by adding a new module named pack.

Unresolved dependency in published assembly package in SBT

In my project there are 3 sub projects under root. build.sbt is as below.
proj_C depends on proj_A and proj_B.
If I created the assembly proj_C package with below command. It success and the assembly package could be imported in other projects.
sbt "project proj_C" assembly
If I publish with "sbt publish", as I defined addArtifact in proj_C settings, an assembly jar package is also generated and then published. But when I try to compile another project which imports this assembly jar, it will below error
[error] unresolved dependency: proj_A;1.0.0: not found
part of build.sbt is as below. Could anyone point me what I made wrong in my way?
Many thanks!
artifact in (Compile, assembly) := {
val art = (artifact in (Compile, assembly)).value
art.withClassifier(Some("assembly"))
}
lazy val assemblySettings = Seq(
assemblyMergeStrategy in assembly := {
{
case PathList("META-INF", xs # _*) => MergeStrategy.discard
case _ => MergeStrategy.first
}
}
)
lazy val root = Project(base = file("."))
.disablePlugins(sbtassembly.AssemblyPlugin)
.aggregate(proj_A, proj_B, proj_C)
.settings(
commonSettings,
skip in publish := true,
name := "proj_root"
)
lazy val proj_A= (project in file("proj_A"))
.disablePlugins(sbtassembly.AssemblyPlugin)
.settings(
commonSettings,
skip in publish := true,
name := "proj_A"
)
lazy val proj_B= (project in file("proj_B"))
.disablePlugins(sbtassembly.AssemblyPlugin)
.settings(
commonSettings,
skip in publish := true,
name := "proj_B"
)
lazy val proj_C= (project in file("proj_C"))
.settings(
commonSettings,
assemblySettings,
addArtifact(artifact in (Compile, assembly), assembly),
name := "proj_C"
) dependsOn(proj_A, proj_B)
First of all, I hope you know that the publishing of the fat jar is not recommended. And to be honest, in your case I really see no benefit in doing so.
If you simply publish A, B, C separately and then add the dependency in your other project it will all be automatically downloaded (along with dependencies of those projects). And the dependency management will be much easier...
But, since you want to add the A-assembly dependency, by the error I guess that you are actually adding the wrong jar. My guess would be that you publish both C.jar and C-assembly.jar, and you added the dependency like:
"your.organisation" %% "C" % "version"
but you should have:
"your.organisation" %% "C" % "version" classifier "assembly"

How to setup sbt-native-packager in a single module project with multiple mains

I have a single module client-server project with a main for each.
I'm trying to use sbt-native-packager to generate start-script for both.
project/P.scala
object Tactic extends Build {
lazy val root =
(project in file(".")).
configs(Client, Server)
.settings( inConfig(Client)(Defaults.configTasks) : _*)
.settings( inConfig(Server)(Defaults.configTasks) : _*)
lazy val Client = config("client") extend Compile
lazy val Server = config("server") extend Compile
}
build.sbt
mainClass in Client := Some("myProject.Client")
mainClass in Server := Some("myProject.Server")
enablePlugins(JavaAppPackaging)
When I run client:stage the directory target/universal/stage/lib is created with all the necessary jars but the bin directory is missing. What am I doing wrong?
Subsidiary question: what is the key to set the starting script name?
I would recommend setting up your project as a multi-module build, instead of creating and using new configurations. I tried your multiple configuration route and it gets hairy very quickly.
For example (I created a shared project for anything shared between client & server):
def commonSettings(module: String) = Seq[Setting[_]](
organization := "org.tactic",
name := s"tactic-$module",
version := "1.0-SNAPSHOT",
scalaVersion := "2.11.6"
)
lazy val root = (project in file(".")
settings(commonSettings("root"))
dependsOn (shared, client, server)
aggregate (shared, client, server)
)
val shared = (project
settings(commonSettings("shared"))
)
val client = (project
settings(commonSettings("client"))
enablePlugins JavaAppPackaging
dependsOn shared
)
val server = (project
settings(commonSettings("server"))
enablePlugins JavaAppPackaging
dependsOn shared
)
Note I'm enabling sbt-native-packager's JavaAppPackaging in the client and server.
Then run stage.
Also, the key for the starting script name is executableScriptName.

Using sbt-aether-deploy with sbt-native-packager

Has anyone published an sbt-native-packager produced artifact (tgz in my case) using sbt-aether-deploy to a nexus repo? (I need this for the timestamped snapshots, specifically the "correct" version tag in nexus' artifact-resolution REST resource).
I can do one or the other but can't figure out how to add the packagedArtifacts in Universal to the artifacts that sbt-aether-deploy deploys to do both.
I suspect the path to pursue would be to the addArtifact() the packagedArtifacts in Universal or creating another AetherArtifact and then to override/replace the deployTask to use that AetherArtifact?
Any help much appreciated.
I am the author of the sbt-aether-deploy plugin, and I just came over this post.
import aether.AetherKeys._
crossPaths := false //needed if you want to remove the scala version from the artifact name
enablePlugins(JavaAppPackaging)
aetherArtifact := {
val artifact = aetherArtifact.value
artifact.attach((packageBin in Universal).value, "dist", "zip")
}
This will also publish the other main artifact.
If you want to disable publishing of the main artifact, then you will need to rewrite the artifact coordinates. Maven requires a main artifact.
I have added a way to replace the main artifact for this purpose, but I can now see that way is kind of flawed. It will still assume that the artifact is published as a jar file. The main artifact type is locked down to that, since the POM packaging is set to jar by default by SBT.
If this is an app, then that limitation is probably OK, since Maven will never resolve that into an artifact.
The "proper" way in Maven terms is to add a classifier to the artifact and change the "packaging" in the POM file to "pom". We will see if I get around to changing that particular part.
Ok, I think I got it amazingly enough. If there's a better way to do it I'd love to hear. Not loving that blind Option.get there..
val tgzCoordinates = SettingKey[MavenCoordinates]("the maven coordinates for the tgz")
lazy val myPackagerSettings = packageArchetype.java_application ++ deploymentSettings ++ Seq(
publish <<= publish.dependsOn(publish in Universal),
publishLocal <<= publishLocal.dependsOn(publishLocal in Universal)
)
lazy val defaultSettings = buildSettings ++ Publish.settings ++ Seq(
scalacOptions in Compile ++= Seq("-encoding", "UTF-8", "-target:jvm-1.7", "-deprecation", "-feature", "-unchecked", "-Xlog-reflective-calls"),
testOptions in Test += Tests.Argument("-oDF")
)
lazy val myAetherSettings = aetherSettings ++ aetherPublishBothSettings
lazy val toastyphoenixProject = Project(
id = "toastyphoenix",
base = file("."),
settings = defaultSettings ++ myPackagerSettings ++ myAetherSettings ++ Seq(
name in Universal := name.value + "_" + scalaBinaryVersion.value,
packagedArtifacts in Universal ~= { _.filterNot { case (artifact, file) => artifact.`type`.contains("zip")}},
libraryDependencies ++= Dependencies.phoenix,
tgzCoordinates := MavenCoordinates(organization.value + ":" + (name in Universal).value + ":tgz:" + version.value).get,
aetherArtifact <<= (tgzCoordinates, packageZipTarball in Universal, makePom in Compile, packagedArtifacts in Universal) map {
(coords: MavenCoordinates, mainArtifact: File, pom: File, artifacts: Map[Artifact, File]) =>
createArtifact(artifacts, pom, coords, mainArtifact)
}
)
)
I took Peter's solution and reworked it slightly, avoiding the naked Option.get by creating the MavenCoordinates directly:
import aether.MavenCoordinates
import aether.Aether.createArtifact
name := "mrb-test"
organization := "me.mbarton"
version := "1.0"
crossPaths := false
packageArchetype.java_application
publish <<= (publish) dependsOn (publish in Universal)
publishLocal <<= (publishLocal) dependsOn (publishLocal in Universal)
aetherPublishBothSettings
aetherArtifact <<= (organization, name in Universal, version, packageBin in Universal, makePom in Compile, packagedArtifacts in Universal) map {
(organization, name, version, binary, pom, artifacts) =>
val nameWithoutVersion = name.replace(s"-$version", "")
createArtifact(artifacts, pom, MavenCoordinates(organization, nameWithoutVersion, version, None, "zip"), binary)
}
The nameWithoutVersion replace works around SBT native packager including the version in the artifact name:
Before: me/mbarton/mrb-test-1.0/1.0/mrb-test-1.0.zip
After: me/mbarton/mrb-test/1.0/mrb-test-1.0.zip
crossPaths avoids the Scala postfix on the version.

Resources