I have an SBT project, with spark dependencies. These dependencies are provided at runtime, and hence I import them under provided scope.
val hadoop = Seq("org.apache.hadoop" % "hadoop-client" % "3.3.1" % provided)
val spark = Seq(
"org.apache.spark" %% "spark-core" % SparkVersion % provided,
"org.apache.spark" %% "spark-sql" % SparkVersion % provided,
"org.apache.spark" %% "spark-mllib" % SparkVersion % provided
)
lazy val coreDto = (project in file("xxxx"))
.enablePlugins(BuildInfoPlugin)
.enablePlugins(PackPlugin)
.settings(
name := "xxxx",
moduleName := "xxxx",
version := "1.0",
libraryDependencies ++= (hadoop ++ spark))
All is well till now. Now I have a new scenario, where I have to publish the jar to our maven repository. And successfully I am able to publish it. The issue now is: provided scope. As the compile time dependencies are not appropriately set.
Question: How do I configure, where the provided scope is ignored during publishing? But still considered when I package it?
Using this to publish in case if helpful
lazy val publishSettings = Seq(
publishMavenStyle := true,
publishTo := {
val url = "https://xxxxl/maven/v1/"
if (version.value.trim.toUpperCase.endsWith("SNAPSHOT"))
Some("snapshots".at(url))
else
Some("releases".at(url))
},
aetherDeploy / logLevel := Level.Info,
aetherOldVersionMethod := true,
credentials += Credentials(Path.userHome / ".sbt" / ".credentials")
)
Related
I'm using a compiler plugin I wrote that depends on the Kyro serialization library. When attempting to use my plugin I set this up in build.sbt (top-level) like this:
lazy val dependencies =
new {
val munit = "org.scalameta" %% "munit" % "0.7.12" % Test
val kyro = "com.esotericsoftware" % "kryo" % "5.0.0-RC9"
}
lazy val commonDependencies = Seq(
dependencies.kyro,
dependencies.munit
)
lazy val root = (project in file("."))
.settings(
libraryDependencies ++= commonDependencies,
Test / parallelExecution := false
)
addCompilerPlugin("co.blocke" %% "dotty-reflection" % reflectionLibVersion)
But when I compile my target project, I get a java.lang.NoClassDefFoundError that it can't find Kyro. I've added kyro to my dependencies, but since this is for the compiler, not my app, it's not picking that up.
How can I properly tell sbt about a dependency my plugin needs?
I can sbt assembly myself a fat jar without an issue with the below build.sbt file. However when I try to publish this "fat jar", sbt publish dumps only 1kb .jar files in the s3 bucket.
Unzipping the .jar file shows that it only contains a manifest file.
How do I get the fat jar into my repo?
update: striked text has been altered since initial question was posed. Removed the name override and it now publishes the build code but without the external libraries
below, my build.sbt file
name := "util_myutil"
version := "1.0.1"
scalaVersion := "2.10.4"
scalacOptions += "-target:jvm-1.7"
libraryDependencies += "org.apache.spark" % "spark-core_2.10" % "1.5.0-cdh5.5.2" % "provided"
unmanagedJars in Compile += file(".lib/my.jar")
unmanagedJars in Compile += file(".lib/some_other.jar")
assemblyOption in assembly := (assemblyOption in assembly).value.copy(includeScala = false)
assemblyJarName in assembly := s"${name.value}-${version.value}.jar"
ivyScala := ivyScala.value map { _.copy(overrideScalaVersion = true) }
resolvers ++= Seq(
"Cloudera repos" at "https://repository.cloudera.com/artifactory/cloudera-repos",
"Cloudera releases" at "https://repository.cloudera.com/artifactory/libs-release",
"Era7 maven releases" at "https://s3-eu-west-1.amazonaws.com/releases.era7.com"
)
s3sse := true
s3region := com.amazonaws.services.s3.model.Region.US_Standard
s3acl := com.amazonaws.services.s3.model.CannedAccessControlList.Private
s3overwrite := true
publishMavenStyle := true
publishTo := {
val suffix = if (isSnapshot.value) "snapshots" else "releases"
Some(s3resolver.value(s"IT Insights Artifacts $suffix", s3("my-mvn-repo." + suffix)))
}
from https://github.com/sbt/sbt-assembly:
add this to your build.sbt:
artifact in (Compile, assembly) := {
val art = (artifact in (Compile, assembly)).value
art.copy(`classifier` = Some("assembly"))
}
addArtifact(artifact in (Compile, assembly), assembly)
I have something like this in my build.sbt:
lazy val someDeps = Seq(
libraryDependencies += "com.example" %% "foo" % "1.3.37",
// more
)
lazy val some_library = project.in(file("libs/somelibrary")).
settings(commonSettings).
settings(
// project-specific settings
libraryDependencies ++= someDeps
)
lazy val something_with_deps_provided = project.in(file("swdp")).
settings(commonSettings).
settings(
// project-specific settings
libraryDependencies ++= someDeps.map(d => d % "provided")
).dependsOn(some_library)
When I now use the sbt-assembly-plugin to create the assembly of something_with_deps_provided, it still puts the dependencies into the resulting jar, ignoring the provided. Is it possible to set a transitive dependency to provided later and if yes, how is it done?
In cases like this, excludeDependencies can be used as described in
SBT manual here:
Exclude Transitive Dependencies.
With your example:
lazy val something_with_deps_provided = project.in(file("swdp"))
.settings(commonSettings)
.dependsOn(some_library)
.settings(
// project-specific settings
excludeDependencies ++= someDeps.map { d =>
ExclusionRule(
organization = d.organization,
name = d.name
)
}
)
The dependencies from someDeps will no longer be included in the
assembly JAR for something_with_deps_provided project.
I want to publish a Scala library with sbt using sbt-pgp 0.8. I've registered groupId org.bitbucket.sergey_kozlov at Sonatype.
My build.sbt:
organization := "org.bitbucket.sergey_kozlov"
name := "playingcards"
version := "0.1-SNAPSHOT"
publishMavenStyle := true
publishTo := {
val nexus = "https://oss.sonatype.org/"
if (isSnapshot.value)
Some("snapshots" at nexus + "content/repositories/snapshots")
else
Some("releases" at nexus + "service/local/staging/deploy/maven2")
}
publishArtifact in Test := false
pomIncludeRepository := { _ => false }
pomExtra :=
<url>https://bitbucket.org/sergey_kozlov/playingcards</url>
<licenses>
<license>
<name>The MIT License</name>
<url>http://www.opensource.org/licenses/mit-license.php</url>
<distribution>repo</distribution>
</license>
</licenses>
<scm>
<url>https://bitbucket.org/sergey_kozlov/playingcards.git</url>
<connection>scm:git:ssh://git#bitbucket.org/sergey_kozlov/playingcards.git</connection>
</scm>
<developers>
<developer>
<id>skozlov</id>
<name>Sergey Kozlov</name>
<email>mail.sergey.kozlov#gmail.com</email>
<roles>
<role>architect</role>
<role>developer</role>
</roles>
</developer>
</developers>
libraryDependencies += "junit" % "junit" % "4.11"
libraryDependencies += "org.scalatest" % "scalatest_2.10" % "2.0" % "test"
There's also ~/.sbt/0.13/plugins/gpg.sbt:
addSbtPlugin("com.typesafe.sbt" % "sbt-pgp" % "0.8")
No other files are under project/ directory that contribute to the build definition.
When I enter publishSigned in sbt console, I get the following error:
[error] (*:publishSigned) java.io.IOException: Access to URL https://oss.sonatype.org/content/repositories/snapshots/playingcards/playingcards_2.10/0.1-SNAPSHOT/playingcards_2.10-0.1-SNAPSHOT-sources.jar was refused by the server: Forbidden
Note that the URL does not contain organization.
How can I publish my artifact correctly?
As you pointed out your URL is missing organization property this is why you get this error. Try to run show organization in sbt console to be sure that your organization property is correct. If it doesn't help try to specify your project explicitly in sbt and set organization property there.
lazy val core = (project in file(".")).settings(
organization := "org.bitbucket.sergey_kozlov"
//other properties here
)
I have the next configuration:
lazy val mainProject = Project(
id = "project-helper",
base = file("."),
settings = Project.defaultSettings ++ Seq(
name := "my-first-project",
version := "0.1-SNAPSHOT",
scalaVersion := "2.10.2",
licenses += ("MIT", url("http://opensource.org/licenses/MIT")),
publishMavenStyle := false,
pomExtra := pomXml,
publishArtifact in Test := false,
resolvers += "Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases",
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % "2.10.3",
"org.scalamacros" % "quasiquotes_2.10.3" % "2.0.0-M3"
),
addCompilerPlugin("org.scalamacros" % "paradise" % "2.0.0-M3" cross CrossVersion.full)
)
)
I want publish this into a bintray.
I do publish but i got error:
java.lang.RuntimeException: Repository for publishing is not specified.
at scala.sys.package$.error(package.scala:27)
at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:1203)
at sbt.Classpaths$$anonfun$getPublishTo$1.apply(Defaults.scala:1203)
at scala.Option.getOrElse(Option.scala:120)
I use a bintray-sbt plugin.
Thanks
See the README:
Publishing
To publish a package to bintray, you need a bintray account. You can do so here. After creating a bintray account you can add
seq(bintrayPublishSettings:_*)
You likely need something like this.
import bintray.Plugin._
lazy val mainProject = Project(id = "project-helper", base = file(".")).
settings(bintrayPublishSettings: _*).
settings(
name := "my-first-project",
version := "0.1-SNAPSHOT",
scalaVersion := "2.10.2",
licenses += ("MIT", url("http://opensource.org/licenses/MIT")),
publishMavenStyle := false,
pomExtra := pomXml,
publishArtifact in Test := false,
resolvers += "Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases",
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-reflect" % "2.10.3",
"org.scalamacros" % "quasiquotes_2.10.3" % "2.0.0-M3"
),
addCompilerPlugin("org.scalamacros" % "paradise" % "2.0.0-M3" cross CrossVersion.full)
)
Same way (but different) could be to just append to settings:
++ bintray.Plugin.bintraySettings