I am trying to connect HBase from my sbt scala project. I need to set hbase-site.xml in the classpath. How can I do this? HBase is located in virtual machine and sbt scala project is located on my windows machine.
I suggest that you try add the following dependency settings in your build.sbt, assuming your hbase-site.xml resides under /path/to/hbase/conf/:
unmanagedJars in Compile ++= {
val baseDir = baseDirectory.value
val customDirs = ("/path/to/hbase/conf") +++ ("/path/to/hadoop/conf")
val customJars = (baseDir ** "*.jar") +++ (customDirs ** "*.jar")
customJars.classpath
}
More sbt library management info can be found here.
[UPDATE]
In addition to the reported error, custom libs in this case shouldn't be limited to *.jar. See revised settings below:
unmanagedJars in Compile ++= {
val baseDir = baseDirectory.value
val customLibs = (baseDir ** "*.jar") +++ file("/path/to/hbase/conf") +++ file("/path/to/hadoop/conf")
customLibs.classpath
}
Here's an alternative way using map:
unmanagedJars in Compile <++= baseDirectory map { base =>
val baseJars = (base ** "*.jar")
val customLibs = baseJars +++ file("/path/to/hbase/conf") +++ file("/path/to/hadoop/conf")
customLibs.classpath
}
Related
I'm using a compiler plugin I wrote that depends on the Kyro serialization library. When attempting to use my plugin I set this up in build.sbt (top-level) like this:
lazy val dependencies =
new {
val munit = "org.scalameta" %% "munit" % "0.7.12" % Test
val kyro = "com.esotericsoftware" % "kryo" % "5.0.0-RC9"
}
lazy val commonDependencies = Seq(
dependencies.kyro,
dependencies.munit
)
lazy val root = (project in file("."))
.settings(
libraryDependencies ++= commonDependencies,
Test / parallelExecution := false
)
addCompilerPlugin("co.blocke" %% "dotty-reflection" % reflectionLibVersion)
But when I compile my target project, I get a java.lang.NoClassDefFoundError that it can't find Kyro. I've added kyro to my dependencies, but since this is for the compiler, not my app, it's not picking that up.
How can I properly tell sbt about a dependency my plugin needs?
Good day! Help me, please. I startup this example
sbt> run
It's okey all play, after
sbt> package
Will build jar file, after double click messge:
Error: A JNI error has occured, please check your installation and try again.
Scala version: 2.12.4. JVM:1.8.0_152. ScalaFX:8.0.102-R11
hello.scala: `
package hello
import scalafx.Includes._
import scalafx.application.JFXApp
import scalafx.application.JFXApp.PrimaryStage
import scalafx.scene.Scene
import scalafx.scene.paint.Color._
import scalafx.scene.shape.Rectangle
object HelloStage extends JFXApp {
stage = new JFXApp.PrimaryStage {
title.value = "Hello Stage"
width = 600
height = 450
scene = new Scene {
fill = LightGreen
content = new Rectangle {
x = 25
y = 40
width = 100
height = 100
fill <== when(hover) choose Green otherwise Red
}
}
}
}
build.sbt:
name := "Scala"
organization := "scalafx.org"
version := "1.0.5"
scalaVersion := "2.12.4"
scalacOptions ++= Seq("-unchecked", "-deprecation", "-Xcheckinit", "-encoding", "utf8")
resourceDirectory in Compile := (scalaSource in Compile).value
libraryDependencies ++= Seq(
"org.scalafx" %% "scalafx" % "8.0.102-R11",)
addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full)
fork := true
This is a Java classpath issue. When you try to execute the resulting JAR file, it cannot find the jar files that it needs to run.
Try the following:
Firstly, copy & paste the following to project/plugins.sbt:
addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.5")
This loads the sbt-assembly plugin, which will create a fat JAR file, containing all of the dependencies.
Secondly, change your build.sbt file to the following:
name := "Scala"
organization := "scalafx.org"
version := "1.0.5"
scalaVersion := "2.12.4"
scalacOptions ++= Seq("-unchecked", "-deprecation", "-Xcheckinit", "-encoding", "utf8")
libraryDependencies += "org.scalafx" %% "scalafx" % "8.0.102-R11"
fork := true
mainClass in assembly := Some("hello.HelloStage")
This simplifies what you originally had. The macro paradise compiler plugin is not required, and I also removed the slightly odd resourceDirectory setting.
To create the fat JAR, run the command:
sbt
sbt> assembly
The JAR file you're looking for is most likely located at target/scala-2.12/Scala-assembly-1.0.5.jar. You should now be good to go...
Alternatively, you can add all the necessary JAR files to your classpath. Another plugin that can help with that (you probably shouldn't use it with sbt-assembly) - is sbt-native-packager, which creates installers for you. You can then install your app and run it like a regular application.
I have a project with layout
MyProject
-- src
-- target
-- myfiles
-- myfile1.txt
-- myfile2.txt
-- built.sbt
Now I want to package myfiles/myfile1.txt and myfiles/myfile2.txt into the output jar.
I tried to include this to the build.sbt file:
mappings in (Compile, packageBin) += {
baseDirectory.value / "myfiles" / "myfile1.txt" -> "File/myfile1.txt"
}
mappings in (Compile, packageBin) += {
baseDirectory.value / "myfiles" / "myfile2.txt" -> "File/myfile2.txt"
}
but when I am using sbt assembly command, they are not included in the jar.
Where is wrong?
I am trying to publish an SBT plugin to a repository. I'm not sure if this has any relevance, but our plugin loads the sbt-twirl plugin - Googling around, it seems like publishConfiguration might be overriden:
new PublishConfiguration(None, "dotM2", arts, Seq(), level)
When I run the publish task, artifacts are deployed to the repo, but the sbt task then fails:
sbt (my-sbt-plugin)> publish
[info] Loading global plugins from ...
...
[info] Done packaging.
[info] published sbt-my-sbt-plugin to http://my.repo.com/.../sbt-my-sbt-plugin-0.1-SNAPSHOT.jar
java.lang.RuntimeException: Repository for publishing is not specified.
.... stack trace here ....
[error] (my-sbt-plugin/*:publishConfiguration) Repository for publishing is not specified.
What is causing the error, and what could I do to stop the publishing from failing?
** Update ** Here is inspect publish
sbt (my-sbt-plugin)> inspect publish
[info] Task: Unit
[info] Description:
[info] Publishes artifacts to a repository.
[info] Provided by:
[info] {file:/path/to/my-sbt-plugin/}my-sbt-plugin/*:publish
[info] Defined at:
[info] (sbt.Classpaths) Defaults.scala:988
[info] Dependencies:
[info] my-sbt-plugin/*:ivyModule
[info] my-sbt-plugin/*:publishConfiguration
[info] my-sbt-plugin/*:publish::streams
[info] Delegates:
[info] my-sbt-plugin/*:publish
[info] {.}/*:publish
[info] */*:publish
[info] Related:
[info] plugin/*:publish
Here's how I've configured publishing (with some of the plugin settings, excluding libraryDependencies and 1 or 2 other settings)
lazy val plugin = project
.settings(publishSbtPlugin: _*)
.settings(
name := "my-sbt-plugin",
sbtPlugin := true,
addSbtPlugin("com.typesafe.sbt" % "sbt-twirl" % "1.0.2")
)
def publishSbtPlugin = Seq(
publishMavenStyle := true,
publishTo := {
val myrepo = "http://myrepo.tld/"
if (isSnapshot.value) Some("The Realm" at myrepo + "snapshots")
else Some("The Realm" at myrepo + "releases")
},
credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")
)
tl;dr Don't use lazy val plugin = project to define a project (for unknown yet reasons)
After few comments it turned out that the issue was that the name of the project plugin as defined using lazy val plugin = project. It seems that the name is somehow reserved. Change the project's name to any other name than plugin and start over.
Specifying a project name other than "plugin" resolved the issue. I simplified the build definition a bit by removing a redundant build.sbt in 1 of the projects and am just using a full build definition in project directory. The root project that hosts the multi-project build is also reconfigured for no publishing:
lazy val root =
Project("sbt-my-plugin-root", file("."))
.settings(noPublishing: _*)
.aggregate(sbtMyPluginModule)
lazy val sbtMyPluginModule =
Project("sbt-my-plugin-module", file("sbt-my-plugin-module"))
.settings(publishSbtPlugin: _*)
.settings(
name := "sbt-my-plugin-module",
organization := "com.my.org",
sbtPlugin := true
)
lazy val noPublishing = seq(
publish := (),
publishLocal := ()
)
lazy val publishSbtPlugin = Seq(
publishMavenStyle := true,
publishArtifact in Test := false,
publishTo := {
val myrepo = "http://myrepo.tld/"
if (isSnapshot.value) Some("The Realm" at myrepo + "snapshots")
else Some("The Realm" at myrepo + "releases")
},
credentials += Credentials(Path.userHome / ".ivy2" / ".credentials")
)
if you trying this on your local then use publishLocal (not publish) as follows:
sbt clean compile publish-local
As part of my project build, I'd like to unzip a zip artifact of a managed dependency into a specific directory of the project. Before starting to use SBT I was doing this via an ANT script that would fetch the zip artifact from a maven dependency and unzip it.
My question(s) are:
how to specify that I want to depend on the zip dependency? I have defined it like so:
"eu.delving" % "sip-creator" % "0.4.6-SNAPSHOT"
but this doesn't fetch the zip artifact
where / how to hook into the build process to run the unzip (and how to refer to the artifact file in that context)?
If you want to extract a set of managed dependencies, the code below should work. I tested it in sbt 0.12.0, but it should also work with 0.11.x.
import sbt._
import Keys._
import Classpaths.managedJars
object TestBuild extends Build {
lazy val jarsToExtract = TaskKey[Seq[File]]("jars-to-extract", "JAR files to be extracted")
lazy val extractJarsTarget = SettingKey[File]("extract-jars-target", "Target directory for extracted JAR files")
lazy val extractJars = TaskKey[Unit]("extract-jars", "Extracts JAR files")
lazy val testSettings = Defaults.defaultSettings ++ Seq(
// define dependencies
libraryDependencies ++= Seq(
"com.newrelic" % "newrelic-api" % "2.2.1"
),
// collect jar files to be extracted from managed jar dependencies
jarsToExtract <<= (classpathTypes, update) map { (ct, up) =>
managedJars(Compile, ct, up) map { _.data } filter { _.getName.startsWith("newrelic-api") }
},
// define the target directory
extractJarsTarget <<= (baseDirectory)(_ / "extracted"),
// task to extract jar files
extractJars <<= (jarsToExtract, extractJarsTarget, streams) map { (jars, target, streams) =>
jars foreach { jar =>
streams.log.info("Extracting " + jar.getName + " to " + target)
IO.unzip(jar, target)
}
},
// make it run before compile
compile in Compile <<= extractJars map { _ => sbt.inc.Analysis.Empty }
)
lazy val test: Project = Project("test", file(".")) settings (testSettings: _*)
}
If you simply have jar files to extract, you can add them as unmanaged dependencies, ie. putting them into the /lib folder. See: https://github.com/harrah/xsbt/wiki/Getting-Started-Library-Dependencies
If you really have zip files (or want to extract the unmanaged dependencies), you can change the above code to list them:
// list jar files to be extracted
jarsToExtract <<= (baseDirectory) map { dir => Seq(dir / "lib" / "newrelic-api-2.2.1.zip") },
You should now be able to manually extract them from sbt and they should automatically be extracted before compile:
> clean
[success] Total time: 0 s, completed Oct 12, 2012 5:39:16 PM
> extract-jars
[info] Extracting newrelic-api-2.2.1.zip to /Users/balagez/Sites/test/extracted
[success] Total time: 0 s, completed Oct 12, 2012 5:39:22 PM
> compile
[info] Extracting newrelic-api-2.2.1.zip to /Users/balagez/Sites/test/extracted
[success] Total time: 0 s, completed Oct 12, 2012 5:39:24 PM
Now you can add a new task or extend the existing one which extracts the zip file from the extracted dependency. If you don't need the contents of the dependency, you can use the task-temporary-directory setting which gives you a temporary directory writable by sbt:
// define the target directory
extractJarsTarget <<= taskTemporaryDirectory,