Load .RData from an R script in Scala using Renjin - r

I have an R script load.R that attempts to do
load("test.RData")
When I run:
engine.eval(new FileReader("load.R"))
An exception occurs:
Exception in thread "main" java.lang.NoClassDefFoundError: org/tukaani/xz/XZInputStream
at org.renjin.primitives.io.connections.Connections.gzfile(Connections.java:79)
at org.renjin.primitives.R$primitive$gzfile.doApply(R$primitive$gzfile.java:75)
at org.renjin.primitives.R$primitive$gzfile.apply(R$primitive$gzfile.java:36)
at org.renjin.primitives.special.InternalFunction.apply(InternalFunction.java:45)
at org.renjin.eval.Context.evaluateCall(Context.java:302)
at org.renjin.eval.Context.evaluate(Context.java:222)
at org.renjin.eval.Context.evaluate(Context.java:163)
at org.renjin.sexp.Closure.doApply(Closure.java:83)
at org.renjin.eval.ClosureDispatcher.apply(ClosureDispatcher.java:73)
at org.renjin.eval.ClosureDispatcher.applyClosure(ClosureDispatcher.java:58)
at org.renjin.sexp.Closure.apply(Closure.java:78)
at org.renjin.eval.Context.evaluateCall(Context.java:302)
at org.renjin.eval.Context.evaluate(Context.java:222)
at org.renjin.primitives.special.AssignLeftFunction.assignLeft(AssignLeftFunction.java:57)
at org.renjin.primitives.special.AssignLeftFunction.apply(AssignLeftFunction.java:41)
at org.renjin.eval.Context.evaluateCall(Context.java:302)
at org.renjin.eval.Context.evaluate(Context.java:222)
at org.renjin.primitives.special.BeginFunction.apply(BeginFunction.java:38)
at org.renjin.eval.Context.evaluateCall(Context.java:302)
at org.renjin.eval.Context.evaluate(Context.java:222)
at org.renjin.primitives.special.IfFunction.apply(IfFunction.java:40)
at org.renjin.eval.Context.evaluateCall(Context.java:302)
at org.renjin.eval.Context.evaluate(Context.java:222)
at org.renjin.primitives.special.BeginFunction.apply(BeginFunction.java:38)
at org.renjin.eval.Context.evaluateCall(Context.java:302)
at org.renjin.eval.Context.evaluate(Context.java:222)
at org.renjin.eval.Context.evaluate(Context.java:163)
at org.renjin.sexp.Closure.doApply(Closure.java:83)
at org.renjin.eval.ClosureDispatcher.apply(ClosureDispatcher.java:73)
at org.renjin.eval.ClosureDispatcher.applyClosure(ClosureDispatcher.java:58)
at org.renjin.sexp.Closure.apply(Closure.java:78)
at org.renjin.eval.Context.evaluateCall(Context.java:302)
at org.renjin.eval.Context.evaluate(Context.java:222)
at org.renjin.eval.Context.evaluateExpressionVector(Context.java:285)
at org.renjin.eval.Context.evaluate(Context.java:220)
at org.renjin.script.RenjinScriptEngine.eval(RenjinScriptEngine.java:145)
at org.renjin.script.RenjinScriptEngine.eval(RenjinScriptEngine.java:104)
at com.infosistema.imp.restservice.data_treatment.DataTreatment.<init>(DataTreatment.scala:34)
at com.infosistema.imp.restservice.data_treatment.Main$.<init>(DataTreatment.scala:45)
at com.infosistema.imp.restservice.data_treatment.Main$.<clinit>(DataTreatment.scala)
at com.infosistema.imp.restservice.data_treatment.Main.main(DataTreatment.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Caused by: java.lang.ClassNotFoundException: org.tukaani.xz.XZInputStream
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 46 more
I don't know R. I'm merely attempting to use R code to process some data in a Java/Scala application using Renjin.
Reading the data section on the documention nothing is said about loading data from .RData files.
UPDATED:
In my build.sbt file I have:
resolvers += "BeDataDriven" at https://nexus.bedatadriven.com/content/groups/public",
libraryDependencies += "org.renjin" % "renjin-script-engine" % "0.8.1886"

It is not enough to just have renjin-script-engine.jar on your classpath, you need its dependencies as well.
If you are using SBT to build your Scala project, then you need to add the following to your build file:
resolvers +=
"BeDataDriven" at "https://nexus.bedatadriven.com/content/groups/public"
lazy val root = (project in file(".")).
settings(
name := "renjin-test",
version := "1.0",
scalaVersion := "2.10.6",
libraryDependencies += "org.renjin" % "renjin-script-engine" % "0.8.1886"
)
For other build tools, see:
http://docs.renjin.org/en/latest/library/project-setup.html

The problem was coursier not being able to resolve all dependencies (without any notice).
Removing addSbtPlugin("io.get-coursier" % "sbt-coursier" % "1.0.0-M15") from plugins.sbt and using sbt's default resolver fixed the issue.

Related

JavaFX Gradle Jlink-Plugin: jpackageImage fails because of "[1.0-SNAPSHOT] contains invalid component [0-SNAPSHOT]"

I try to package my app using JPackage task of Jlink-badass plugin (I'm using Gradle as a builder tool), but I keep getting the following error during the JPackageImage phase (even with a simple hello world javafx application):
> Task :jpackageImage
java.lang.IllegalArgumentException: "Version [1.0-SNAPSHOT] contains invalid component [0-SNAPSHOT]"
> Task :jpackageImage FAILED
Execution failed for task ':jpackageImage'.
> Process 'command 'C:\Users\youss\Desktop\openjdk-17_windows-x64_bin\jdk-17/bin/jpackage.exe'' finished with non-zero exit value 1
* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':jpackageImage'.
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:145)
at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:282)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:143)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:131)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:77)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:74)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:402)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:389)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:382)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:368)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:61)
Caused by: org.gradle.process.internal.ExecException: Process 'command 'C:\Users\youss\Desktop\openjdk-17_windows-x64_bin\jdk-17/bin/jpackage.exe'' finished with non-zero exit value 1
at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:414)
at org.gradle.process.ExecResult$assertNormalExitValue.call(Unknown Source)
at org.beryx.jlink.impl.JPackageImageTaskImpl.execute(JPackageImageTaskImpl.groovy:57)
at org.beryx.jlink.JPackageImageTask.jpackageTaskAction(JPackageImageTask.groovy:87)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:104)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51)
at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29)
at org.gradle.api.internal.tasks.execution.TaskExecution$2.run(TaskExecution.java:239)
at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29)
at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:224)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:207)
at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:190)
at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:168)
at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:89)
at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:40)
at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:53)
at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:50)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:50)
at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:40)
at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:68)
at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:38)
at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:48)
at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:36)
at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:41)
at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:74)
at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55)
at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:51)
at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:29)
at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:61)
at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:42)
at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:60)
at org.gradle.internal.execution.steps.BroadcastChangingOutputsStep.execute(BroadcastChangingOutputsStep.java:27)
at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:188)
at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$1(BuildCacheStep.java:75)
at org.gradle.internal.Either$Right.fold(Either.java:175)
at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:59)
at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:73)
at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:48)
at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:38)
at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:27)
at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:36)
at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:22)
at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:109)
at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:56)
at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:56)
at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:38)
at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:73)
at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:44)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27)
at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:89)
at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:50)
at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:114)
at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:57)
at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:76)
at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:50)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.lambda$execute$2(SkipEmptyWorkStep.java:93)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:93)
at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:34)
at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38)
at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:43)
at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:31)
at org.gradle.internal.execution.steps.AssignWorkspaceStep.lambda$execute$0(AssignWorkspaceStep.java:40)
at org.gradle.api.internal.tasks.execution.TaskExecution$3.withWorkspace(TaskExecution.java:284)
at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:40)
at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:30)
at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:37)
at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:27)
at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:44)
at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:33)
at org.gradle.internal.execution.impl.DefaultExecutionEngine$1.execute(DefaultExecutionEngine.java:76)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:142)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:131)
at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:77)
at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:56)
at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:74)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:402)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:389)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:382)
at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:368)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.lambda$run$0(DefaultPlanExecutor.java:127)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:191)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.executeNextNode(DefaultPlanExecutor.java:182)
at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:124)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:61)
My Gradle.Build
plugins {
id 'java'
id 'application'
id 'org.openjfx.javafxplugin' version '0.0.10'
id 'org.beryx.jlink' version '2.24.4'
}
group 'com.example'
version '1.0-SNAPSHOT'
repositories {
mavenCentral()
}
ext {
junitVersion = '5.7.1'
}
sourceCompatibility = '17'
targetCompatibility = '17'
tasks.withType(JavaCompile) {
options.encoding = 'UTF-8'
}
application {
mainModule = 'com.example.demo4'
mainClass = 'com.example.demo4.Runner'
}
javafx {
version = '17.0.1'
modules = ['javafx.controls', 'javafx.fxml']
}
dependencies {
testImplementation("org.junit.jupiter:junit-jupiter-api:${junitVersion}")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:${junitVersion}")
}
test {
useJUnitPlatform()
}
jlink {
options = ['--strip-debug', '--compress', '2', '--no-header-files', '--no-man-pages']
launcher {
name = 'Karrty'
}
}
More details
-I'm using JDK 17.
What I have tried
-I tried using different JDKs like 16 but still the same error.
not sure if this is a bug or something is it something that I doing is wrong
for whoever suffers from the same issue, just delete
version '1.0-SNAPSHOT'
in your Gradle.build, and then the tool will work as it is supposed to.

Including external jars after launching Pyspark in Jupyter notebook

I have been trying in vain to include external jars into pyspark/Jupyter notebook env after the notebook has been launched. (Alternatively, the jars are included successfully either with:
1. Use --jar ojdbc6.jar
2. include the jar in spark-defaults.conf )
I have tried below all failed:
Idea 1:
import pyspark
from pyspark.sql import SparkSession
spark = SparkSession \
.builder \
.appName("testjars") \
.config("spark.jars", "C:\\Users\\U620311\\ojdbc6.jar") \
.config("spark.driver.extraClassPath", "C:\\Users\\U620311\\ojdbc6.jar") \
.config("spark.executor.extraClassPath", "C:\\Users\\U620311\\ojdbc6.jar") \
.getOrCreate()
spark.sparkContext._conf.getAll()
Invoke jar file as below:
DB_USER = ""
DB_PASS = ""
DB_DRIVER = "oracle.jdbc.driver.OracleDriver"
JDBC_URL = "<connection string>"
query = '''(select * from <tablename>) t1'''
ojdbcDF = (sqlContext.read.format("jdbc").option("driver", DB_DRIVER).option("url", JDBC_URL).option("dbtable", query).option("user", DB_USER).option("password", DB_PASS).load())
Idea 2:
I have played with all permutations. Include extraclasspath but not spark.jars and vice versa. no success.
Idea 3:
import os
os.environ['PYSPARK_SUBMIT_ARGS'] = '--jars ojdbc6.jar pyspark-shell'
import pyspark
sc.stop()
sc = pyspark.SparkContext()
from pyspark.sql import SQLContext
sqlContext = SQLContext(sc)
DB_USER = "<uid>"
DB_PASS = "<pwd>"
DB_DRIVER = "oracle.jdbc.driver.OracleDriver"
JDBC_URL = "<connection string>"
query = '''(select * from <tablename>) t1'''
ojdbcDF = (sqlContext.read.format("jdbc").option("driver", DB_DRIVER).option("url", JDBC_URL).option("dbtable", query).option("user", DB_USER).option("password", DB_PASS).load())
All of the above lead to the same error message. Would you please help in identifying if there is missing configuration or missing code. Please suggest an alternatives to try.
Py4JJavaError: An error occurred while calling o373.load.
: java.lang.ClassNotFoundException: oracle.jdbc.driver.OracleDriver
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.spark.sql.execution.datasources.jdbc.DriverRegistry$.register(DriverRegistry.scala:38)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions$$anonfun$6.apply(JDBCOptions.scala:78)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions$$anonfun$6.apply(JDBCOptions.scala:78)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.<init>(JDBCOptions.scala:78)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions.<init>(JDBCOptions.scala:34)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:32)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:330)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:152)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:125)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:280)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:748)

How to configure SBT to skip proguard task when some system property is given.

I am using sbt, sbt-native-packager and sbt-proguard to make packages for 3 environments such as dev, staging and production environments. What I would like to do is to be able to specify whether or not to use proguard to make a package. For example, the following command make package using proguard:
$ sbt -Dforceobfuscate=true -Denv=dev rpm:pakageBin
but the following command make package without proguard
$ sbt -Denv=dev rpm:packageBin
What I tried two setting. One is overriding task by system property condition and two is overriding task with dynamic task.
1.Just overriding task by system property.
ProguardKeys.proguard in Proguard := {
if (sys.props.get("forceobfuscate").isDefined) {
( ProguardKeys.proguard in Proguard ).value
} else {
Seq(new File("dummy"))
}
}
This setting always run proguard task (I don't know why).
Also I tried the following setting for just making sure overriding task can stop task.
ProguardKeys.proguard in Proguard := {
if (sys.props.get("forceobfuscate").isDefined) {
// ( ProguardKeys.proguard in Proguard ).value
Seq(new File("dummy"))
} else {
Seq(new File("dummy"))
}
}
Indeed, above setting always skip proguard task.
2.Overriding task using dynamic task.
def obfuscateOrNot = Def.taskDyn {
if (sys.props.get("forceobfuscate").isDefined) {
Def.task {
( ProguardKeys.proguard in Proguard ).value
}
} else {
Def.task {
Seq(new File("dummy"))
}
}
}
ProguardKeys.proguard in Proguard := {
obfuscateOrNot.value
}
Then run the command:
$ sbt rpm:packageBin
Okay, this time skip proguard task. But when I run the command:
$ sbt -Dforceobfuscate=true rpm:packageBin
I got the following error:
sbt.Init$RuntimeUndefined: References to undefined settings at runtime.
sbt.Init$RuntimeUndefined: References to undefined settings at runtime.
setting(ScopedKey(Scope(Select(ProjectRef(file:/work/backend/,backend)),Select(ConfigKey(proguard)),Global,Global),proguard)) at RangePosition(/work/backend/build.sbt,LineRange(244,247)) referenced from ScopedKey(Scope(Select(ProjectRef(file:/work/backend/,backend)),Select(ConfigKey(proguard)),Global,Global),proguard)
at sbt.Init$class.sbt$Init$$handleUndefined(Settings.scala:511)
at sbt.Init$$anon$4.apply(Settings.scala:516)
at sbt.Init$$anon$4.apply(Settings.scala:516)
at sbt.$tilde$greater$$anon$6.apply(TypeFunctions.scala:39)
at sbt.$tilde$greater$$anon$6.apply(TypeFunctions.scala:39)
at sbt.std.FullInstance$$anonfun$flatten$1$$anonfun$apply$3.apply(TaskMacro.scala:51)
at sbt.std.FullInstance$$anonfun$flatten$1$$anonfun$apply$3.apply(TaskMacro.scala:51)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
[error] (proguard:proguard) sbt.Init$RuntimeUndefined: References to undefined settings at runtime.
[error] setting(ScopedKey(Scope(Select(ProjectRef(file:/work/backend/,backend)),Select(ConfigKey(proguard)),Global,Global),proguard)) at RangePosition(/work/backend/build.sbt,LineRange(244,247)) referenced from ScopedKey(Scope(Select(ProjectRef(file:/work/backend/,backend)),Select(ConfigKey(proguard)),Global,Global),proguard)
[error] Total time: 1 s, completed 2017/07/29 12:09:24
I think the error says that there is no setting for proguard but I am not sure how to set setting using dynamic task.
I would like to make above commands work well. Any solution is fine.
I found a solution. The solution is making proguard Seq[Setting[_]] value depending on condition.
lazy val customProguardSettings =
if (sys.props.get("forceobfuscate").isDefined) {
proguardSettings ++ Seq( ... )
} else {
Nil
}
customProguardSettings

How to use Spark 1.2.0 in Play 2.2.3 project as it fails with NoSuchMethodError: akka.util.Helpers?

Have you ever had a problem with Play framework? In my case, first of all I have build all in one jar: spark-assebmly-1.2.0-hadoop2.4.0.jar, and Spark works perfectly from a shell. But there are two questions:
Should I use this assebmled Spark_jar in Play_project and how?? Because I try to move it into the lib_directiry and it did n`t help to provide any Spark_imports.
If I'm defining Spark library like: "org.apache.spark" %% "spark-core" % "1.2.0"
PLAY FRAMEWORK CODE:
Build.scala :
val appDependencies = Seq(
jdbc
,"org.apache.spark" %% "spark-streaming" % "1.2.0"
,"org.apache.spark" %% "spark-core" % "1.2.0"
,"org.apache.spark" %% "spark-sql" % "1.2.0"
TestEntity.scala :
package models
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.hadoop.conf.Configuration
import models.SparkMain
import org.apache.spark.rdd.RDD
import org.apache.spark.SparkContext._
object TestEntity {
val TestEntityPath = "/home/t/PROD/dict/TestEntity .txt"
val TestEntitySpark= SparkMain.sc.textFile(TestEntityPath, 4).cache
val TestEntityData = TestEntitySpark.flatMap(_.split(","))
def getFive() : Seq[String] = {
println("TestEntity.getFive")
TestEntityData.take(5)
}
}
SparkMain.scala :
package models
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.hadoop.conf.Configuration
import org.apache.spark.rdd.RDD
import org.apache.spark.SparkContext._
import org.apache.spark.streaming.{ Seconds, StreamingContext }
import StreamingContext._
import org.apache.spark.sql.SQLContext
import org.apache.spark.SparkConf
object SparkMain {
val driverPort = 8080
val driverHost = "localhost"
val conf = new SparkConf(false) // skip loading external settings
.setMaster("local[4]") // run locally with enough threads
.setAppName("firstSparkApp")
.set("spark.logConf", "true")
.set("spark.driver.port", s"$driverPort")
.set("spark.driver.host", s"$driverHost")
.set("spark.akka.logLifecycleEvents", "true")
val sc = new SparkContext(conf)
}
and controller code, which use Spark stuff :
def test = Action {
implicit req => {
val chk = TestEntity.getFive
Ok("it works")
}
}
..in runtime a have this errors:
[info] o.a.s.SparkContext - Spark configuration:
spark.akka.logLifecycleEvents=true
spark.app.name=firstSparkApp
spark.driver.host=localhost
spark.driver.port=8080
spark.logConf=true
spark.master=local[4]
[warn] o.a.s.u.Utils - Your hostname, uisprk resolves to a loopback address: 127.0.1.1; using 10.0.2.15 instead (on interface eth0)
[warn] o.a.s.u.Utils - Set SPARK_LOCAL_IP if you need to bind to another address
[info] o.a.s.SecurityManager - Changing view acls to: t
[info] o.a.s.SecurityManager - Changing modify acls to: t
[info] o.a.s.SecurityManager - SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(t); users with modify permissions: Set(t)
[error] application -
! #6l039e8d5 - Internal server error, for (GET) [/ui] ->
play.api.Application$$anon$1: Execution exception[[RuntimeException: java.lang.NoSuchMethodError: akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/Config;)Lcom/typesafe/config/Config;]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10-2.2.3.jar:2.2.3]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10-2.2.3.jar:2.2.3]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$13$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:166) [play_2.10-2.2.3.jar:2.2.3]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$13$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:163) [play_2.10-2.2.3.jar:2.2.3]
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33) [scala-library-2.10.4.jar:na]
at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:185) [scala-library-2.10.4.jar:na]
Caused by: java.lang.RuntimeException: java.lang.NoSuchMethodError: akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/Config;)Lcom/typesafe/config/Config;
at play.api.mvc.ActionBuilder$$anon$1.apply(Action.scala:314) ~[play_2.10-2.2.3.jar:2.2.3]
at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:109) ~[play_2.10-2.2.3.jar:2.2.3]
at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4$$anonfun$apply$5.apply(Action.scala:109) ~[play_2.10-2.2.3.jar:2.2.3]
at play.utils.Threads$.withContextClassLoader(Threads.scala:18) ~[play_2.10-2.2.3.jar:2.2.3]
at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:108) ~[play_2.10-2.2.3.jar:2.2.3]
at play.api.mvc.Action$$anonfun$apply$1$$anonfun$apply$4.apply(Action.scala:107) ~[play_2.10-2.2.3.jar:2.2.3]
Caused by: java.lang.NoSuchMethodError: akka.util.Helpers$.ConfigOps(Lcom/typesafe/config/Config;)Lcom/typesafe/config/Config;
at akka.remote.RemoteSettings.<init>(RemoteSettings.scala:48) ~[akka-remote_2.10-2.3.4-spark.jar:na]
at akka.remote.RemoteActorRefProvider.<init>(RemoteActorRefProvider.scala:114) ~[akka-remote_2.10-2.3.4-spark.jar:na]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.7.0_72]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) ~[na:1.7.0_72]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.7.0_72]
at java.lang.reflect.Constructor.newInstance(Constructor.java:526) ~[na:1.7.0_72]
How to tie the library? through dependency or assembled_jar?
Any advice, please.
nosuchmethodeersrror exception is 100 % due to mismatch of jars version at compile time and runtime.
check the versions of jar. Also I have some questions about architecture of your app
Instead of calling spark code from play framework you can also call spark submit from shell scripts which looks better in your case. Even you can do it from your play application. no need to include jar in play app classpath.
The problem with the configuration is Akka dependency of Apache Spark and Play Framework -- they both depend on Akka and, as you've faced it, different and incompatible versions should be resolved at build time with evicted command in sbt.
You may want to use update command and find the reports in target/resolution-cache/reports quite useful.

Gradle and Android and Robolectric - Empty test classpath

I am migrating my project to Gradle and I had a separate working Robolectric setup along with my App.
The Robolectric tests are under the test/ folder and they should include the app jar file (thanks to another SO question) and other dependecies at runtime.
The tests are launched by Gradle, but the classpath looks empty.
The gradle.build:
apply plugin: 'java'
test {
// The classpath.each prints:
// /test-robolectric/build/classes/test
// /test-robolectric/build/resources/test
// /test-robolectric/build/classes/main
// /test-robolectric/build/resources/main
classpath.each { file ->
println file.absolutePath
}
reports {
html.enabled = true
}
// show standard out and standard error of the test JVM(s) on the console
testLogging.showStandardStreams = false
// listen to events in the test execution lifecycle
beforeTest { descriptor ->
logger.lifecycle("Running test: " + descriptor)
}
}
dependencies {
compile project(':test-common:test-common-jar')
compile project(path: ':app', configuration: 'testJar')
compile 'org.robolectric:robolectric:2.3-SNAPSHOT'
compile 'com.google.android:android:4.+'
compile 'com.android.support:appcompat-v7:19+#jar'
compile 'com.android.support:support-v4:19.+'
compile 'com.google.inject:guice:4.0-beta:no_aop'
compile 'com.google.code.gson:gson:2.2.4'
compile 'org.functionaljava:functionaljava:3.1'
compile 'javax.inject:javax.inject:1#jar'
compile 'com.squareup:otto:1.3.4#jar'
compile 'net.jcip:jcip-annotations:1.0'
compile 'de.akquinet.android.androlog:androlog:1.0.5'
compile 'com.j256.ormlite:ormlite-android:4.48'
compile 'com.j256.ormlite:ormlite-core:4.48'
compile 'junit:junit:4.11'
compile 'org.mockito:mockito-all:1.9.+'
compile 'org.hamcrest:hamcrest-core:1.+'
compile 'com.squareup:fest-android:1.0.+'
}
repositories {
mavenLocal()
mavenCentral()
maven {
url 'http://repository.grepcode.com/java/ext'
}
}
I have also tried to put the dependecies declaration inside the test{} section to no avail. What am I missing here?
The specific exception:
java.lang.NoClassDefFoundError: android/content/Context
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:58)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:49)
at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:69)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.messaging.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.messaging.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at org.gradle.messaging.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
[...]

Resources