How to get the list of subprojects dynamically in sbt 0.13 - sbt

How can I programmatically (in build.sbt) find all the subprojects of the current root project in sbt 0.13?
(I have not tried Project.componentProjects yet, because it's new in sbt 1.0).
lazy val root = (project in file(".") ... )
val myTask = taskKey[Unit]("some description")
myTask := {
val masterRoot = baseDirectory.value
// This does not work
// val subProjects: Seq[ProjectReference] = root.aggregate
// So I tried to specify the subproject list explicitly; still does not work
val subProjects = Seq[Project](subPrj1)
subProjects.foreach { subproject =>
// All of this works if the "subproject" is hard-coded to "subPrj1"
val subprojectTarget = target.in(subproject).value / "classes"
val cp = (dependencyClasspath in(subproject, Compile, compile)).value
}
}
Got these errors:
build.sbt: error: Illegal dynamic reference: subproject
val subprojectTarget = target.in(subproject).value / "classes"
^
build.sbt: error: Illegal dynamic reference: subproject
val cp = (dependencyClasspath in(subproject, Compile, compile)).value

You can access a list of all subprojects via buildStructure.value.allProjectRefs.
The other part is of your problem is an aweful issue that I've also faced quite often. I was able to work around such problems by first creating a List[Task[A] and then using a recursive function to lift it into a Task[List[A]].
def flattenTasks[A](tasks: Seq[Def.Initialize[Task[A]]]): Def.Initialize[Task[List[A]]] =
tasks.toList match {
case Nil => Def.task { Nil }
case x :: xs => Def.taskDyn {
flattenTasks(xs) map (x.value :: _)
}
}
myTask := {
val classDirectories: List[File] = Def.taskDyn {
flattenTasks {
for (project ← buildStructure.value.allProjectRefs)
yield Def.task { (target in project).value / "classes" }
}
}.value
}
I've used this approach e.g. here: utility methods actual usage

Related

SBT clone git dependencies to a custom path using a plugin

I'm creating an aggregate SBT project which depends on several other Git projects. I understand that I can refer to them as a dependency using RootProject(uri("...")) and SBT clones them into an SBT-managed path.
However, I need to download these into a custom path. The idea is to create a workspace that automatically downloads the related Git projects that can be worked on as well.
I was able to create a plugin with a task that clones the git repos using sbt-git plugin:
BundleResolver.scala
def resolve: Def.Initialize[Task[Seq[String]]] = Def.task {
val log = streams.value.log
log.info("starting bundle resolution")
val bundles = WorkspacePlugin.autoImport.workspaceBundles.value
val bundlePaths = bundles.map(x => {
val bundleName = extractBundleName(x)
val localPath = file(".").toPath.toAbsolutePath.getParent.resolveSibling(bundleName)
log.info(s"Cloning bundle : $bundleName")
val (resultCode, outStr, errStr) = runCommand(Seq("git", "clone", x, localPath.toString))
resultCode match {
case 0 =>
log.info(outStr)
log.info(s"cloned $bundleName to path $localPath")
case _ =>
log.err(s"failed to clone $bundleName")
log.err(errStr)
}
localPath.toString
})
bundlePaths
}
WorkspacePlugin.scala
object WorkspacePlugin extends AutoPlugin {
override def trigger = allRequirements
override def requires: Plugins = JvmPlugin && GitPlugin
object autoImport {
// settings
val workspaceBundles = settingKey[Seq[String]]("Dependency bundles for this Workspace")
val stagingPath = settingKey[File]("Staging path")
// tasks
val workspaceClean = taskKey[Unit]("Remove existing Workspace depedencies")
val workspaceImport = taskKey[Seq[String]]("Download the dependency bundles and setup builds")
}
import autoImport._
override lazy val projectSettings = Seq(
workspaceBundles := Seq(), // default to no dependencies
stagingPath := Keys.target.value,
workspaceClean := BundleResolver.clean.value,
workspaceImport := BundleResolver.resolve.value,
)
override lazy val buildSettings = Seq()
override lazy val globalSettings = Seq()
}
However, this will not add the cloned repos as sub projects to the main project. How can I achieve this?
UPDATE:: I had an idea to extend RootProject logic, so that I can create custom projects that would accept a git url, clone it in a custom path, and return a Project from it.
object WorkspaceProject {
def apply(uri: URI): Project = {
val bundleName = GitResolver.extractBundleName(uri.toString)
val localPath = file(".").toPath.toAbsolutePath.getParent.resolveSibling(bundleName)
// clone the project
GitResolver.clone(uri, localPath)
Project.apply(bundleName.replaceAll(".", "-"), localPath.toFile)
}
}
I declared this in a plugin project, but can't access it where I'm using it. Do you think it'll work? How can I access it in my target project?
Can't believe it was this simple.
In my plugin project, I created a new object to use in place of RootProject
object WorkspaceProject {
def apply(uri: URI): RootProject = {
val bundleName = GitResolver.extractBundleName(uri.toString)
val localPath = file(".").toPath.toAbsolutePath.getParent.resolve(bundleName)
if(!localPath.toFile.exists()) {
// clone the project
GitResolver.clone(uri, localPath)
}
RootProject(file(localPath.toString))
}
}
Then use it like this:
build.sbt
lazy val depProject = WorkspaceProject(uri("your-git-repo.git"))
lazy val root = (project in file("."))
.settings(
name := "workspace_1",
).dependsOn(depProject)

Extract setting to top level val

Given:
resourceGenerators in Compile += Def.task {
val jar = (update in Compile).value
.matching((_: ModuleID) == nemesisProto)
.head
IO.unzip(jar, (resourceManaged in Compile).value / "protobuf").toSeq
}.taskValue
PB.protoSources in Compile := Seq((resourceManaged in Compile).value / "protobuf")
Is is possible to refactor (resourceManaged in Compile).value / "protobuf" to a common place?
I tried assigning it to a val:
val protobufResourceFile = (resourceManaged in Compile).value / "protobuf"
resourceGenerators in Compile += Def.task {
val jar = (update in Compile).value
.matching((_: ModuleID) == nemesisProto)
.head
IO.unzip(jar, protobufResourceFile).toSeq
}.taskValue
PB.protoSources in Compile := Seq(protobufResourceFile)
Only to get the following error:
error: `value` can only be used within a task or setting macro, such as :=, +=, ++=, Def.task, or Def.setting.
val protobufResourceFolder = (resourceManaged in Compile).value / "protobuf"
^
Almost. As the error message states you simply cannot unwrap a value outside the sbt dsl. So, something like this is usually done using a SettingKey:
val protobufResourceFile = settingKey[File]("Protobuf resource file ...")
protobufResourceFile := (resourceManaged in Compile).value / "protobuf"
resourceGenerators in Compile += Def.task {
val jar = (update in Compile).value
.matching((_: ModuleID) == nemesisProto)
.head
IO.unzip(jar, protobufResourceFile.value).toSeq
}.taskValue
PB.protoSources in Compile := Seq(protobufResourceFile.value)
Though, in this specific case it may be overkill.

Finishing a forked process blocks SBT with a custom output strategy

In SBT, I fork a Java process with:
class FilteredOutput extends FilterOutputStream(System.out) {
var buf = ArrayBuffer[Byte]()
override def write(b: Int) {
buf.append(b.toByte)
if (b == '\n'.toInt)
flush()
}
override def flush(){
if (buf.nonEmpty) {
val arr = buf.toArray
val txt = try new String(arr, "UTF-8") catch { case NonFatal(ex) ⇒ "" }
if (!txt.startsWith("pydev debugger: Unable to find real location for"))
out.write(arr)
buf.clear()
}
super.flush()
}
}
var process = Option.empty[Process]
process = Some(Fork.java.fork(ForkOptions(outputStrategy = new FilteredOutput()), Seq("my.company.MyClass")))
as a result of a custom task.
Later on, I terminate it with:
process.map { p =>
log info "Killing process"
p.destroy()
}
by means of another custom task.
The result is that SBT doesn't accept more input and gets blocked. Ctrl+C is the only way of restoring control back, but SBT dies as a consequence.
The problem has to do with the custom output strategy, that filters some annoying messages.
With jstack I haven't seen any deadlock.
SBT version 0.13.9.
The solution is to avoid closing System.out:
class FilteredOutput extends FilterOutputStream(System.out) {
var buf = ArrayBuffer[Byte]()
override def write(b: Int) {
...
}
override def flush(){
...
}
override def close() {}
}

How to get Kotlin AST?

I have a string with Kotlin source in it. How can I compile it at run-time and get abstract syntax tree and types info to analyze?
I have some investigation of Kotlin compiler. Some proof of concept to getting of AST can be seen on my GitHub repo.
It's a sketch only, but can be helpful:
class KotlinScriptParser {
companion object {
private val LOG = Logger.getLogger(KotlinScriptParser.javaClass.name)
private val messageCollector = object : MessageCollector {
override fun report(severity: CompilerMessageSeverity, message: String, location: CompilerMessageLocation) {
val path = location.path
val position = if (path == null) "" else "$path: (${location.line}, ${location.column}) "
val text = position + message
if (CompilerMessageSeverity.VERBOSE.contains(severity)) {
LOG.finest(text)
} else if (CompilerMessageSeverity.ERRORS.contains(severity)) {
LOG.severe(text)
} else if (severity == CompilerMessageSeverity.INFO) {
LOG.info(text)
} else {
LOG.warning(text)
}
}
}
private val classPath: ArrayList<File> by lazy {
val classpath = arrayListOf<File>()
classpath += PathUtil.getResourcePathForClass(AnnotationTarget.CLASS.javaClass)
classpath
}
}
fun parse(vararg files: String): TopDownAnalysisContext {
// The Kotlin compiler configuration
val configuration = CompilerConfiguration()
val groupingCollector = GroupingMessageCollector(messageCollector)
val severityCollector = MessageSeverityCollector(groupingCollector)
configuration.put(CLIConfigurationKeys.MESSAGE_COLLECTOR_KEY, severityCollector)
configuration.addJvmClasspathRoots(PathUtil.getJdkClassesRoots())
// The path to .kt files sources
files.forEach { configuration.addKotlinSourceRoot(it) }
// Configuring Kotlin class path
configuration.addJvmClasspathRoots(classPath)
configuration.put(JVMConfigurationKeys.MODULE_NAME, JvmAbi.DEFAULT_MODULE_NAME)
configuration.put<List<AnalyzerScriptParameter>>(JVMConfigurationKeys.SCRIPT_PARAMETERS, CommandLineScriptUtils.scriptParameters())
val rootDisposable = Disposer.newDisposable()
try {
val environment = KotlinCoreEnvironment.createForProduction(rootDisposable, configuration, EnvironmentConfigFiles.JVM_CONFIG_FILES)
val ktFiles = environment.getSourceFiles()
val sharedTrace = CliLightClassGenerationSupport.NoScopeRecordCliBindingTrace()
val moduleContext = TopDownAnalyzerFacadeForJVM.createContextWithSealedModule(environment.project,
environment.getModuleName())
val project = moduleContext.project
val allFiles = JvmAnalyzerFacade.getAllFilesToAnalyze(project, null, ktFiles)
val providerFactory = FileBasedDeclarationProviderFactory(moduleContext.storageManager, allFiles)
val lookupTracker = LookupTracker.DO_NOTHING
val packagePartProvider = JvmPackagePartProvider(environment)
val container = createContainerForTopDownAnalyzerForJvm(
moduleContext,
sharedTrace,
providerFactory,
GlobalSearchScope.allScope(project),
lookupTracker,
packagePartProvider)
val additionalProviders = ArrayList<PackageFragmentProvider>()
additionalProviders.add(container.javaDescriptorResolver.packageFragmentProvider)
return container.lazyTopDownAnalyzerForTopLevel.analyzeFiles(TopDownAnalysisMode.LocalDeclarations, allFiles, additionalProviders)
} finally {
rootDisposable.dispose()
if (severityCollector.anyReported(CompilerMessageSeverity.ERROR)) {
throw RuntimeException("Compilation error")
}
}
}
}
fun main(args: Array<String>) {
val scriptFile = "/media/data/java/blackfern/kotlin-compile-test/test.kt"
val parser = KotlinScriptParser()
// Getting a root element of the AST
val analyzeContext = parser.parse(scriptFile)
// Sample AST investigation
val function = analyzeContext.functions.keys.first()
val body = function.bodyExpression as KtBlockExpression
}
There's no standard API to do this at the moment. You can play with the Kotlin compiler and REPL source code to try to achieve this.

Scala: How do I dynamically instantiate an object and invoke a method using reflection?

In Scala, what's the best way to dynamically instantiate an object and invoke a method using reflection?
I would like to do Scala-equivalent of the following Java code:
Class class = Class.forName("Foo");
Object foo = class.newInstance();
Method method = class.getMethod("hello", null);
method.invoke(foo, null);
In the above code, both the class name and the method name are passed in dynamically. The above Java mechanism could probably be used for Foo and hello(), but the Scala types don't match one-to-one with that of Java. For example, a class may be declared implicitly for a singleton object. Also Scala method allows all sorts of symbols to be its name. Both are resolved by name mangling. See Interop Between Java and Scala.
Another issue seems to be the matching of parameters by resolving overloads and autoboxing, described in Reflection from Scala - Heaven and Hell.
There is an easier way to invoke method reflectively without resorting to calling Java reflection methods: use Structural Typing.
Just cast the object reference to a Structural Type which has the necessary method signature then call the method: no reflection necessary (of course, Scala is doing reflection underneath but we don't need to do it).
class Foo {
def hello(name: String): String = "Hello there, %s".format(name)
}
object FooMain {
def main(args: Array[String]) {
val foo = Class.forName("Foo").newInstance.asInstanceOf[{ def hello(name: String): String }]
println(foo.hello("Walter")) // prints "Hello there, Walter"
}
}
The answers by VonC and Walter Chang are quite good, so I'll just complement with one Scala 2.8 Experimental feature. In fact, I won't even bother to dress it up, I'll just copy the scaladoc.
object Invocation
extends AnyRef
A more convenient syntax for reflective
invocation. Example usage:
class Obj { private def foo(x: Int, y: String): Long = x + y.length }
You can call it reflectively one of
two ways:
import scala.reflect.Invocation._
(new Obj) o 'foo(5, "abc") // the 'o' method returns Any
val x: Long = (new Obj) oo 'foo(5, "abc") // the 'oo' method casts to expected type.
If you call the oo
method and do not give the type
inferencer enough help, it will most
likely infer Nothing, which will
result in a ClassCastException.
Author Paul Phillips
The instanciation part could use the Manifest: see this SO answer
experimental feature in Scala called manifests which are a way to get around a Java constraint regarding type erasure
class Test[T](implicit m : Manifest[T]) {
val testVal = m.erasure.newInstance().asInstanceOf[T]
}
With this version you still write
class Foo
val t = new Test[Foo]
However, if there's no no-arg constructor available you get a runtime exception instead of a static type error
scala> new Test[Set[String]]
java.lang.InstantiationException: scala.collection.immutable.Set
at java.lang.Class.newInstance0(Class.java:340)
So the true type safe solution would be using a Factory.
Note: as stated in this thread, Manifest is here to stay, but is for now "only use is to give access to the erasure of the type as a Class instance."
The only thing manifests give you now is the erasure of the static type of a parameter at the call site (contrary to getClass which give you the erasure of the dynamic type).
You can then get a method through reflection:
classOf[ClassName].getMethod("main", classOf[Array[String]])
and invoke it
scala> class A {
| def foo_=(foo: Boolean) = "bar"
| }
defined class A
scala>val a = new A
a: A = A#1f854bd
scala>a.getClass.getMethod(decode("foo_="),
classOf[Boolean]).invoke(a, java.lang.Boolean.TRUE)
res15: java.lang.Object = bar
In case you need to invoke a method of a Scala 2.10 object (not class) and you have the names of the method and object as Strings, you can do it like this:
package com.example.mytest
import scala.reflect.runtime.universe
class MyTest
object MyTest {
def target(i: Int) = println(i)
def invoker(objectName: String, methodName: String, arg: Any) = {
val runtimeMirror = universe.runtimeMirror(getClass.getClassLoader)
val moduleSymbol = runtimeMirror.moduleSymbol(
Class.forName(objectName))
val targetMethod = moduleSymbol.typeSignature
.members
.filter(x => x.isMethod && x.name.toString == methodName)
.head
.asMethod
runtimeMirror.reflect(runtimeMirror.reflectModule(moduleSymbol).instance)
.reflectMethod(targetMethod)(arg)
}
def main(args: Array[String]): Unit = {
invoker("com.example.mytest.MyTest$", "target", 5)
}
}
This prints 5 to standard output.
Further details in Scala Documentation.
Working up from #nedim's answer, here is a basis for a full answer,
main difference being here below we instantiate naive classes. This code does not handle the case of multiple constructors, and is by no means a full answer.
import scala.reflect.runtime.universe
case class Case(foo: Int) {
println("Case Case Instantiated")
}
class Class {
println("Class Instantiated")
}
object Inst {
def apply(className: String, arg: Any) = {
val runtimeMirror: universe.Mirror = universe.runtimeMirror(getClass.getClassLoader)
val classSymbol: universe.ClassSymbol = runtimeMirror.classSymbol(Class.forName(className))
val classMirror: universe.ClassMirror = runtimeMirror.reflectClass(classSymbol)
if (classSymbol.companion.toString() == "<none>") // TODO: use nicer method "hiding" in the api?
{
println(s"Info: $className has no companion object")
val constructors = classSymbol.typeSignature.members.filter(_.isConstructor).toList
if (constructors.length > 1) {
println(s"Info: $className has several constructors")
}
else {
val constructorMirror = classMirror.reflectConstructor(constructors.head.asMethod) // we can reuse it
constructorMirror()
}
}
else
{
val companionSymbol = classSymbol.companion
println(s"Info: $className has companion object $companionSymbol")
// TBD
}
}
}
object app extends App {
val c = Inst("Class", "")
val cc = Inst("Case", "")
}
Here is a build.sbt that would compile it:
lazy val reflection = (project in file("."))
.settings(
scalaVersion := "2.11.7",
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-compiler" % scalaVersion.value % "provided",
"org.scala-lang" % "scala-library" % scalaVersion.value % "provided"
)
)

Resources