How could I convert a pom xml to sbt dependencies? - sbt

I have some projects that have all the dependencies stored inside pom.xml files.
How could I retrieve the dependencies from inside so I could easily place them to a project that uses sbt?
Copy pasting all of them is just time consuming..

This scala script, which runs from command line, takes care of that, converting the pom.xml file to sbt dependencies printed on screen. Then you only need to copy paste once for each pom.xml file.
Note: the pom.xml must be in the same folder as the script. Then from command line you execute: scala scriptname.scala
import scala.xml._
(XML.load("pom.xml") \\ "dependencies") \ "dependency" foreach ((dependency: Node) => {
val groupId = (dependency \ "groupId").text
val artifactId = (dependency \ "artifactId").text
val version = (dependency \ "version").text
val scope = (dependency \ "scope").text
val classifier = (dependency \ "classifier").text
val artifactValName: String = artifactId.replaceAll("[-\\.]", "_")
print("val %s = \"%s\" %% \"%s\" %% \"%s\"".format(artifactValName, groupId, artifactId, version))
scope match {
case "" => print("\n")
case _ => print(" %% \"%s\"\n".format(scope))
}
None
});

I've enhanced George Pligor's answer (and fixed a few bugs) so a complete build.sbt file is created that incorporates the dependencies from pom.xml. To convert a Maven pom.xml to build.sbt:
Place this code into a file called PomToSbt.scala next to pom.xml
Type scala PomToSbt.scala > build.sbt
The dependencies from pom.xml will be extracted and placed into a complete build.sbt file.
Here is the code:
import scala.xml._
val lines = (XML.load("pom.xml") \\ "dependencies") \ "dependency" map { dependency =>
val groupId = (dependency \ "groupId").text
val artifactId = (dependency \ "artifactId").text
val version = (dependency \ "version").text
val scope = (dependency \ "scope").text
val classifier = (dependency \ "classifier").text
val artifactValName: String = artifactId.replaceAll("[-\\.]", "_")
val scope2 = scope match {
case "" => ""
case _ => s""" % "$scope""""
}
s""" "$groupId" %% "$artifactId" % "$version"$scope2"""
}
val buildSbt = io.Source.fromURL("https://raw.githubusercontent.com/mslinn/sbtTemplate/master/build.sbt").mkString
val libText = "libraryDependencies ++= Seq("
val buildSbt2 = buildSbt.replace(libText, libText + lines.mkString("\n", ",\n", ""))
println(buildSbt2)
I made a gist; if any updates are required I will make them there.

Mike, here's the code that works with Scala 11 at least:
import scala.xml._
//build.sbt file
val lines = (XML.load("pom.xml") \\ "dependencies") \ "dependency" map { dependency =>
val groupId = (dependency \ "groupId").text
val artifactId = (dependency \ "artifactId").text
val version = (dependency \ "version").text
val scope = (dependency \ "scope").text
val classifier = (dependency \ "classifier").text
val artifactValName: String = artifactId.replaceAll("[-\\.]", "_")
val scope2 = scope match {
case "" => ""
case _ => s""" % "$scope""""
}
s""" "$groupId" %% "$artifactId" % "$version"$scope2"""
}
val buildSbt: String = io.Source.fromURL("https://raw.githubusercontent.com/mslinn/sbtTemplate/master/build.sbt").mkString
val libText = "libraryDependencies ++= Seq\\("
val buildSbt2 = buildSbt.replaceFirst(libText, libText + lines.mkString("\n", ",\n", ""))
println(buildSbt2)

This blog entry explains one possible way. There's a comment pointing to a plugin that handles more complex cases.

import scala.xml._
// To convert a Maven pom.xml to build.sbt:
// 1) Place this code into a file called PomToSbt.scala next to pom.xml
// 2) Type scala PomtoSbt.scala > build.sbt
// The dependencies from pom.xml will be extracted and place into a complete build.sbt file
// Because most pom.xml files only refernence non-Scala dependencies, I did not use %%
val lines = (XML.load("pom.xml") \\ "dependencies") \ "dependency" map { dependency =>
val groupId = (dependency \ "groupId").text
val artifactId = (dependency \ "artifactId").text
val version = (dependency \ "version").text
val scope = (dependency \ "scope").text
val classifier = (dependency \ "classifier").text
val artifactValName: String = artifactId.replaceAll("[-\\.]", "_")
val scope2 = scope match {
case "" => ""
case _ => s""" % "$scope""""
}
s""" "$groupId" % "$artifactId" % "$version"$scope2"""
}
val buildSbt = io.Source.fromURL("https://raw.githubusercontent.com/mslinn/sbtTemplate/master/build.sbt").mkString
val libText = "libraryDependencies ++= Seq("
val buildSbt2 = buildSbt.replace(libText, libText + lines.mkString("\n", ",\n", ""))
println(buildSbt2)

Related

SBT clone git dependencies to a custom path using a plugin

I'm creating an aggregate SBT project which depends on several other Git projects. I understand that I can refer to them as a dependency using RootProject(uri("...")) and SBT clones them into an SBT-managed path.
However, I need to download these into a custom path. The idea is to create a workspace that automatically downloads the related Git projects that can be worked on as well.
I was able to create a plugin with a task that clones the git repos using sbt-git plugin:
BundleResolver.scala
def resolve: Def.Initialize[Task[Seq[String]]] = Def.task {
val log = streams.value.log
log.info("starting bundle resolution")
val bundles = WorkspacePlugin.autoImport.workspaceBundles.value
val bundlePaths = bundles.map(x => {
val bundleName = extractBundleName(x)
val localPath = file(".").toPath.toAbsolutePath.getParent.resolveSibling(bundleName)
log.info(s"Cloning bundle : $bundleName")
val (resultCode, outStr, errStr) = runCommand(Seq("git", "clone", x, localPath.toString))
resultCode match {
case 0 =>
log.info(outStr)
log.info(s"cloned $bundleName to path $localPath")
case _ =>
log.err(s"failed to clone $bundleName")
log.err(errStr)
}
localPath.toString
})
bundlePaths
}
WorkspacePlugin.scala
object WorkspacePlugin extends AutoPlugin {
override def trigger = allRequirements
override def requires: Plugins = JvmPlugin && GitPlugin
object autoImport {
// settings
val workspaceBundles = settingKey[Seq[String]]("Dependency bundles for this Workspace")
val stagingPath = settingKey[File]("Staging path")
// tasks
val workspaceClean = taskKey[Unit]("Remove existing Workspace depedencies")
val workspaceImport = taskKey[Seq[String]]("Download the dependency bundles and setup builds")
}
import autoImport._
override lazy val projectSettings = Seq(
workspaceBundles := Seq(), // default to no dependencies
stagingPath := Keys.target.value,
workspaceClean := BundleResolver.clean.value,
workspaceImport := BundleResolver.resolve.value,
)
override lazy val buildSettings = Seq()
override lazy val globalSettings = Seq()
}
However, this will not add the cloned repos as sub projects to the main project. How can I achieve this?
UPDATE:: I had an idea to extend RootProject logic, so that I can create custom projects that would accept a git url, clone it in a custom path, and return a Project from it.
object WorkspaceProject {
def apply(uri: URI): Project = {
val bundleName = GitResolver.extractBundleName(uri.toString)
val localPath = file(".").toPath.toAbsolutePath.getParent.resolveSibling(bundleName)
// clone the project
GitResolver.clone(uri, localPath)
Project.apply(bundleName.replaceAll(".", "-"), localPath.toFile)
}
}
I declared this in a plugin project, but can't access it where I'm using it. Do you think it'll work? How can I access it in my target project?
Can't believe it was this simple.
In my plugin project, I created a new object to use in place of RootProject
object WorkspaceProject {
def apply(uri: URI): RootProject = {
val bundleName = GitResolver.extractBundleName(uri.toString)
val localPath = file(".").toPath.toAbsolutePath.getParent.resolve(bundleName)
if(!localPath.toFile.exists()) {
// clone the project
GitResolver.clone(uri, localPath)
}
RootProject(file(localPath.toString))
}
}
Then use it like this:
build.sbt
lazy val depProject = WorkspaceProject(uri("your-git-repo.git"))
lazy val root = (project in file("."))
.settings(
name := "workspace_1",
).dependsOn(depProject)

How to get the list of subprojects dynamically in sbt 0.13

How can I programmatically (in build.sbt) find all the subprojects of the current root project in sbt 0.13?
(I have not tried Project.componentProjects yet, because it's new in sbt 1.0).
lazy val root = (project in file(".") ... )
val myTask = taskKey[Unit]("some description")
myTask := {
val masterRoot = baseDirectory.value
// This does not work
// val subProjects: Seq[ProjectReference] = root.aggregate
// So I tried to specify the subproject list explicitly; still does not work
val subProjects = Seq[Project](subPrj1)
subProjects.foreach { subproject =>
// All of this works if the "subproject" is hard-coded to "subPrj1"
val subprojectTarget = target.in(subproject).value / "classes"
val cp = (dependencyClasspath in(subproject, Compile, compile)).value
}
}
Got these errors:
build.sbt: error: Illegal dynamic reference: subproject
val subprojectTarget = target.in(subproject).value / "classes"
^
build.sbt: error: Illegal dynamic reference: subproject
val cp = (dependencyClasspath in(subproject, Compile, compile)).value
You can access a list of all subprojects via buildStructure.value.allProjectRefs.
The other part is of your problem is an aweful issue that I've also faced quite often. I was able to work around such problems by first creating a List[Task[A] and then using a recursive function to lift it into a Task[List[A]].
def flattenTasks[A](tasks: Seq[Def.Initialize[Task[A]]]): Def.Initialize[Task[List[A]]] =
tasks.toList match {
case Nil => Def.task { Nil }
case x :: xs => Def.taskDyn {
flattenTasks(xs) map (x.value :: _)
}
}
myTask := {
val classDirectories: List[File] = Def.taskDyn {
flattenTasks {
for (project ← buildStructure.value.allProjectRefs)
yield Def.task { (target in project).value / "classes" }
}
}.value
}
I've used this approach e.g. here: utility methods actual usage

Extract setting to top level val

Given:
resourceGenerators in Compile += Def.task {
val jar = (update in Compile).value
.matching((_: ModuleID) == nemesisProto)
.head
IO.unzip(jar, (resourceManaged in Compile).value / "protobuf").toSeq
}.taskValue
PB.protoSources in Compile := Seq((resourceManaged in Compile).value / "protobuf")
Is is possible to refactor (resourceManaged in Compile).value / "protobuf" to a common place?
I tried assigning it to a val:
val protobufResourceFile = (resourceManaged in Compile).value / "protobuf"
resourceGenerators in Compile += Def.task {
val jar = (update in Compile).value
.matching((_: ModuleID) == nemesisProto)
.head
IO.unzip(jar, protobufResourceFile).toSeq
}.taskValue
PB.protoSources in Compile := Seq(protobufResourceFile)
Only to get the following error:
error: `value` can only be used within a task or setting macro, such as :=, +=, ++=, Def.task, or Def.setting.
val protobufResourceFolder = (resourceManaged in Compile).value / "protobuf"
^
Almost. As the error message states you simply cannot unwrap a value outside the sbt dsl. So, something like this is usually done using a SettingKey:
val protobufResourceFile = settingKey[File]("Protobuf resource file ...")
protobufResourceFile := (resourceManaged in Compile).value / "protobuf"
resourceGenerators in Compile += Def.task {
val jar = (update in Compile).value
.matching((_: ModuleID) == nemesisProto)
.head
IO.unzip(jar, protobufResourceFile.value).toSeq
}.taskValue
PB.protoSources in Compile := Seq(protobufResourceFile.value)
Though, in this specific case it may be overkill.

How to get Kotlin AST?

I have a string with Kotlin source in it. How can I compile it at run-time and get abstract syntax tree and types info to analyze?
I have some investigation of Kotlin compiler. Some proof of concept to getting of AST can be seen on my GitHub repo.
It's a sketch only, but can be helpful:
class KotlinScriptParser {
companion object {
private val LOG = Logger.getLogger(KotlinScriptParser.javaClass.name)
private val messageCollector = object : MessageCollector {
override fun report(severity: CompilerMessageSeverity, message: String, location: CompilerMessageLocation) {
val path = location.path
val position = if (path == null) "" else "$path: (${location.line}, ${location.column}) "
val text = position + message
if (CompilerMessageSeverity.VERBOSE.contains(severity)) {
LOG.finest(text)
} else if (CompilerMessageSeverity.ERRORS.contains(severity)) {
LOG.severe(text)
} else if (severity == CompilerMessageSeverity.INFO) {
LOG.info(text)
} else {
LOG.warning(text)
}
}
}
private val classPath: ArrayList<File> by lazy {
val classpath = arrayListOf<File>()
classpath += PathUtil.getResourcePathForClass(AnnotationTarget.CLASS.javaClass)
classpath
}
}
fun parse(vararg files: String): TopDownAnalysisContext {
// The Kotlin compiler configuration
val configuration = CompilerConfiguration()
val groupingCollector = GroupingMessageCollector(messageCollector)
val severityCollector = MessageSeverityCollector(groupingCollector)
configuration.put(CLIConfigurationKeys.MESSAGE_COLLECTOR_KEY, severityCollector)
configuration.addJvmClasspathRoots(PathUtil.getJdkClassesRoots())
// The path to .kt files sources
files.forEach { configuration.addKotlinSourceRoot(it) }
// Configuring Kotlin class path
configuration.addJvmClasspathRoots(classPath)
configuration.put(JVMConfigurationKeys.MODULE_NAME, JvmAbi.DEFAULT_MODULE_NAME)
configuration.put<List<AnalyzerScriptParameter>>(JVMConfigurationKeys.SCRIPT_PARAMETERS, CommandLineScriptUtils.scriptParameters())
val rootDisposable = Disposer.newDisposable()
try {
val environment = KotlinCoreEnvironment.createForProduction(rootDisposable, configuration, EnvironmentConfigFiles.JVM_CONFIG_FILES)
val ktFiles = environment.getSourceFiles()
val sharedTrace = CliLightClassGenerationSupport.NoScopeRecordCliBindingTrace()
val moduleContext = TopDownAnalyzerFacadeForJVM.createContextWithSealedModule(environment.project,
environment.getModuleName())
val project = moduleContext.project
val allFiles = JvmAnalyzerFacade.getAllFilesToAnalyze(project, null, ktFiles)
val providerFactory = FileBasedDeclarationProviderFactory(moduleContext.storageManager, allFiles)
val lookupTracker = LookupTracker.DO_NOTHING
val packagePartProvider = JvmPackagePartProvider(environment)
val container = createContainerForTopDownAnalyzerForJvm(
moduleContext,
sharedTrace,
providerFactory,
GlobalSearchScope.allScope(project),
lookupTracker,
packagePartProvider)
val additionalProviders = ArrayList<PackageFragmentProvider>()
additionalProviders.add(container.javaDescriptorResolver.packageFragmentProvider)
return container.lazyTopDownAnalyzerForTopLevel.analyzeFiles(TopDownAnalysisMode.LocalDeclarations, allFiles, additionalProviders)
} finally {
rootDisposable.dispose()
if (severityCollector.anyReported(CompilerMessageSeverity.ERROR)) {
throw RuntimeException("Compilation error")
}
}
}
}
fun main(args: Array<String>) {
val scriptFile = "/media/data/java/blackfern/kotlin-compile-test/test.kt"
val parser = KotlinScriptParser()
// Getting a root element of the AST
val analyzeContext = parser.parse(scriptFile)
// Sample AST investigation
val function = analyzeContext.functions.keys.first()
val body = function.bodyExpression as KtBlockExpression
}
There's no standard API to do this at the moment. You can play with the Kotlin compiler and REPL source code to try to achieve this.

Is there a way to automatically generate the gradle dependencies declaration in build.gradle?

This is in the context of converting an existing java project into a gradle project.
Is there a tool or webservice that would help generate the dependencies declaration in the build.gradle by pointing to a directory that contain all the dependent jars ?
In general there no such tool but if all the jars all structured (I mean paths: com/google/guava/guava and so on) it should be easy to write such script on your own.
Mind that it can also be done by importing the whole folder in the following way:
repositories {
flatDir {
dirs 'lib'
}
}
or
dependencies {
runtime fileTree(dir: 'libs', include: '*.jar')
}
In my comments to #Opal 's answer, I said that I was working on a quick and dirty groovy script to achieve this. I forgot to attach the script after that. Apologies for the same.
Here's my quick and dirty script. Does solve the purpose partially.
Hoping that someone can improve on it.
#! /usr/bin/env groovy
#Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.5.2' )
import static groovyx.net.http.ContentType.JSON
import groovyx.net.http.RESTClient
import groovy.json.JsonSlurper
import groovy.util.slurpersupport.GPathResult
import static groovyx.net.http.ContentType.URLENC
//def artifactid = "activation"
//def version = "1.1"
//def packaging = "jar"
//
//def mavenCentralRepository = new RESTClient( "http://search.maven.org/solrsearch/select?q=a:%22${artifactid}%22%20AND%20v:%22${version}%22%20AND%20p:%22${packaging}%22&rows=20&wt=json".toString() )
////basecamp.auth.basic userName, password
//
//def response = mavenCentralRepository.get([:])
//println response.data.response.docs
//
//def slurper = new JsonSlurper()
//def parsedJson = slurper.parseText(response.data.toString());
//
//println parsedJson.response.docs.id
def inputFile = new File("input.txt");
def fileList = []
fileList = inputFile.readLines().collect {it.toString().substring(it.toString().lastIndexOf('/') + 1)}
def artifactIDvsVersionMap = [:]
fileList.collectEntries(artifactIDvsVersionMap) {
def versionIndex = it.substring(0,it.indexOf('.')).toString().lastIndexOf('-')
[it.substring(0,versionIndex),it.substring(versionIndex+1).minus(".jar")]
}
println artifactIDvsVersionMap
new File("output.txt").delete();
def output = new File("output.txt")
def fileWriter = new FileWriter(output, true)
def parsedGradleParameters = null
try {
parsedGradleParameters = artifactIDvsVersionMap.collect {
def artifactid = it.key
def version = it.value
def packaging = "jar"
def mavenCentralRepository = new RESTClient( "http://search.maven.org/solrsearch/select?q=a:%22${artifactid}%22%20AND%20v:%22${version}%22%20AND%20p:%22${packaging}%22&rows=20&wt=json".toString() )
def response = mavenCentralRepository.get([:])
println response.data.response.docs.id
def slurper = new JsonSlurper()
def parsedJson = slurper.parseText(response.data.toString());
def dependency = parsedJson.response.docs.id
fileWriter.write("compile '${dependency}'")
fileWriter.write('\n')
sleep (new Random().nextInt(20));
return parsedJson.response.docs.id
}
} finally {
fileWriter.close()
}
println parsedGradleParameters
Groovy pros - Pardon if the code is not not clean. :)

Resources