Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use eclipse gradle plugin to get classpath with sources #439

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Use eclipse gradle plugin to get classpath with sources #439

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
32 changes: 20 additions & 12 deletions shared/src/main/kotlin/org/javacs/kt/classpath/GradleClassPathResolver.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,18 @@ internal class GradleClassPathResolver(private val path: Path, private val inclu

Copy link
Contributor

@RenFraser RenFraser May 17, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than building string commands, invoking the build and parsing the string output, line by line, I think that this whole file could just become a single call to the classpath, using the Eclipse model from the Gradle project connection. There's more information on the third party tooling web doc. They say that to find examples of calling Gradle's API from Java, you need to go through the JavaDoc, starting at the GradleConnector.

This could simplify the class a fair bit and improve the time it takes for the server to spin up. What do you think @aiguofer?

package org.javacs.kt.classpath

import org.gradle.tooling.GradleConnector
import org.gradle.tooling.model.eclipse.EclipseProject
import java.io.File
import java.nio.file.Path

internal class GradleClassPathResolver(private val path: Path) : ClassPathResolver {
    override val resolverType: String = "Gradle"
    private val projectDirectory: Path get() = path.parent
    override val classpath: Set<ClassPathEntry> = GradleConnector
        .newConnector()
        .forProjectDirectory(File(projectDirectory.toString()))
        .connect()
        .use { connector ->
            val model = connector.getModel(EclipseProject::class.java)
            model.classpath.map {
                ClassPathEntry(
                    compiledJar = it.source.toPath(),
                    sourceJar = null,
                )
            }.toSet()
        }
}

cc @themkat

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main issue is that apparently the EclipseProject doesn't catch all the dependencies, especially for android dependencies. @Strum355 mentioned above that for SCIP they had to use a combination of EclipseProject, IdeaProject, and a gradle task that also builds some strings and passes them in to be parsed. I currently have a locally working branch that does it similarly, but the gradle task with string parsing definitely makes it slower.

When I have some time I'll try to make some comparisons of the outputs of each.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds great, thanks @aiguofer! Strum's comment now makes much more sense!

return readDependenciesViaGradleCLI(projectDirectory, scripts, tasks)
.apply { if (isNotEmpty()) LOG.info("Successfully resolved dependencies for '${projectDirectory.fileName}' using Gradle") }
.map { ClassPathEntry(it, null) }.toSet()
}
override val buildScriptClasspath: Set<Path> get() {
override val buildScriptClasspath: Set<Path>
get() {
return if (includeKotlinDSL) {
val scripts = listOf("kotlinDSLClassPathFinder.gradle")
val tasks = listOf("kotlinLSPKotlinDSLDeps")

return readDependenciesViaGradleCLI(projectDirectory, scripts, tasks)
.apply { if (isNotEmpty()) LOG.info("Successfully resolved build script dependencies for '${projectDirectory.fileName}' using Gradle") }
.map {
it.compiledJar
}.toSet()
} else {
emptySet()
}
Expand Down Expand Up @@ -70,24 +73,20 @@ private fun getGradleCommand(workspace: Path): Path {
}
}

private fun readDependenciesViaGradleCLI(projectDirectory: Path, gradleScripts: List<String>, gradleTasks: List<String>): Set<Path> {
private fun readDependenciesViaGradleCLI(projectDirectory: Path, gradleScripts: List<String>, gradleTasks: List<String>): Set<ClassPathEntry> {
LOG.info("Resolving dependencies for '{}' through Gradle's CLI using tasks {}...", projectDirectory.fileName, gradleTasks)

val tmpScripts = gradleScripts.map { gradleScriptToTempFile(it, deleteOnExit = false).toPath().toAbsolutePath() }
val gradle = getGradleCommand(projectDirectory)

val command = listOf(gradle.toString()) + tmpScripts.flatMap { listOf("-I", it.toString()) } + gradleTasks + listOf("--console=plain")
val dependencies = findGradleCLIDependencies(command, projectDirectory)
?.also { LOG.debug("Classpath for task {}", it) }
.orEmpty()
.filter { it.toString().lowercase().endsWith(".jar") || Files.isDirectory(it) } // Some Gradle plugins seem to cause this to output POMs, therefore filter JARs
.toSet()

tmpScripts.forEach(Files::delete)
return dependencies
}

private fun findGradleCLIDependencies(command: List<String>, projectDirectory: Path): Set<Path>? {
private fun findGradleCLIDependencies(command: List<String>, projectDirectory: Path): Set<ClassPathEntry> {
val (result, errors) = execAndReadStdoutAndStderr(command, projectDirectory)
if ("FAILURE: Build failed" in errors) {
LOG.warn("Gradle task failed: {}", errors)
Expand All @@ -101,13 +100,22 @@ private fun findGradleCLIDependencies(command: List<String>, projectDirectory: P
return parseGradleCLIDependencies(result)
}

private val artifactPattern by lazy { "kotlin-lsp-gradle (.+)(?:\r?\n)".toRegex() }
private val artifactPattern by lazy { "kotlin-lsp-gradle path:(.+) source:(.+)".toRegex() }
private val gradleErrorWherePattern by lazy { "\\*\\s+Where:[\r\n]+(\\S\\.*)".toRegex() }

private fun parseGradleCLIDependencies(output: String): Set<Path>? {
private fun parseGradleCLIDependencies(output: String): Set<ClassPathEntry> {
LOG.debug(output)
val artifacts = artifactPattern.findAll(output)
.mapNotNull { Paths.get(it.groups[1]?.value) }
.filterNotNull()
.map {
val path = it.groups[1]?.value
val source = it.groups[2]?.value
val jarPath = if (path == "null" || path == null) null else Path.of(path)
val sourceJarPath = if (source == "null" || source == null) null else Path.of(source)
if (jarPath != null && (path!!.lowercase().endsWith(".jar") || Files.isDirectory(jarPath))) {
LOG.debug { "Adding path:$jarPath source: $sourceJarPath to classpath" }
return@map ClassPathEntry(jarPath, sourceJarPath)
} else return@map null
}.filterNotNull()

return artifacts.toSet()
}
37 changes: 30 additions & 7 deletions shared/src/main/resources/projectClassPathFinder.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,13 @@
import org.gradle.plugins.ide.eclipse.model.ClasspathEntry
import org.gradle.plugins.ide.eclipse.model.EclipseModel
import org.gradle.plugins.ide.eclipse.model.Library

allprojects { project ->
apply plugin: "eclipse"

task kotlinLSPProjectDeps { task ->
doLast {
def classpaths = new HashMap()
System.out.println ""
System.out.println "gradle-version $gradleVersion"
System.out.println "kotlin-lsp-project ${project.name}"
Expand Down Expand Up @@ -28,33 +35,35 @@ allprojects { project ->
File.separator + variantBase +
File.separator + "classes"

System.out.println "kotlin-lsp-gradle $buildClasses"
classpaths[buildClasses.toString()] = null

def userClasses = project.getBuildDir().absolutePath +
File.separator + "intermediates" +
File.separator + "javac" +
File.separator + variantBase +
File.separator + "compile" + variantBase.capitalize() + "JavaWithJavac" + File.separator + "classes"

System.out.println "kotlin-lsp-gradle $userClasses"
classpaths[userClasses.toString()] = null

def userVariantClasses = project.getBuildDir().absolutePath +
File.separator + "intermediates" +
File.separator + "javac" +
File.separator + variantBase +
File.separator + "classes"

System.out.println "kotlin-lsp-gradle $userVariantClasses"
classpaths[userVariantClasses.toString()] = null

variant.getCompileClasspath().each {
System.out.println "kotlin-lsp-gradle $it"
classpaths[it.toString()] = null
}
}


} else if (project.hasProperty('sourceSets')) {
// Print the list of all dependencies jar files.
sourceSets.forEach {
it.compileClasspath.forEach {
System.out.println "kotlin-lsp-gradle $it"
classpaths[it.toString()] = null
}
}
}
Expand All @@ -69,18 +78,32 @@ allprojects { project ->
kotlinExtension.targets.names.each {
def classpath = configurations["${it}CompileClasspath"]
classpath.files.each {
System.out.println "kotlin-lsp-gradle $it"
classpaths[it.toString()] = null
}
}
}

EclipseModel eclipseModel = project.extensions.getByType(EclipseModel.class)

for (ClasspathEntry dep : (eclipseModel.getClasspath().resolveDependencies())) {
if (dep instanceof Library) {
if (classpaths.get(dep.getPath()) == null){
classpaths[dep.getPath()] = dep.getSourcePath()?.getPath()
}
}
}

for (e in classpaths) {
System.out.println "kotlin-lsp-gradle path:$e.key source:$e.value"
}
}
}

task kotlinLSPAllGradleDeps {
doLast {
fileTree("$gradle.gradleHomeDir/lib")
.findAll { it.toString().endsWith '.jar' }
.forEach { System.out.println "kotlin-lsp-gradle $it" }
.forEach { System.out.println "kotlin-lsp-gradle path:$it source:null" }
}
}
}