Deferred files causing stochastic build failures

[aptenodytes-forsteri]

We have a project with 317 layout .xml files (maybe the large number is relevant).

In CI and occasionally locally, the build fails because the generated stubs source jar for layout bindings has an empty file at the root instead of the valid generate file.

For example:

 unzip -l bazel-bin/path/to/app/lib_dev-stubs_binding.srcjar | grep FragmentFoo
        0  1980-02-01 00:00   FragmentFooBinding.java
     1715  1980-02-01 00:00   com/ourcompany/databinding/FragmentFoo2Binding.java

In the above example, you'll see that FragmentFoo2Binding.java is in the correct spot and has a valid size. But FragmentFooBinding.java is empty and isn't at the proper path.

If I set verbose=true for the SourceJarCreator, I'll see that we're hitting this line:

grab-bazel-common/tools/db-compiler-lite/src/main/java/com/grab/databinding/stub/util/jars/SourceJarCreator.kt

Line 163 in 987bb4e

System.err.println("""could not determine jar entry name for $path. Body:\n$body}""")

for FragmentFooBinding. And the body will be empty.

One workaround I see is to re-read the file when visiting deferred files.

For example:

        filenameHelper.visitDeferredEntries { path, jarFilename, bytes ->
            if (jarFilename == null) {
                val newBytes = Files.readAllBytes(path) // ADDED THIS
                if (verbose) {
                    val body = newBytes.toString(Charset.defaultCharset())
                    System.err.println("""could not determine jar entry name for $path. Body:\n$body}""")
                    addEntry(path.fileName.toString(), path, newBytes)
                } else {
                    // if not verbose silently add files at the root.
                    addEntry(path.fileName.toString(), path, newBytes)
                }
            } else {
                System.err.println("adding deferred source file $path -> $jarFilename")
                addEntry(jarFilename, path, bytes)
            }
        }

Perhaps something about bazel causes this file not to be available yet in the first go around? If it's not the sheer number of layout files, maybe it is something in my .bazelrc?

[aptenodytes-forsteri]

One thing to note is that the files that fail to generate properly are stochastic. It is a different file everytime. So sometimes it will be FragmentFooBinding.java, other times it will be FragmentBarBinding.java - it all depends on which file gets arbitrary deferred.

[aptenodytes-forsteri]

I think I just saw an example of my workaround still failing. I don't fully understand why some files get deferred, and how best to wait for them to become available before attempting to read their contents.

[aptenodytes-forsteri]

I am on Ubuntu 18.04. Perhaps it's an operating system issue where the files aren't sync'd properly before the next operation?

It's as if the files are not guaranteed to be written after generate, almost like generate(packageName, layoutBindings) is async even though I know it's not.

        val dataBindingClasses = command.bindingClassGenerator().generate(
            packageName,
            layoutBindings
        )
       // All files are not actually guaranteed to be written here. Need some sort of filesystem sync?
        command.srcJarPackager.packageSrcJar(
            inputDir = dataBindingClasses,
            outputFile = stubClassJar,
            verbose=true
        )

[aptenodytes-forsteri]

Not being a java/kotlin expert, I also tried something like this:

    private fun addJavaLikeSourceFile(sourceFile: Path) {
        val fis = FileInputStream(sourceFile.toString())
        val channel = fis.getChannel()
        channel.force(true);
        fis.close();
        channel.close();

Maybe something like this ensures that the filesystem is sync'd before trying to read the files?

[aptenodytes-forsteri]

Now trying something like this:

    private fun addJavaLikeSourceFile(sourceFile: Path) {
        var count = 0;
        var bytes = Files.readAllBytes(sourceFile)
        while ((bytes == null || bytes.size == 0) && count < 30) {
            println("$sourceFile does not yet exist $count")
            Thread.sleep(100)
            count++
            bytes = Files.readAllBytes(sourceFile)
        }

[arunkumar9t2]

Hey thanks for the report, will investigate this and get back.

[colinrgodsey]

possibly related to this, but we have a project that has a very large number of Android resources (15k+), and when building all of our targets one of the R.java files ends up becoming corrupt:

    public static int MenuItem_android_orderInCategory = 0;

    public static int MenuItem_android_title = 0;

    public static int MenuItem_android_titleCondensed = 0;

  droid_titleCondensed = 0;

    public static int MenuItem_android_visible = 0;

    public static int MenuItem_contentDescription = 0;

    public static int MenuItem_iconTint = 0;

    public static int MenuItem_iconTintMode = 0;

The issue is mitigated by reducing the number of build targets we run in one pass.