Compilation times with Derive4J-generated code

[AndreasVoellmy]

This is not really an issue - just a discussion :)

I've noticed quite long compilation times in a Java project that uses (among other things) Derive4J annotation processor. I have not tracked down the problem, and it might not have anything to do with Derive4J at all. However, I suspect that type inference of code written with Derive4J fluent cases and casesOf API might be really slow (I am using Java 1.8 compiler). For example, I came across some these compiler issues that discuss exponential slow down seen in type inference and which might be relevant to Derive4J:

https://bugs.openjdk.java.net/browse/JDK-8145742
https://bugs.openjdk.java.net/browse/JDK-8055984
https://bugs.openjdk.java.net/browse/JDK-8067767

Has anyone else here noticed long compilation using Derive4J? If so have you figured out why? If it is the type-inference issue, that might be fixed in Java 9 (all those bugs I linked above are resolved in Java 9). Regardless of that, are there any workarounds for avoiding the long compilation times?

[AndreasVoellmy]

I created a benchmark to help quantify and explore this issue. See https://github.com/AndreasVoellmy/derive4jbenchmark

It consists of 200 small @Data-annotated classes. It takes me about 45 seconds to run the mvn package command, which seems really long.

[AndreasVoellmy]

To see if javac is the culprit, I ran the build normally first to get all the generated code. Then I touched all the source and generated Java code so that their file mod times are updated. Then I re-ran the Maven build, but with -proc:none as an argument to javac. That apparently turns off annotation processing, but still does compilation. It ran in 2.5 seconds. So that suggests that it isn't type inference, but is more likely due to Derive4J (e.g. scanning for @Data-annotated classes, generating code and writing these files out.

[jbgi]

@AndreasVoellmy thank-you very much for investigating this. I will have a look shortly at what can be optimized, there is certainly ways to improve that time.

ps: some past release did some of the processing in parallel but it caused racing conditions with jar loading (that is not thread-safe in the context of annotation processing)

[jbgi]

@AndreasVoellmy It looks like maven does not recompile sources in generated-sources when using the -proc:none flag. If I move the generated sources into src/main/java I get back the long compile time.
So it looks like the biggest hit is compiling/writing generated classes (~75%), not generating the java files (~25%).

The ways I see to mitigate the problem are:

1. make better use of incremental compilation.
2. reduce the number of generated files, this mainly (and unfortunately) means disabling caseOf or/and cases derivations (that can be done via the org.derive4j.Derive#make property).
3. improve derive4j generation speed (but will only impact the 25% hit and it is possible that a large part of it is due to javac api that are used)

[AndreasVoellmy]

I am already doing 2 to the extent that I can (e.g. I don't need both cases and caseOf), but even so, compilation times are high.

1. Seems the most promising and useful. What do you mean by "incremental compilation"? Do you mean that the annotation processor could avoid writing out a Java class if it is identical to what was previously generated, and then the Java compiler would avoid recompiling the class?

[jbgi]

I just meant using mvn compile instead of mvn clean compile. Then the annotation processor only process classes that have been modified since last compile.

[AndreasVoellmy]

The OpenJDK issues I mentioned above seem to be related to type inference. Are there any opportunities to have Derive4J-generated code eliminate code that triggers the Java compiler's type inference?

[AndreasVoellmy]

Sorry - accidentally hit close!

[jbgi]

Right, I forgot that I wanted to try out expliciting type args in generated code. Let's see what this brings.