Please stop incrementing the classfile version number when there are no format changes (original) (raw)

Luke Hutchison luke.hutch at gmail.com
Sat Oct 12 15:14:00 UTC 2019

On Sat, Oct 12, 2019 at 6:09 AM Mike Hearn <mike at plan99.net> wrote:

Take this example. When Java 10 came out I wanted to try JPMS with Kotlin in a hobby project. But that didn't work: Kotlin's stdlib wasn't modularised. I investigated why not and it turned out JetBrains had tried but hit a critical problem - Android had a tool that barfed when trying to parse module-info.class, because of course that file isn't a class, it's a module. So modularising anything used by Android devs was impossible. Google eventually fixed it but it was a low priority issue, Android devs have upgrade latency, and nobody wants to release a modular JAR until most Android devs have upgraded.

In turn that caused JetBrains to punt modularisation of the Kotlin ecosystem until Kotlin 1.4, their next major release, so that added even more latency, and because modules can only depend on other modules that means the entire Kotlin ecosystem can't easily adopt JPMS and thus jlink, ensuring there was mostly no reason to upgrade to Java 9+. Kotlin 1.4 still hasn't happened nearly two years later:

Yes, this is an important point, since module-info.class was the most common compatibility issue that came up with the constant pool changes in JDK 9. I'll give more context for how it affected ClassGraph, since it is illustrative:

ClassGraph builds on JDK 8 in JDK 7 compatibility mode, which is a nice way to support JDK 7 while still allowing the use of @FunctionalInterface (defined in JDK 8). However, ClassGraph also wanted to be forwards compatible with the module system, so I added module-info.class, which was the only class built for JDK 9. Immediately after release, I started getting bug reports that all sorts of build and runtime environments were breaking for people, because projects were using multiple classfile scanners somewhere in the classpath, often unknown to the user, due to the transitive dependency problem.

I saw a clever suggestion that the safest place to put module-info.class was in META-INF/versions/9 , since classfiles should not be contained in META-INF in JDK 8 or earlier. So I moved module-info.class there. This resolved about half the problems, as far as I could tell. However, there were still classpath scanners (including the Android one that Mike referred to) that scanned the entire classpath recursively for classes, and would throw an exception and bomb on the new module descriptor no matter where it was.

There were two classes of problems caused by module-info.class: (1) classpath scanners that would throw an exception based on the classfile format number alone (the "once every 6 months problem"); and (2) classpath scanners that would throw an exception due to assuming the module descriptor was a traditional class, trying to parse it, and failing (the "once every ten years or so problem").

In both cases, the robustness principle that I linked earlier in this thread would have saved the day for most users: https://en.wikipedia.org/wiki/Robustness_principle In this context, this translates to: (1) the classfile format and the compiler should support parsing of a valid subset of the classfile through a self-descriptive format, to allow graceful failover when new features are introduced ("be conservative in what you send"); and (2) the scanners involved should possibly have ignored "corrupt" classfiles, rather than dying with an exception ("be liberal in what you accept").

More than 2 years after the release of JDK 9, I am still getting bug reports where simply including ClassGraph on the classpath on an old version of an app server (JBoss or Spring) causes the application to crash with an exception (because another scanner in the runtime platform finds ClassGraph's module-info.class). People blame ClassGraph for this, since it is supposed to be (and is) compatible back to JDK 7. They are usually not able to upgrade their runtime environment to fix the issue. (This is not just limited to ClassGraph of course -- even newer log4j releases cause this issue, so yes, it will get fixed in time as more people hit this: https://issues.jboss.org/browse/JBEAP-15716?_sscc=t ).

I am considering switching to automatic module naming to solve this for once and for all, but that is just sidestepping the problem -- and as I have pointed out, the faster JDK release cadence will cause these sorts of problems much more often in future if scanners just bomb on classfile versions they don't recognize, even if they could actually handle the classfile just fine.

More information about the jdk-dev mailing list