Please stop incrementing the classfile version number when there are no format changes (original) (raw)
Brian Goetz brian.goetz at oracle.com
Fri Oct 11 20:21:40 UTC 2019
- Previous message: Please stop incrementing the classfile version number when there are no format changes
- Next message: Please stop incrementing the classfile version number when there are no format changes
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Not true at all. The classfile version is in a well-known place in the classfile format, and tools should not attempt to interpret classfiles for whose version they don't understand. Your argument below is a stark demonstration of the moral hazard of our working hard to keep the classfile format stable -- users assume (with absolutely no basis) that it should be safe to just blindly parse a classfile whose version they do not recognize, and treat it as a bug that somehow this isn't possible. The JVMS has been specified such that it is allowable to entirely redo the bytecode set in every version. The protection against classfile misinterpretation is not trying to reason about classfile versions you don't recognize.
I understand you are thinking "well, it's possible to define the classfile format in a way that is more self-describing, and then it would be easier, so isn't it obvious that we should just do that?" But, you're making the classic mistake of considering only the benefit, and not the cost. For example, fatter classfile structures mean longer startup times for 100% of Java developers. Should we really punish the 100% with worse startup for the sake of the .00001% of those who are maintaining bytecode-parsing tools, but aren't committed to keeping up? That seems like putting the burden in the wrong place.
ASM works hard to have a release ready for JDK N by the time JDK is ready to ship. If other bytecode libraries want to play in this space, that's the bar, and it's a reasonable one. It's not like changes to the classfile format aren't broadly discussed in public forums for a long time prior to making a change. The ASM guys have figured out what good citizenship looks like here (thanks Remi!); other libraries should take a page from their book.
On 10/11/2019 4:01 PM, Luke Hutchison wrote:
At the very least, when the classfile format is updated, old tools should not break when encountering classfiles containing new record types -- the tools should still be able to handle the subset of classfile functionality that they have always understood, ignoring anything new that they don't understand.
However, this is not possible due to a problem with the design of the classfile format: record lengths are not always specifically encoded in the format itself, so when new record types are added to the classfile format, old parsers can't continue to parse the classfile as soon as the first record of an unknown type is encountered. This is an issue with at least one part of the classfile format specification, the constant pool. I ran into this problem with the ClassGraph library. When constant pool tag types 19 (for module) and 20 (for package) were encountered in a classfile generated by JDK 9+, ClassGraph simply had to stop parsing the classfile and throw an exception, until the library was fixed to handle these entry types. There was no way to correctly guess how large those new constant pool entries would be, based on the definition of the cpinfo record type (which does not include a length field): https://docs.oracle.com/javase/specs/jvms/se13/html/jvms-4.html#jvms-4.4 https://github.com/classgraph/classgraph/blob/master/src/main/java/io/github/classgraph/Classfile.java#L1084 By contrast, method and field attributes always have a size field, so even if you don't know how to parse a new attribute type, you can just skip over the number of bytes stated in the length field to move on to the next attribute. https://docs.oracle.com/javase/specs/jvms/se13/html/jvms-4.html#jvms-4.7 If the entire classfile format followed this design principle, then a lot of compatibility issues could be solved by having tools simply parse the subset of the format that they already knew how to handle. This would allow for a much stronger backwards compatibility claim to be made. To fix this specific issue, the classfile spec should be updated to say that if any new constant pool attributes that are added, with a tag index greater than 20, they will instead use a new cpinfowithlen entry type that includes a length field. The rest of the classfile format spec should similarly be carefully considered to make sure a historically valid subset of functionality can always be parsed by older tools. Then the exact classfile format number won't matter as much -- a tool can simply parse what it can understand.
On Fri, Oct 11, 2019 at 1:32 PM Christian Stein <sormuras at gmail.com> wrote: On Fri, Oct 11, 2019 at 3:51 PM Brian Goetz <brian.goetz at oracle.com> wrote:
[...] The way the class format is versioned is proving disastrous for the community. Again, you’re acting like this is a new thing. Given that we’ve been doing it this way for nearly 25 years, it can’t be all that disastrous. An extreme thought experiment: Update the class file version every 6 weeks, days, ... and see what happens to "all" build tools. A lot of build tools, ranging from foundation tools like
javac
to Maven will just continue to work. This also goes for Gradle using its Kotlin-based DSL, if I'm not mistaken. "If it hurts, do it more often." I guess, "all" build tools would soon be agnostic to the class file version -- or updated their delivery process to match the 6 ... hours cadence. Cheers, Christian
- Previous message: Please stop incrementing the classfile version number when there are no format changes
- Next message: Please stop incrementing the classfile version number when there are no format changes
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]