RFR(M): 8081674: EmptyStackException at startup if running with extended or unsupported charset (original) (raw)

Mandy Chung mandy.chung at oracle.com
Wed Jun 10 19:02:35 UTC 2015

On Jun 10, 2015, at 9:23 AM, Volker Simonis <volker.simonis at gmail.com> wrote:

Hy Mandy, that's a real good proposal and it only requires changes in the jdk repository which makes it even more attractive. I've just checked and it does indeed solve the initial EmptyStackException problem in both jdk8 and jdk9. Here are the corresponding webrevs: http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk8/

ClassLoader.java It should have done that. Nit: Can you move line 1693 to 1868 closer to loadLibrary0 (the caller of findBuiltinLib)? Also make it private.

Same applies to jdk9.

Otherwise looks good. No need to generate a webrev as long as you make the change before the push.

http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk9/

The only difference between 8 and 9 is the different location of the source files. Compared to your proposal I've only updated the copyright years where necessary and removed one comment in JavajavalangClassLoaderfindBuiltinLib() which seemed unnecessary to me now that findBuiltinLib() isn't located in NativeLibrary anymore.

Thanks for taking it out.

On jdk9 we are now running in another problem which is caused by the fact that the extended charsets are now being loaded by ServiceLoader (I've detailed that already in the bug report). But I think we can leave the fix of that problem to another change as this seems to require some more reasoning (see Alan's comments in the bug report).

Yes this is a harder problem that have to investigate further.

So are you OK now with pushing this change to jdk9 and requesting a downport to 8u-dev afterwards?

Yes. Approved.

Mandy

Thank you and best regards, Volker

On Wed, Jun 10, 2015 at 5:14 PM, Mandy Chung <mandy.chung at oracle.com> wrote: Have you checked out this patch moving out findBuiltinLib from NativeLibrary class? http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8081674/webrev.00/

Mandy

On Jun 10, 2015, at 12:34 AM, Volker Simonis <volker.simonis at gmail.com> wrote:

Mandy, the example/stacktrace I sent was with 8u. The problem isn't related to libzip, it is only the first library which gets loaded with System.loadLibrary(). The problem will occur with every library which gets loaded by System.loadLibrary() because java.lang.ClassLoader$NativeLibrary.findBuiltinLib() always calls jniFindClass() if we're running on a unsupported locale. On Tue, Jun 9, 2015 at 8:03 PM, Mandy Chung <mandy.chung at oracle.com> wrote: Does my patch work fine on 8u? If it works fine, I prefer to get that simple fix in 8u and take the time to have a better fix in 9 (jdk9 is still under development and I have assumed that it's not a show stopper to you. Let me know if it blocks you).

A question to Sherman - do we have adequate tests to verify the move of System.loadLibrary("zip") from system initialization to ZipFile initialization for 8u? Mandy

On 06/09/2015 10:09 AM, Volker Simonis wrote: Hi Mandy, thanks for looking into this. Uunfortunately your fix only helps to fix "java -version" :( Running even a minimal HelloWorld will still fail with the following stack trace which is still caused by the same EmptyStackException: Error: A JNI error has occurred, please check your installation and try again Exception in thread "main" java.lang.ExceptionInInitializerError at sun.misc.Unsafe.ensureClassInitialized(Native Method) at sun.misc.SharedSecrets.getJavaUtilZipFileAccess(SharedSecrets.java:158) at sun.misc.URLClassPath$JarLoader.(URLClassPath.java:765) at sun.misc.URLClassPath$3.run(URLClassPath.java:530) at sun.misc.URLClassPath$3.run(URLClassPath.java:520) at java.security.AccessController.doPrivileged(Native Method) at sun.misc.URLClassPath.getLoader(URLClassPath.java:519) at sun.misc.URLClassPath.getLoader(URLClassPath.java:492) at sun.misc.URLClassPath.getNextLoader(URLClassPath.java:457) at sun.misc.URLClassPath.access$100(URLClassPath.java:64) at sun.misc.URLClassPath$1.next(URLClassPath.java:239) at sun.misc.URLClassPath$1.hasMoreElements(URLClassPath.java:250) at java.net.URLClassLoader$3$1.run(URLClassLoader.java:601) at java.net.URLClassLoader$3$1.run(URLClassLoader.java:599) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader$3.next(URLClassLoader.java:598) at java.net.URLClassLoader$3.hasMoreElements(URLClassLoader.java:623) at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) at sun.misc.CompoundEnumeration.next(CompoundEnumeration.java:45) at sun.misc.CompoundEnumeration.hasMoreElements(CompoundEnumeration.java:54) at java.util.ServiceLoader$LazyIterator.hasNextService(ServiceLoader.java:354) at java.util.ServiceLoader$LazyIterator.hasNext(ServiceLoader.java:393) at java.util.ServiceLoader$1.hasNext(ServiceLoader.java:474) at java.nio.charset.Charset$1.getNext(Charset.java:350) at java.nio.charset.Charset$1.hasNext(Charset.java:365) at java.nio.charset.Charset$2.run(Charset.java:410) at java.nio.charset.Charset$2.run(Charset.java:407) at java.security.AccessController.doPrivileged(Native Method) at java.nio.charset.Charset.lookupViaProviders(Charset.java:406) at java.nio.charset.Charset.lookup2(Charset.java:477) at java.nio.charset.Charset.lookup(Charset.java:464) at java.nio.charset.Charset.isSupported(Charset.java:505) at sun.launcher.LauncherHelper.makePlatformString(LauncherHelper.java:580) Caused by: java.util.EmptyStackException at java.util.Stack.peek(Stack.java:102) at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1759) at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1870) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1843) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1122) at java.util.zip.ZipFile.(ZipFile.java:86) ... 34 more It's obvious that the way jniFindClass is looking for the class context by calling the NativeLibrary::getFromClass method is hacky but I think that the proposed fix is the quite simple and non-intrusive. And we probably don't want a complete rework of this code for 8 anyway. So why not fix it the proposed way in 8 and 9 for now? That will still leave us time to come up with a better clean-up at least for 9. What do you think? Regards, Volker On Tue, Jun 9, 2015 at 1:35 AM, Mandy Chung <mandy.chung at oracle.com> wrote: Hi Volker, Your patch reminds me the question I have about loading zip library. In JDK 9 (and earlier release), zip library is loaded by the VM during startup (see ClassLoader::loadziplibrary). I think loadLibrary("zip") is no longer needed to be called from System::initializeSystemClass method and instead it can be loaded by java.util.zip.ZipFile static initializer. Do you mind to try the patch (below)? That may be a simpler fix. I never like the way jniFindClass to look for the class context by calling the NativeLibrary::getFromClass method. I will have to look deeper other alternative to clean that up. If taking out loadLibrary("zip") resolves your issue, this will give us time to come up with a better clean-up in the future. Mandy [1] https://bugs.openjdk.java.net/browse/JDK-4429040 diff --git a/src/share/classes/java/lang/System.java b/src/share/classes/java/lang/System.java --- a/src/share/classes/java/lang/System.java +++ b/src/share/classes/java/lang/System.java @@ -1192,10 +1192,6 @@ setOut0(newPrintStream(fdOut, props.getProperty("sun.stdout.encoding"))); setErr0(newPrintStream(fdErr, props.getProperty("sun.stderr.encoding"))); - // Load the zip library now in order to keep java.util.zip.ZipFile - // from trying to use itself to load this library later. - loadLibrary("zip"); - // Setup Java signal handlers for HUP, TERM, and INT (where available). Terminator.setup(); diff --git a/src/share/classes/java/util/zip/ZipFile.java b/src/share/classes/java/util/zip/ZipFile.java --- a/src/share/classes/java/util/zip/ZipFile.java +++ b/src/share/classes/java/util/zip/ZipFile.java @@ -83,6 +83,7 @@ static { /* Zip library is loaded from System.initializeSystemClass */ + System.loadLibrary("zip"); initIDs(); }

On 06/08/2015 07:23 AM, Volker Simonis wrote: Hi, can I please get a review at least for the part of this fix which is common for jdk8 and jdk9. It's only a few lines in src/share/vm/prims/jni.cpp for the hotspot repository and one line src/java.base/share/classes/java/lang/ClassLoader.java for the jdk repository. Thanks, Volker On Tue, Jun 2, 2015 at 6:12 PM, Volker Simonis <volker.simonis at gmail.com> wrote: Hi, can I please have review (and a sponsor) for these changes: https://bugs.openjdk.java.net/browse/JDK-8081674 http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.jdk http://cr.openjdk.java.net/~simonis/webrevs/2015/8081674.hs Please notice that the fix requires synchronous changes in the jdk and the hotspot forest. The changes themselves are by far not that big as this explanation but I found the problem to be quite intricate so I tried to explain it as good as I could. I'd suggest to read the HTML-formatted explanation in the webrev although the content is equal to the one in this mail: Using an unsupported character encoding (e.g. viVN.TCVN on Linux) results in an immediate VM failure with jdk 8 and 9: export LANG=viVN.TCVN java -version Error occurred during initialization of VM java.util.EmptyStackException at java.util.Stack.peek(Stack.java:102) at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1751) at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1862) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1835) at java.lang.Runtime.loadLibrary0(Runtime.java:870) at java.lang.System.loadLibrary(System.java:1119) at java.lang.System.initializeSystemClass(System.java:1194) This is a consequence of "8005716: Enhance JNI specification to allow support of static JNI libraries in Embedded JREs". With jdk 9 we get this error even if we're running with a supported charset which is in the ExtendedCharsets (as opposed to being in the StandardCharsets) class which is a consequence of delegating the loading of the ExtendedCharsets class to the ServiceLoader in jdk 9. export LANG=eo.iso-8859-3 output-jdk9/images/jdk/bin/java -version Error occurred during initialization of VM java.util.EmptyStackException at java.util.Stack.peek(Stack.java:102) at java.lang.ClassLoader$NativeLibrary.getFromClass(ClassLoader.java:1737) at java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Native Method) at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1866) at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1840) at java.lang.Runtime.loadLibrary0(Runtime.java:874) at java.lang.System.loadLibrary(System.java:1111) at java.lang.System.initializeSystemClass(System.java:1186) Here's why the exception happens for an unsupported charset (see the mixed stack trace below for the full details): java.lang.System.loadLibrary() wants to load libzip.so. It calls java.lang.Runtime.loadLibrary0() which at the very beginning calls the native method ClassLoader$NativeLibrary.findBuiltinLib() which checks if the corresponding library is already statically linked into the VM (introduced by 8005716). JavajavalangClassLoader00024NativeLibraryfindBuiltinLib(), the native implementation of findBuiltinLib() in Classloader.c calls GetStringPlatformChars() to convert the library name into the native platform encoding. GetStringPlatformChars() calls the helper function jnuEncodingSupported() to check if the platform encoding which is stored in the property "sun.jnu.encoding" is supported by Java. jnuEncodingSupported() is implemented as follows: static jboolean isJNUEncodingSupported = JNIFALSE; static jboolean jnuEncodingSupported(JNIEnv *env) { jboolean exe; if (isJNUEncodingSupported == JNITRUE) { return JNITRUE; } isJNUEncodingSupported = (jboolean) JNUCallStaticMethodByName ( env, &exe, "java/nio/charset/Charset", "isSupported", "(Ljava/lang/String;)Z", jnuEncoding).z; return isJNUEncodingSupported; } Once the function finds that the platform encoding is supported (by calling java.nio.charset.Charset.isSupported()) it caches this value and always returns it on following calls. However if the platform encoding is not supported, it ALWAYS calls java.nio.charset.Charset.isSupported() an every subsequent invocation. In order to call the Java method Charset.isSupported() (in JNUCallStaticMethodByName() in file jniutil.c), we have to call jniFindClass() to convert the symbolic class name "java.nio.charset.Charset" into a class reference. But unfortunately, jniFindClass() (from jni.cpp in libjvm.so) has a special handling if called from java.lang.ClassLoader$NativeLibrary to ensure that JNIOnLoad/JNIOnUnload are executed in the correct class context: instanceKlassHandle k (THREAD, thread->securitygetcallerclass(0)); if (k.notnull()) { loader = Handle(THREAD, k->classloader()); // Special handling to make sure JNIOnLoad and JNIOnUnload are executed // in the correct class context. if (loader.isnull() && k->name() == vmSymbols::javalangClassLoaderNativeLibrary()) { JavaValue result(TOBJECT); JavaCalls::callstatic(&result, k, vmSymbols::getFromClassname(), vmSymbols::voidclasssignature(), thread); if (HASPENDINGEXCEPTION) { Handle ex(thread, thread->pendingexception()); CLEARPENDINGEXCEPTION; THROWHANDLE0(ex); } So if that's the case and jniFindClass() was reallycalled from ClassLoader$NativeLibrary, then jniFindClass() calles ClassLoader$NativeLibrary().getFromClass() to find out the corresponding context class which is supposed to be saved there in a field of type java.util.Stack named "nativeLibraryContext": // Invoked in the VM to determine the context class in // JNILoad/JNIUnload static Class<?> getFromClass() { return ClassLoader.nativeLibraryContext.peek().fromClass; } Unfortunately, "nativeLibraryContext" doesn't contain any entry at this point and the invocation of Stack.peek() will throw the exception shown before. In general, the "nativeLibraryContext" stack will be filled later on in Runtime.loadLibrary0() like this: NativeLibrary lib = new NativeLibrary(fromClass, name, isBuiltin); nativeLibraryContext.push(lib); try { lib.load(name, isBuiltin); } finally { nativeLibraryContext.pop(); } such that it always contains at least one element later when jniFindClass() will be invoked. So in summary, the problem is that the implementors of 8005716 didn't took into account that calling ClassLoader$NativeLibrary.findBuiltinLib() may trigger a call to jniFindClass() if we are running on a system with an unsupported character encoding. I'd suggest the following fix for this problem: Change ClassLoader$NativeLibrary().getFromClass() to return null if the stack is empty instead of throwing an exception: static Class<?> getFromClass() { return ClassLoader.nativeLibraryContext.empty() ? null : ClassLoader.nativeLibraryContext.peek().fromClass; } Unfortunately this also requires a HotSpot change in jniFindClass() in order to properly handle the new 'null' return value: if (k.notnull()) { loader = Handle(THREAD, k->classloader()); // Special handling to make sure JNIOnLoad and JNIOnUnload are executed // in the correct class context. if (loader.isnull() && k->name() == vmSymbols::javalangClassLoaderNativeLibrary()) { JavaValue result(TOBJECT); JavaCalls::callstatic(&result, k, vmSymbols::getFromClassname(), vmSymbols::voidclasssignature(), thread); if (HASPENDINGEXCEPTION) { Handle ex(thread, thread->pendingexception()); CLEARPENDINGEXCEPTION; THROWHANDLE0(ex); } oop mirror = (oop) result.getjobject(); if (oopDesc::isnull(mirror)) { loader = Handle(THREAD, SystemDictionary::javasystemloader()); } else { loader = Handle(THREAD, InstanceKlass::cast(javalangClass::asKlass(mirror))->classloader()); protectiondomain = Handle(THREAD,

InstanceKlass::cast(javalangClass::asKlass(mirror))->protectiondomain()); } } } else { // We call ClassLoader.getSystemClassLoader to obtain the system class loader. loader = Handle(THREAD, SystemDictionary::javasystemloader()); } These changes are sufficient to solve the problem in Java 8. Unfortunately, that's still not enough in Java 9 because there the loading of the extended charsets has been delegated to ServiceLoader. But ServiceLoader calls ClassLoader.getBootstrapResources() which calls sun.misc.Launcher.getBootstrapClassPath(). This leads to another problem during class initialization of sun.misc.Launcher if running on an unsupported locale. The first thing done in sun.misc.Launcher. is the initialisation of the bootstrap URLClassPath in the Launcher. However this initialisation will eventually call Charset.isSupported() and if we are running on an unsupported locale this will inevitably end in another recursive call to ServiceLoader. But as explained below, ServiceLoader will query the Launcher's bootstrap URLClassPath which will be still uninitialized at that point. So we'll have to additionally guard guard against this situation on JDK 9 like this: private static Charset lookupExtendedCharset(String charsetName) { if (!sun.misc.VM.isBooted() || // see lookupViaProviders() sun.misc.Launcher.getBootstrapClassPath() == null) return null; This fixes the crashes, but still at the price of not having the extended charsets available during initialization until Launcher.getBootstrapClassPath is set up properly. This may be still a problem if the jdk is installed in a directory which contains characters specific to an extended encoding or if we have such characters in the command line arguments. Thank you and best regards, Volker Mixed stack trace of the initial EmptyStackException for unsupported charsets described before: Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code) j java.util.Stack.peek()Ljava/lang/Object;+1 j java.lang.ClassLoader$NativeLibrary.getFromClass()Ljava/lang/Class;+3 v ~StubRoutines::callstub V [libjvm.so+0x9d279a] JavaCalls::callhelper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x6b4 V [libjvm.so+0xcad591] os::osexceptionwrapper(void ()(JavaValue, methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x45 V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*)+0x8b V [libjvm.so+0x9d1d3b] JavaCalls::callstatic(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 V [libjvm.so+0x9d1e3f] JavaCalls::callstatic(JavaValue*, KlassHandle, Symbol*, Symbol*, Thread*)+0x9d V [libjvm.so+0x9e6588] jniFindClass+0x428 C [libjava.so+0x20208] JNUCallStaticMethodByName+0xff C [libjava.so+0x21cae] jnuEncodingSupported+0x61 C [libjava.so+0x22125] JNUGetStringPlatformChars+0x125 C [libjava.so+0xedcd] JavajavalangClassLoader00024NativeLibraryfindBuiltinLib+0x8b j java.lang.ClassLoader$NativeLibrary.findBuiltinLib(Ljava/lang/String;)Ljava/lang/String;+0 j java.lang.ClassLoader.loadLibrary0(Ljava/lang/Class;Ljava/io/File;)Z+4 j java.lang.ClassLoader.loadLibrary(Ljava/lang/Class;Ljava/lang/String;Z)V+228 j java.lang.Runtime.loadLibrary0(Ljava/lang/Class;Ljava/lang/String;)V+54 j java.lang.System.loadLibrary(Ljava/lang/String;)V+7 j java.lang.System.initializeSystemClass()V+113 v ~StubRoutines::callstub V [libjvm.so+0x9d279a] JavaCalls::callhelper(JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x6b4 V [libjvm.so+0xcad591] os::osexceptionwrapper(void ()(JavaValue, methodHandle*, JavaCallArguments*, Thread*), JavaValue*, methodHandle*, JavaCallArguments*, Thread*)+0x45 V [libjvm.so+0x9d20cf] JavaCalls::call(JavaValue*, methodHandle, JavaCallArguments*, Thread*)+0x8b V [libjvm.so+0x9d1d3b] JavaCalls::callstatic(JavaValue*, KlassHandle, Symbol*, Symbol*, JavaCallArguments*, Thread*)+0x139 V [libjvm.so+0x9d1e3f] JavaCalls::callstatic(JavaValue*, KlassHandle, Symbol*, Symbol*, Thread*)+0x9d V [libjvm.so+0xe3cceb] callinitializeSystemClass(Thread*)+0xb0 V [libjvm.so+0xe44444] Threads::initializejavalangclasses(JavaThread*, Thread*)+0x21a V [libjvm.so+0xe44b12] Threads::createvm(JavaVMInitArgs*, bool*)+0x4a6 V [libjvm.so+0xa19bd7] JNICreateJavaVM+0xc7 C [libjli.so+0xa520] InitializeJVM+0x154 C [libjli.so+0x8024] JavaMain+0xcc

More information about the hotspot-dev mailing list