RFR (L): 8003868: fix shark for latest HotSpot and LLVM [Was: Re: RFR: Fix shark for latest Hotspot and LLVM] (original) (raw)

Roman Kennke rkennke at redhat.com
Thu Nov 22 08:48:10 PST 2012

Am Mittwoch, den 21.11.2012, 14:52 -0800 schrieb Christian Thalinger:

On Nov 21, 2012, at 2:22 PM, Christian Thalinger <christian.thalinger at oracle.com> wrote:

> > On Nov 21, 2012, at 2:17 PM, Christian Thalinger <christian.thalinger at oracle.com> wrote: > >> >> On Nov 21, 2012, at 12:54 PM, Roman Kennke <rkennke at redhat.com> wrote: >> >>> Am Mittwoch, den 21.11.2012, 12:47 -0800 schrieb Christian Thalinger: >>>> On Nov 21, 2012, at 11:43 AM, Christian Thalinger <christian.thalinger at oracle.com> wrote: >>>> >>>>> On Nov 21, 2012, at 9:31 AM, Roman Kennke <rkennke at redhat.com> wrote: >>>>> >>>>>> Hi there, >>>>>> >>>>>> during the last days I worked on fixing the Shark compiler for Hotspot >>>>>> to get it to build and run again, with the latest Hotspot code and LLVM. >>>>>> Here are some details: >>>>>> >>>>>> - A lot of changes are just to make it build and the compiler happy. For >>>>>> example, I had to remove a lot of 'const' qualifiers because of API >>>>>> changes in LLVM. >>>>>> - Most other changes have to do with the split of the oop and metadata >>>>>> class hierarchies in Hotspot. >>>>>> - Then there have been a few changes caused by LLVM changes and >>>>>> improvements, most notably the LLVM intrinsics for atomic operations >>>>>> (memory barrier and cmpxchg) have been removed and now have a >>>>>> representation directly in LLVM's IR. This makes our code a little >>>>>> nicer. >>>>>> >>>>>> I tested this by running a number of applications, most notably Eclipse >>>>>> (which is notoriously difficult on VMs), Java2Demo, SwingSet2 and a >>>>>> bunch of other stuff. >>>>>> >>>>>> I would like to get this integrated into OpenJDK now if possible. You >>>>>> can find the full webrev here: >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev.00/ >>>>> >>>>> The changes seem to touch almost only shark files so these should be fine. One question though: >>>>> _>>>>> + develop(bool, SharkShowCompiledMethods, false, _ >>>>> >>>>> Isn't PrintCompilation doing that already? >>>>> >>>>> The shared code changes look good. I filed: >>>>> >>>>> 8003868: fix shark for latest HotSpot and LLVM >>>>> >>>>> -- Chris >>>>> >>>>>> >>>>>> There are also a very minor change required in JDK: >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/shark/webrev-jdk-00/ >>>>>> >>>>>> In order to build it, apply the patches on hsx/hotspot-comp 's hotspot >>>>>> and jdk repositories respectivly. Find my build script here: >>>>>> >>>>>> http://cr.openjdk.java.net/~rkennke/shark/Build8-zero-shark >>>>>> >>>>>> (Review and adjust variables to your settings, most notably you will >>>>>> need to change LLVMCONFIG to point to your LLVM 3.1 installation.) >>>>>> >>>>>> Please let me know if there are any issues or how we can get this >>>>>> integrated into Hotspot. >>>> >>>> Finally I installed LLVM on one of our machines to be able to do a Shark build once in a while. When I try to do a jvmgshark build I get: >>>> >>>> In file included from /usr/local/include/llvm/Support/PointerLikeTypeTraits.h:18, >>>> from /usr/local/include/llvm/ADT/PointerIntPair.h:17, >>>> from /usr/local/include/llvm/Use.h:28, >>>> from /usr/local/include/llvm/Value.h:17, >>>> from /usr/local/include/llvm/Argument.h:17, >>>> from /home/cthaling/8003868/src/share/vm/shark/llvmHeaders.hpp:39, >>>> from /home/cthaling/8003868/src/share/vm/shark/sharkEntry.hpp:29, >>>> from /home/cthaling/8003868/src/share/vm/compiler/disassembler.cpp:51: _>>>> /usr/local/include/llvm/Support/DataTypes.h:53:3: error: #error "Must #define STDCCONSTANTMACROS before " "#including Support/DataTypes.h" >>>> >>>> and: >>>> >>>> In file included from /usr/local/include/llvm/Attributes.h:18, >>>> from /usr/local/include/llvm/Argument.h:18, >>>> from /home/cthaling/8003868/src/share/vm/shark/llvmHeaders.hpp:39, >>>> from /home/cthaling/8003868/src/share/vm/shark/sharkEntry.hpp:29, >>>> from /home/cthaling/8003868/src/share/vm/compiler/disassembler.cpp:51: >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isInt(int64t)’: >>>> /usr/local/include/llvm/Support/MathExtras.h:38: error: there are no arguments to ‘INT64C’ that depend on a template parameter, so a declaration of ‘INT64C’ must be available >>>> /usr/local/include/llvm/Support/MathExtras.h:38: note: (if you use ‘-fpermissive’, G++ will accept your code, but allowing the use of an undeclared name is deprecated) >>>> /usr/local/include/llvm/Support/MathExtras.h:38: error: there are no arguments to ‘INT64C’ that depend on a template parameter, so a declaration of ‘INT64C’ must be available >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isUInt(uint64t)’: >>>> /usr/local/include/llvm/Support/MathExtras.h:64: error: there are no arguments to ‘UINT64C’ that depend on a template parameter, so a declaration of ‘UINT64C’ must be available >>>> /usr/local/include/llvm/Support/MathExtras.h: In function ‘bool llvm::isIntN(unsigned int, int64t)’: >>>> /usr/local/include/llvm/Support/MathExtras.h:96: error: ‘INT64C’ was not declared in this scope >>>> >>>> Not sure if the latter is because of the former one. Have you seen this before? >>> >>> Yes, it's caused by the former. And yes, I have seen it before. IIRC, >>> this happens when certain cflags are not set correctly. The script >>> jdk/make/jdkgenericprofile.sh will call llvm-config to figure out the >>> correct flags. In order for this to work, you need to have the full path >>> to the llvm-config script set in the LLVMCONFIG env variable. Were you >>> using the build script that I provided? >> >> No. I took your script and got the important environment variables. But I missed the LLVM* ones. Usually we only build hotspot so we don't have this jdk script. >> >> Now that I have the LLVM* variables it's even worse: >> >> /home/cthaling/8003868/src/share/vm/oops/oop.hpp:72: error: cast from type ‘markOopDesc* const volatile*’ to type ‘markOopDesc**’ casts away constness >> >> It's probably this guy: -Wcast-qual > > Got it: > > $ java -version > java version "1.8.0-ea" > Java(TM) SE Runtime Environment (build 1.8.0-ea-b65) > OpenJDK 64-Bit Shark VM (build 25.0-b11-internal-jvmg, mixed mode) I ran the compiler regression tests and Shark crashes in 5091921: cthaling at intelsdv03.us.oracle.com:~/8003868/test$ jtreg -workDir:$EXPORTHOME/jtreg -reportDir:$EXPORTHOME/jtreg -testjdk:$JAVAHOME -verbose:summary compiler/5091921/ Directory "/export/twisti/jtreg/scratch" not found: creating Passed: compiler/5091921/Test5091921.java Passed: compiler/5091921/Test6186134.java Passed: compiler/5091921/Test6196102.java Passed: compiler/5091921/Test6357214.java Passed: compiler/5091921/Test6559156.java Passed: compiler/5091921/Test6753639.java Passed: compiler/5091921/Test6850611.java Passed: compiler/5091921/Test6890943.java Passed: compiler/5091921/Test6897150.java Passed: compiler/5091921/Test6905845.java Passed: compiler/5091921/Test6931567.java /net/sqenfs-1.us.oracle.com/export1/comp/vm/tool/jtreg/execution/linux/bin/jtreg: line 139: 27784 Segmentation fault (core dumped) "${JTJAVA}" javaOpts−Dprogram=‘basename"javaOpts -Dprogram=basename &quot;</annotation></semantics></math></span><span class="katex-html" aria-hidden="true"><span class="base"><span class="strut" style="height:0.8778em;vertical-align:-0.1944em;"></span><span class="mord mathnormal">ja</span><span class="mord mathnormal" style="margin-right:0.03588em;">v</span><span class="mord mathnormal">a</span><span class="mord mathnormal">Opt</span><span class="mord mathnormal">s</span><span class="mspace" style="margin-right:0.2222em;"></span><span class="mbin">−</span><span class="mspace" style="margin-right:0.2222em;"></span></span><span class="base"><span class="strut" style="height:0.8778em;vertical-align:-0.1944em;"></span><span class="mord mathnormal" style="margin-right:0.02778em;">D</span><span class="mord mathnormal">p</span><span class="mord mathnormal">ro</span><span class="mord mathnormal" style="margin-right:0.03588em;">g</span><span class="mord mathnormal" style="margin-right:0.02778em;">r</span><span class="mord mathnormal">am</span><span class="mspace" style="margin-right:0.2778em;"></span><span class="mrel">=</span><span class="mspace" style="margin-right:0.2778em;"></span></span><span class="base"><span class="strut" style="height:0.6944em;"></span><span class="mord">‘</span><span class="mord mathnormal">ba</span><span class="mord mathnormal">se</span><span class="mord mathnormal">nam</span><span class="mord mathnormal">e</span><span class="mord">&quot;</span></span></span></span>0" -jar "${JTHOME}/lib/jtreg.jar" $jtregOpts You can also run all them with a simple make in test/ by setting: PRODUCTHOME=$JAVAHOME TESTDIRS=compiler

This is interesting. I am not getting segfaults, but trip over assertions inside LLVM (probably the same thingy). I tried looking (and running through verifier) at the LLVM IR code, but nothing suspicious. Then I turned off optimizations in LLVM and suddenly everything passed (except for some tests that time out... probably performance tests that time out because shark is too slow?). It could be a bug in LLVM, I suspect this:

http://llvm.org/bugs/show_bug.cgi?id=12470

I am not sure though. I added a -XX:SharkOptimizationLevel switch to select an optimization level at runtime. I will also check out the LLVM-3.2 RC code and see if that's any better. I will keep you posted.

Roman

More information about the hotspot-dev mailing list