ByteBuffer performance issue in Java 9? (original) (raw)
Uwe Schindler uschindler at apache.org
Fri Mar 10 18🔞38 UTC 2017
- Previous message: JDK 9 rampdown and a plea for mercy
- Next message: ByteBuffer performance issue in Java 9?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi,
we just noticed this issue: https://bugs.openjdk.java.net/browse/JDK-8176513
As Apache Lucene relies heavily on performance of ByteBuffers (especially MappedByteBuffer), this would be a desaster if it would get even slower than Java 8. We were so happy that there was much work going on to improve the performance of ByteBuffers so they were at some point almost as fast as byte[]. Because of that we are currently working on making Lucene work fine with Java 9, because once it is out, we would inform all users and recommend to use Java 9 because of these optimizations. The tests of Lucene are already running perfectly, no problems like data corrumption (remember Java 7 GA or 7u40). Also Apache Solr people try to fix the remaining Hadoop/Kerberos-Auth issues because of Jigsaw. We also have unmapping working using Unsafe::invokeCleaner, so we are prepared...
I am not yet sure if this bug affects us, we have to run perf tests first. Nevertheless, this is our usage pattern:
- we only read from ByteBuffers (MappedByteBuffer), never store anything (we use mmap to execute Lucene searches on the mapped index with often 10th sometimes also 100th of gigabytes of data files that were mmapped).
- we do sequential reads (position() and then getByte(), getXxX)
- we also do positional reads like this test (for the so called DocValues in Lucene, which is a column based store). This is mainly like sorting search results based on a compare function that accesses a ByteBuffer as random-access source for comparison values.
- we don't use IntBuffer, we work directly on ByteBuffer
I asked Mike McCandless to run the performance tests with a recent Java 9 version to see if it affects us. Is there any information, when this bug was introduced into Java 9? Nevertheless, please fix it for Java 9 GA, if it affects us it would be a major loss of search performance for us!
It would be good to get some information about your plans :-) Thanks!
Uwe
Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/
- Previous message: JDK 9 rampdown and a plea for mercy
- Next message: ByteBuffer performance issue in Java 9?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]