JEP 248: Make G1 the Default Garbage Collector (original) (raw)

Kirk Pepperdine kirk at kodewerk.com
Thu Apr 30 14:44:45 UTC 2015

Hi Uwe,

I’m currently dealing with a customer trying to use Lucene/Solr/Elasticsearch and I expected that would be a perfect candidate for the G1 but.... I think that other “off-heap” solutions might also suffer. And, as you now know, it takes a serious amount of digging to sort these problems out. I am certain there are very few dev teams with the talent on board to work through the diagnostic process.

Regards, Kirk PS, yes it was indeed nice to meet you @ JAX.

On Apr 30, 2015, at 4:29 PM, Uwe Schindler <uschindler at apache.org> wrote:

Hi Kirk, hi Mark,

the Lucene/Solr/Elasticsearch people still recommend to their users to not use G1GC, although for this type of application (full text search with the requirement for very low response times and no pauses) is a good candidate for G1GC. On the other hand, heap sizes for typical Lucene applications should not be too high, because most of the processing is done on memory mapped files off-heap. So heaps should be at most 1/4 of the physical RAM available, because Lucene relies on the fact that the index files reside in file system cache (too large heaps are contra-productive here). See also our recommendations for Apache Solr and Elasticsearch: http://wiki.apache.org/solr/ShawnHeisey#GCTuningforSolr http://www.elastic.co/guide/en/elasticsearch/guide/current/don8217ttouchthesesettings.html Currently Lucene's indexing sometimes caused serious data corruption with G1GC - leading to data loss, which was mainly caused by some bugs around G1GC and its use of additional memory barriers and the very close interaction with Hotspot, that seemed to broke some optimizations. We had (only in combination with G1GC during our test suites) simple "assert" statements sometimes failing that should never fail unless there is a bug in the JVM. We are aware that Java 8u40 declared G1GC as "production ready", so we are still looking at failures in our extensive testing infrastructure. Indeed, I have no seen any G1GC related problems recently, but that is not necessarily a sign for correctness. Uwe P.S.: It was nice to meet you last week on JAX! ----- Uwe Schindler uschindler at apache.org ASF Member, Apache Lucene PMC / Committer Bremen, Germany http://lucene.apache.org/

-----Original Message----- From: hotspot-dev [mailto:hotspot-dev-bounces at openjdk.java.net] On Behalf Of Kirk Pepperdine Sent: Wednesday, April 29, 2015 9:11 AM To: hotspot-dev at openjdk.java.net Source Developers Subject: Re: JEP 248: Make G1 the Default Garbage Collector

Hi all, Is the G1 ready for this? I see many people moving to G1 but also I’m not sure that we’ve got the tunable correct. I’ve been sorting through a number of recent tuning engagements and my conclusion is that I would like the collector to be aggressive about collecting tenured regions at the beginning of a JVM’s life time but then become less aggressive over time. The reason is the residual waste that I see left behind because certain regions never hit the threshold needed to be included in the CSET. But, on aggregate, the number of regions in this state does start to retain a significant about of dead data. The only way to see the effects is to run regular Full GCs.. which of course you don’t really want to do. However, the problem seems to settle down a wee bit over time which is why I was thinking that being aggressive about what is collected in the early stages of a JVMs life should lead to better packing and hence less waste. Note, I don’t really care about the memory waste, only it’s effect on cycle frequencies and pause times. Sorry but I don’t have anything formal about this as I (and I believe many others) are still sorting out what to make of the G1 in prod. Generally the overall results are good but sometimes it’s not that way up front and how to improve things is sometimes challenging. On a side note, the move to Tiered in 8 has also caused a bit of grief. Metaspace has caused a bit of grief and even parallelStream, which works, has come with some interesting side effect. Everyone has been so enamored with Lambdas (rightfully so) that the other stuff has been completely forgotten and some of it has surprised people. I guess I’ll be submitting a talk for J1 on some of the field experience I’ve had with the other stuff. Regards, Kirk

On Apr 28, 2015, at 11:02 PM, mark.reinhold at oracle.com wrote: New JEP Candidate: http://openjdk.java.net/jeps/248 - Mark

More information about the hotspot-dev mailing list