Question regarding JEP 8204089 Timely Reducing Unused Committed Memory (original) (raw)
Thomas Schatzl thomas.schatzl at oracle.com
Mon Jun 4 12:26:21 UTC 2018
- Previous message (by thread): Question regarding JEP 8204089 Timely Reducing Unused Committed Memory
- Next message (by thread): Question regarding JEP 8204089 Timely Reducing Unused Committed Memory
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Hi all,
On Fri, 2018-06-01 at 16:44 +0300, Ruslan Synytsky wrote:
Hi Kirk, thank you for the additional important highlights.
On 1 June 2018 at 08:21, Kirk Pepperdine <kirk.pepperdine at gmail.com> wrote: > Hi,
[...]
> In an era when the GC engineers are working so very hard to make as > much of the collection process as concurrent as possible, it just > feels very wrong to rely on a huge STW event to get something done. > In that spirit, it behoves us to explore how committed memory maybe > released at the tail end of a normally triggered GC cycle. I > beleive at the end of a mixed cycle was mentioned. I believe this > would give those that want/need to minimize their JVM’s footprint > an even greater cost advantage then attempting to reduce the > footprint at some (un???)random point in time. >
How hard/expensive is to implement in G1 something similar like Shenandoah does?
Let me detail a bit how G1 triggers collections and when it releases memory:
For the first part, let's make a distinction between the trigger for young collections and the start of old generation space reclamation ("marking").
Young collections are at the moment triggered by space exhaustion in the young generation.
Old gen reclamation is currently only triggered by old generation space going over a given occupancy threshold. This can be either caused by young collections copying objects into the old generation ("promotion") or humongous (large) object allocation.
In both cases, G1 triggers a so-called initial-mark young collection, i.e. a regular young collection with some additional setup for the concurrent marking. Concurrent marking results in Remark and Cleanup pauses that are scheduled on a time basis. During that time, when the young generation is exhausted, regular young collections are performed.
After the Cleanup pause, there is some "last" regular young generation pause where GC and marking kind of sync up. After that one, again caused by exhaustion of the young gen, so called mixed collections are triggered. These are young generation collections with minimal young gen size and some old generation regions added.
These mixed collections continue (mostly) until G1 thinks enough memory has been freed.
Basically, to release memory back to the operating system, the programming effort would be to call G1CollectedHeap::shrink() at the end of the appropriate pause.
If you were looking into effort, there are two caveats (compared to simply triggering a full gc):
full gc currently, in addition to compaction, also throws away SoftReferences (cached objects), and cleans out some references from VM internal data structures to the Java heap, making all of these freeable. Assuming you want to have similar impact regarding soft references and internal data structures, you need to start a concurrent cycle, and wait until the Remark pause with your shrinking request (one can still try without!). Only the following mixed GCs compact the Java heap (at the moment G1 will also not start mixed GCs without a previous marking cycle); but the Remark pause will free completely empty regions.
if you also want to compact old gen, the best time for the shrink() call would probably be the end of the mixed gc phase. However garbage collections are currently tied to exhaustion of young gen. So if your application is truly or almost idle, and does not allocate anything, there will be no GC pauses for potentially a long time.
One would need to start the mixed gc based on some timout. Note that there are already some timers running for some VM cleanup tasks.
To improve the impact of the compaction and shrinking you might also want to tweak some internal variables for the duration of this "idle cycle".
Btw, one step in that direction would be to generally just attempt to shrink the heap at the end of mixed gc, e.g. https://bugs.openjdk.java. net/browse/JDK-6490394 .
Implementing something like Shenandoah, i.e. uncommit the regions that stayed empty for more than ShenandoahUncommitDelay milliseconds would mean iterating through all regions and check if they are "old" enough to uncommit (and do so) at regular intervals in some STW pause. When allocating, also give them some sort of timestamp. Again, you need to make sure that the regions are looked through in some regular interval (either piggy-backing by some existing regular pauses or force one).
This would not be particularly hard to implement either, but only seems extra work: as you might still want to compact the heap in some way (this is optional) by e.g. doing a marking + mixed cycle (and then wait for the ShenandoahUncommitDelay), so you may as well uncommit within these pauses already.
Some other unanswered question so far has been up to what degree memory will be freed during this time: I guess at most until -Xms?
Thanks, Thomas
- Previous message (by thread): Question regarding JEP 8204089 Timely Reducing Unused Committed Memory
- Next message (by thread): Question regarding JEP 8204089 Timely Reducing Unused Committed Memory
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]