Review request for 5049299 (original) (raw)

Florian Weimer fweimer at bfk.de
Thu May 28 09:10:18 UTC 2009

But those running Linux won't benefit from such a change because on Linux there is no transient doubling of process size: all that happens is that the page table entries in the new process are mapped copy on write. The extra pages count towards the overcommit limit, but that's wholly artifical.

On Linux in vm.overcommit_memory=2 mode, the whole heap (not just the committed part) counts against the system's total memory allocation limit. After a fork, the heap counts twice. Copy-on-write is just an optimization in this mode, it does not change the physical memory requirements of the workload (in vm.overcommit_memory=1 mode, it does).

I guess the issue might even be worse on Linux than on Solaris because the heap initialization code in Hotspot is wrong. It uses MAP_NORESERVE, which means that the VM will die with SIGSEGV when it tries to write to a page for which the kernel cannot provide backing storage. This SIGSEGV is not turned into an OutOfMemoryError, leading to hard-to-diagnose bug reports. (It's not an OOM situation from the kernel's point of view, so you don't get a trace from the OOM killer in the system log, either.)

A better way seems to be to allocate the heap with PROT_NONE, and later use mprotect with PROT_READ|PROT_WRITE (and perhaps PROT_EXEC) to allocate chunks from the kernel. This will fail deterministically in the garbage collector if no physical memory is available. The PROT_NONE mapping is only there to reserve a continuous chunk of address space (so that calls to malloc or dlopen do not create mappings in the middle of the Java heap). When I tested this some time ago, a PROT_NONE mapping did not count towards the system's memory allocation limit, hence the potential failure in the mprotect call. The main problem with this approach is that this is not a documented way of using the kernel API; it might work accidentally now and change behavior in the future.

Sorry for hijacking this thread, but I thought this clarification was helpful.

-- Florian Weimer <fweimer at bfk.de> BFK edv-consulting GmbH http://www.bfk.de/ Kriegsstraße 100 tel: +49-721-96201-1 D-76133 Karlsruhe fax: +49-721-96201-99

More information about the core-libs-dev mailing list