Review request for 5049299 (original) (raw)

Michael McMahon Michael.McMahon at Sun.COM
Tue May 26 11:10:34 UTC 2009

Martin Buchholz wrote:

Below is a java program and a shell extract demonstrating the problem on a Ubuntu-hardy-like system. The fork() fails from within a large Java process (but normal commands started from an independent shell continue to work fine).

Linux overcommit heuristics are hard to understand. The failing fork problem becomes more serious on a server system running only one significant large program, which is using most of the memory on the system, perhaps by sysctl'ing the vm.overcommitratio to a value higher than the default 50%. On systems with two large processes using most of memory, you may see rare failures when both try to fork() at the same time.. I played around with your test and I see similar behaviour on my Ubuntu 8.04 system. When you switch on "always overcommit" (not the default) then you get the expected behaviour, ie. you can go quite close to the VM limit, and manage to spawn a small command from Java.

However, in the default "heuristic" mode, it doesn't work. The comment on the web page you mentioned says:

"Heuristic overcommit handling. Obvious overcommits of address space are refused. Used for a typical system. It ensures a seriously wild allocation fails while allowing overcommit to reduce swap usage:

Perhaps, this specifically does not suit the Java VM situation where fork() of a very large process, is considered an "obvious overcommit".

In that context, I'll look into using the clone() system call on Linux. It seems to be quite versatile and configurable. It looks likely that we can avoid the use of the helper command with it as well.

More information about the core-libs-dev mailing list