[Python-Dev] FreeBSD 7 amd64 and large memory tests (original) (raw)

James Y Knight foom at fuhm.net
Wed Sep 17 17:00:41 CEST 2008

On Sep 17, 2008, at 10:53 AM, Andrew MacIntyre wrote:

Martin v. Löwis wrote:

I haven't yet tried posting a query to a FreeBSD list, as it could simply be a bug on amd64, but I was wondering whether there was anything (other than deactivating tests and documenting use of ulimit -v on this platform) that could be done to work around this behaviour. I think it should be possible to debug this (for people with access to such a system, and appropriate skills). I find it hard to believe that a large malloc will simply crash the process, rather than returning NULL. More likely, there is a NULL returned somewhere, and Python (or libc) fails to check for it. A simple C program doing a repetitive malloc(), much as pymalloc would with continuous demand, does indeed not see any NULL from malloc() when swap is exhausted but the process gets KILLed (the allocated memory does have to be written to to force the situation...) I'll take this up with FreeBSD folk, but I'm open to ideas as to how best to deal with the problem in the context of the test suite pending resolution by FreeBSD.

Linux does the same thing, unless the user has explicitly configured
that behavior off. Search the web for linux overcommit. It's
controlled by the vm.overcommit_memory sysctl. Although linux's
default is some heuristic that might make Python's test case work
right in most cases, depending on malloc returning NULL is not
something you can actually depend on.

James

More information about the Python-Dev mailing list