[Python-Dev] stack check on Unix: any suggestions? (original) (raw)

M.-A. Lemburg mal@lemburg.com
Thu, 31 Aug 2000 16:44:09 +0200

Jack Jansen wrote:

I'm confused now: how is this counting-stack-limit different from the maximum recursion depth we already have? The whole point of PyOSStackCheck is to do an actual check of whether there's space left for the stack so we can hopefully have an orderly cleanup before we hit the hard limit. If computing it is too difficult because getrlimit isn't available or doesn't do what we want we should probe it, as the windows code does or my example code posted yesterday does. Note that the testing only has to be done every first time the stack goes past a certain boundary: the probing can remember the deepest currently known valid stack location, and everything that is shallower is okay from that point on (making PyOSStackCheck a subroutine call and a compare in the normal case).

getrlimit() will not always work: in case there is no limit imposed on the stack, it will return huge numbers (e.g. 2GB) which wouldn't make any valid assumption possible.

Note that you can't probe for this since you can not be sure whether the OS overcommits memory or not. Linux does this heavily and I haven't yet even found out why my small C program happily consumes 20MB of memory without segfault at recursion level 60000 while Python already segfaults at recursion level 9xxx with a memory footprint of around 5MB.

So, at least for Linux, the only safe way seems to make the limit a user option and to set a reasonably low default.

-- Marc-Andre Lemburg

Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/