[Python-Dev] Re: Another test_compiler mystery (original) (raw)

Armin Rigo arigo at tunes.org
Thu Aug 12 22:44:31 CEST 2004

Hello,

On Thu, Aug 12, 2004 at 03:41:37PM -0400, Tim Peters wrote:

Sounds like a good idea not to run out to stack .

I don't remember if it was mentioned here, but maybe we'd better check directly whether the C stack is too large rather than (or possibly in addition to) using a limit on the number of Python iterations.

This is not really ANSI C, but I can't think of a setting in which the following trick would fail to be a good indication of the amount of stack space consumed: save a pointer to a local variable "early", e.g. in Py_Initialize(); then in any other function call, the distance between this pointer and a pointer to some local equals the amount of stack space consumed by Python minus a few bytes.

If this sounds too much of a hack, the (usual) recursion limit could be kept to limit nested Python frames; but all C-level recursions (including eval_frame) could additionally use the above trick. Its advantage is that it is an extremely fast check.

If the total stack size is difficult to know in advance, we can still use PyOS_CheckStack(), but more efficiently and less randomly than now, by maintaining a "high tide" pointer that records how much stack we are sure we have, and calling PyOS_CheckStack() only to attempt to push the "high tide" mark further.

While I'm in a hacking mood, there might be a way to prevent PyErr_Clear() to clear away "asynchronuous" exceptions like RuntimeError, MemoryError, and KeyboardInterrupt: for these exceptions, let's just store them away instead of clearing them, and re-raise them at the next occasion. The "next occasion" would have to be defined more precisely, but there is probably a way to ensure that it will at least be before the next non-trivial opcode operation starts.
It would overwrite an exception set later, like those spurious KeyError we get for dict lookups. It might be a better-than-nothing quick fix to all these PyErr_Clear() all around the code base.

A bientôt,

Armin.

More information about the Python-Dev mailing list