[Python-Dev] Re: Another test_compiler mystery (original) (raw)

Tim Peters tim.peters at gmail.com
Thu Aug 12 08:25:21 CEST 2004

Here's some puzzling evidence (WinXP Pro SP1). The test driver, for convenience:

""" import compiler, sys

f = open('../Lib/test/test_parser.py') guts = f.read() f.close()

def ouch(n): if n == 0: return compiler.compile(guts, "", "exec") else: return ouch(n-1)

for n in range(0, 50): try: ouch(n) msg = 'ok' except KeyboardInterrupt: raise except Exception, msg: msg = str(sys.exc_info()[0]) + ' ' + str(msg) print n, msg """

The loop starts at 0 this time, and I'm only looking at a debug-build Python.

The stack size for an .exe can be specified to 4-byte granularity. Here's the largest stack size at which that produces no output at all:

C:\Code\python\PCbuild>editbin /stack:1310720 python_d.exe Microsoft (R) COFF Binary File Editor Version 6.00.8447 Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

C:\Code\python\PCbuild>python_d temp.py

C:\Code\python\PCbuild>echo %ERRORLEVEL% 128

Add 4 measly bytes, and it's a world of difference:

C:\Code\python\PCbuild>editbin /stack:1310724 python_d.exe Microsoft (R) COFF Binary File Editor Version 6.00.8447 Copyright (C) Microsoft Corp 1992-1998. All rights reserved.

C:\Code\python\PCbuild>python_d temp.py 0 ok 1 ok ... 41 ok 42 ok

C:\Code\python\PCbuild>echo %ERRORLEVEL% 128

So it still vanishes early, but gets thru the entire compiler.compile() business + 42 additional stacked Python calls! That's awfully impressive for four bytes. That suggests "the problem" isn't in detecting Python-level recursion.

Under the debugger, it dies with an access violation at that point. Alas, the C-level Python call stack is no longer anywhere in evidence then. Instead it's 7 levels deep in ntdll.dll, and is "in the middle" of four consecutive Pentium PUSH instructions. That's evidence that the stack has been blown .

The debugger Output window does show ntdll.dll suffering a Stack Overflow exception. It then shows a variety of Access Violations in ntdll.dll trying to read and write various locations, presumably in a failing attempt to report the stack overflow.

More information about the Python-Dev mailing list