[Python-Dev] RE: Painful death in debug build (original) (raw)

Tim Peters tim.one@home.com
Sat, 6 Oct 2001 01:02:09 -0400

[Neil Schemenauer]

Using electric fence requires huge amounts of (virtual) memory. Every allocation requires two pages of memory.

Yikes! [NealS]

I doubt running the entire test suite would ever work. I'm having trouble running just testdescr alone with all the tests commented out!

FWIW, I can provoke the bug by running test_descr.inherits() alone:

C:\Code\python\PCbuild>python_d Adding parser accelerators ... Done. Python 2.2a4+ (#24, Oct 5 2001, 18:16:47) [MSC 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information.

from test import testdescr [21862 refs] testdescr.inherits() Testing inheritance from basic types...

And then it blows up.

I think I'm running into MAXMAPCOUNT (which defaults to 64k).

I've just grabbed Linux 2.4.10-ac7 which add an AVL tree for VMAs and a dynamically configurable maxmapcount. We'll see how that goes.

You may as well rewrite the kernel while you're at it .

Someone should really write a portable debug version of malloc and free for Python that works like the Microsoft implementation. It can't be too hard to lay down poison before and after the hunk of memory on allocation and then check it on deallocation.

It's extremely useful for establishing the presence of off-by-one stores, but this is the first time I've seen this particular check trigger in Python. Another trick it plays is to fill free()'ed memory with a different unlikely byte pattern, then delay recycling the memory. When things blow up this "dead landfill" value is amazingly often present (indicating that someone is still trying to read from free'd memory). The third trick is that newly malloc'ed memory is filled with a third "clean landfill" byte, which helps catch use of uninitialized memory. All the special byte values are picked so that they're unlikely to yield valid addresses, or integers. In bang for the buck, these tricks are huge wins.

if-only-they-pointed-to-the-true-causes-too-ly y'rs - tim