[Python-Dev] pymalloc killer (original) (raw)

Guido van Rossum guido@python.org
Fri, 29 Mar 2002 20:10:53 -0500

[Tim]

This is the kind of heroic effort I don't want to impose on users: you have encyclopedic knowledge of how the Python implementation may be abusing this stuff, and you invented the rules . Random extension authors are going to have a much harder time of it -- as far as they're concerned, PyMem{DEL, FREE, Del, Free} are all just ways to spell "platform free(), but I'm not supposed to call free() directly for some reason I don't understand -- I think it might have had something to do with DLLs on Windows".

On the other hand, the less sophisticated extension writers never release the GIL at all.

[me]

> Given the alternatives: > > 1. introduce new APIs PyMalloc{New,Del}Object and tell all extension > writers that they have to changes their extensions once again to > use these brand new APIs if they want to benefit from pymalloc; or > > 2. fix the issues with PyOSReadline, make PyMem{DEL,FREE,Del,Free} > synonyms for Tim's clever PyMemNukeIt, and continue to promote > using PyObject{New,Del} for use by extension writers; > > I'm all for #2.

You're presenting #1 as user-hostile and #2 as user-friendly. But if part of #2 is also saying that it's now definitely illegal to call PyMem{Free, FREE, Del, DEL} without holding the GIL, and horrible things may start to happen in 2.3 if you're doing so, then it's also user-hostile in that respect. #1 is user-friendly in the "nothing breaks" sense. I haven't given up on combining the best of both, but I am getting close .

All I can do is encourage you to keep trying. I find it quite encouraging that you found a constant-time test for non-arena memory at all. Making it thread-safe should only add a tiny constant time.

--Guido van Rossum (home page: http://www.python.org/~guido/)