[Python-Dev] pymalloc killer (original) (raw)

Kevin Jacobs jacobs@penguin.theopalgroup.com
Fri, 29 Mar 2002 06:57:54 -0500 (EST)

Some wild-ass thoughts:

On Fri, 29 Mar 2002, Tim Peters wrote:

A nasty subtlety: if PyMemNukeIt is called when the GIL isn't held, it's going to take a lot of thought to see whether this can be made safe when called simultaneously by multiple threads; e.g., the finger can change "mid-stream"

Then make multiple thread-specific fingers, which will presumably result in higher hit rates due to better locality. To prevent fingers from being invalidated, do not remove arenas that are deallocated from the tree -- just mark them inactive.

then, and, worse, some thread that does hold the GIL may legitimately cause a new arena to be allocated midstream, mutating the address vector during the search. This may be intractable enough to kill the idea of mapping PyMemXXX to this.

Why not use a smaller (non-global) lock to protect arena modification and finger flushing? Ideally, the lock should be cheap for readers (which I presume will be the vast majority of accesses) and relatively expensive for writers (which I assume will occur only when arenas are added or deactivated).

-Kevin

-- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (216) 986-0710 x 19 E-mail: jacobs@theopalgroup.com Fax: (216) 986-0714 WWW: http://www.theopalgroup.com