[Python-Dev] pymalloc killer (original) (raw)

Tim Peters tim.one@comcast.net
Sat, 30 Mar 2002 15:01:05 -0500

[Guido]

How about if the PyMemFree guy saved the address of the vector before using it, and checked that it was still the same afterwards, and if the PyMemMalloc guy didn't use realloc to resize the vector but copied it to a newly malloc'ed vector, stored the new vector's address, and then freed the old vector?

It already does the latter (it deliberately avoided realloc all along -- an even stronger reason to avoid realloc is that if the realloc failed, we'd lose the original vector of base addresses entirely and so would have to kill the program; as is, it can continue gracefully).

(I guess this is the same as what Martin suggested.

He suggested letting the old vector leak, which is the same idea God gave me in my sleep .

His point that the global holding the vector address should be declared volatile is a good one.)

I see no point to that. If someone can explain why it's a good idea, sure, but I don't see it.

Let's simplify things enormously: p is a pointer to "something". Thread A contains

*p

Thread B contains

free(p)

Without a lock, they're screwed period. Thread A has to first read up p, and then dereference it, and all the volatiles in the universe can't stop free(p) from happening between thread A's two machine instructions. It doesn't matter either if Thread B does

oldp = p;
p = newp;
free(oldp);

because thread A can still pick up "p" before the "p = newp" line. Indeed, the current obmalloc code is exactly like that, and it's not safe (although I bet if I left it alone, the hole wouldn't be hit in my lifetime!).

Neither would it do any real good for Thread A to do

if (p == p && p == p && p == p && wink)
    ... *p ...

any number of times, volatile or not. Nothing short of a pair of critical sections is a genuine cure for a genuine race.

Letting the old vector leak is fine; in most programs, it won't lose more than about 1KB of memory total. Note that there are obscure reasons for why it's OK to reference a stale vector in the cases where a race is possible (this is already explained in obmalloc.c comments, so I won't repeat that here).