[Python-Dev] Darwin's realloc(...) implementation never shrinks allocations (original) (raw)

bacchusrx bacchusrx at skorga.org
Mon Jan 3 22:52:25 CET 2005

On Mon, Jan 03, 2005 at 03:55:19PM -0500, Bob Ippolito wrote:

>Note that, with respect to http://python.org/sf/1092502, the author >of the (original) program was using the documented interface to a >file object. It's fileobject.read() that decides to ask for huge >numbers of bytes from recv() (specifically, in the >max(self.rbufsize, left) condition). Patched to use a fixed >recvsize, you of course sidestep the realloc() nastiness in this >particular case.

While using a reasonably sized recvsize is a good idea, using a smaller request size simply means that it's less likely that the strings will be significantly resized. It is still highly likely they will be resized and that doesn't solve the problem that over-allocated strings will persist until the entire request is fulfilled.

You're right. I should have said, "you're more likely to get away with it." The underlying issue still exists. My point is that the problem is not analogous to the guy who tried to read 2GB directly from a socket (as in http://python.org/sf/756104).

Googling for MemoryError exceptions, you can find a number of spurious problems on Darwin that are probably due to this bug: SpamBayes for instance, or the thread at

http://mail.python.org/pipermail/python-list/2004-November/250625.html

bacchusrx.

More information about the Python-Dev mailing list