[Python-Dev] Using PEP384 Stable ABI for the lzma extension module (original) (raw)

Antoine Pitrou solipsis at pitrou.net
Wed Oct 5 19:02:58 CEST 2011

On Wed, 5 Oct 2011 09:38:10 -0700 Toshio Kuratomi <a.badger at gmail.com> wrote:

On Wed, Oct 05, 2011 at 06:14:08PM +0200, Antoine Pitrou wrote: > Le mercredi 05 octobre 2011 à 18:12 +0200, "Martin v. Löwis" a écrit : > > >> Not sure what you are using it for. If you need to extend the buffer > > >> in case it is too small, there is absolutely no way this could work > > >> without copies in the general case because of how computers use > > >> address space. Even PyBytesResize will copy the data. > > > > > > That's not a given. Depending on the memory allocator, a copy can be > > > avoided. That's why the "str += str" hack is much more efficient under > > > Linux than Windows, AFAIK. > > > > Even Linux will have to copy a block on realloc in certain cases, no? > > Probably so. How often is totally unknown to me :) > http://www.gnu.org/software/libc/manual/htmlnode/Changing-Block-Size.html

It depends on whether there's enough free memory after the buffer you currently have allocated. I suppose that this becomes a question of what people consider "the general case" :-)

But under certain circumstances (if a large block is requested), the allocator uses mmap(), no? In which case mremap() should allow to resize without copying anything.

Regards

Antoine.

More information about the Python-Dev mailing list