[Python-Dev] Using PEP384 Stable ABI for the lzma extension module (original) (raw)

Charles-François Natali neologix at free.fr
Thu Oct 6 10:09:34 CEST 2011

> > > That's not a given. Depending on the memory allocator, a copy can be > > > avoided. That's why the "str += str" hack is much more efficient under > > > Linux than Windows, AFAIK. > > > > Even Linux will have to copy a block on realloc in certain cases, no? > > Probably so. How often is totally unknown to me :) > http://www.gnu.org/software/libc/manual/htmlnode/Changing-Block-Size.html

It depends on whether there's enough free memory after the buffer you currently have allocated.  I suppose that this becomes a question of what people consider "the general case" :-) But under certain circumstances (if a large block is requested), the allocator uses mmap(), no?

That's right, if the block requested is bigger than mmap_threshold (256K by default with glibc, forgetting the sliding window algorithm): I'm not sure of what percentage of strings/buffers are concerned in a "typical" program.

In which case mremap() should allow to resize without copying anything.

Yes, there's no copying. Note however that it doesn't come for free, the kernel will still zero-fill the pages before handling them to user-space. It is still way faster than on, let's say, Solaris.

cf

More information about the Python-Dev mailing list