[Python-Dev] Do we need length_hint at all? (Was PEP 0424: A method for exposing a length hint) (original) (raw)

Maciej Fijalkowski fijall at gmail.com
Mon Jul 16 11:14:54 CEST 2012

On Mon, Jul 16, 2012 at 11:02 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

Mark Shannon, 16.07.2012 10:37: > If resizing of lists is too slow, then we should reconsider the 9/8 factor > and/or look to tweak the resizing code.

The thing is that the performance is platform specific. On systems with a fast memory allocator, especially on Linux, the difference is negligible. However, with a slow memory allocator, especially a slow realloc(), e.g. on Windows or Solaris, this can substantially hurt the performance, up to a quadratically increasing runtime in the worst case. The length hint was implemented specifically to work around this problem. Stefan It's not the actual allocation (typically), it's the copying that's the problem.

As far as data goes - preallocation can make 4x difference (on PyPy, although the dominant operation is not different on CPython) on ''.join(some-iterable). It depends grossly on the sizes of the list, so you can't claim that there is a precise speedup of a constant factor, however, there are cases where it can be significant (as in the copying is by far the dominating operation), most notable giant templates with iterators written in C.

Speaking of which - I find this bikeshed disgusting. The purpose of the PEP is to codify whatever is already written in code in CPython. If you guys don't want it, we'll just implement it anyway and try to follow the CPython current implementation from 2.7.

Cheers, fijal -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20120716/cf774556/attachment.html>

More information about the Python-Dev mailing list