[Python-Dev] iterzip() (original) (raw)

Raymond Hettinger python@rcn.com
Sun, 28 Apr 2002 23:04:04 -0400

From: "Guido van Rossum" <guido@python.org>

> I'm already working on a separate module for iterators galore > (and will cross-check to Haskell to make sure I didn't miss anything).

+! > I posted this one separately because zip() eats memory like crazy > and because a Python generator version crawls like a snail. Do you have use cases where the memory use matters? I.e. where it needs more memory than you have RAM?

No. Am I'm not proposing a backport to Pippy ;)

> IMHO, This is a better way to loop over multiple sequences and > has a chance at becoming the tool of choice. I scanned all of my > Python code and found that iterzip() was a better choice in every > case except a matrix transpose coded as zip(*mat). Did you time any of these?

I just timed it and am shocked. iterzip() has exactly the same code as zip() except for the final append result to list. So, I expected only a microscopic speed-up. I don't see where the huge performance improvement came from.

First timing:

19.439999938 zip 1.43000006676 iterzip 30.1000000238 genzip

Second timing:

20.0999999046 zip 1.60000002384 iterzip 29.0599999428 genzip

Timing code:

time iterzip

from future import generators import time

def run(f, iterables): start = time.time() for tup in f(*iterables): pass return time.time() - start

def genzip(*iterables): iterables = map(iter, iterables) while 1: yield tuple([i.next() for i in iterables])

n = 1000000 for f in [zip, iterzip, genzip]: print run(f, [xrange(n), xrange(n), xrange(n)]), f.name

Raymond Hettinger