[Python-Dev] iterators (original) (raw)

Guido van Rossum guido@beopen.com
Tue, 22 Aug 2000 08:03:28 -0500

> [MAL] > > How about a third variant: > > > > #3: _> > iter = .iterator() _> > while iter: _> > = iter.next() > > > > > > This adds a slot call, but removes the malloc overhead introduced > > by returning a tuple for every iteration (which is likely to be > > a performance problem). > > Are you sure the slot call doesn't cause some malloc overhead as well?

Quite likely not, since the slot in question doesn't generate Python objects (nbnonzero).

Agreed only for built-in objects like lists. For class instances this would be way more expensive, because of the two calls vs. one!

> Ayway, the problem with this one is that it requires a dynamic > iterator (one that generates values on the fly, e.g. something reading _> lines from a pipe) to hold on to the next value between "while iter" _> and "iter.next()".

Hmm, that depends on how you look at it: I was thinking in terms of reading from a file -- feof() is true as soon as the end of file is reached. The same could be done for iterators.

But feof() needs to read an extra character into the buffer if the buffer is empty -- so it needs buffering! That's what I'm trying to avoid.

We might also consider a mixed approach:

#5: _iter = .iterator() _while iter: try: _ = iter.next() except ExhaustedIterator: break Some iterators may want to signal the end of iteration using an exception, others via the truth text prior to calling .next(), e.g. a list iterator can easily implement the truth test variant, while an iterator with complex .next() processing might want to use the exception variant.

Belt and suspenders. What does this buy you over "while 1"?

Another possibility would be using exception class objects as singleton indicators of "end of iteration":

#6: _iter = .iterator() while 1: try: _rc = iter.next() except ExhaustedIterator: break else: if rc is ExhaustedIterator: break = rc

Then I'd prefer to use a single protocol:

#7:
__iter = <object>.iterator()
while 1:
   rc = __iter.next()
   if rc is ExhaustedIterator:
   break
   <variable> = rc
   <block>

This means there's a special value that you can't store in lists though, and that would bite some introspection code (e.g. code listing all internal objects).

--Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)