[Python-Dev] iterators (original) (raw)

M.-A. Lemburg mal@lemburg.com
Tue, 22 Aug 2000 09:58:12 +0200

Guido van Rossum wrote:

[BDFL] > > The statement > > > > for in : > > > > should translate into this kind of pseudo-code: > > > > # variant 1 _> > temp = .newiterator() > > while 1: _> > try: = temp.next() > > except ExhaustedIterator: break > > > > > > or perhaps (to avoid the relatively expensive exception handling): > > > > # variant 2 _> > temp = .newiterator() > > while 1: _> > _flag, <variable = _temp.next()_ _> > if not flag: break > > [MAL] > How about a third variant: > > #3: _> iter = .iterator() _> while iter: _> = iter.next() > > > This adds a slot call, but removes the malloc overhead introduced > by returning a tuple for every iteration (which is likely to be > a performance problem). Are you sure the slot call doesn't cause some malloc overhead as well?

Quite likely not, since the slot in question doesn't generate Python objects (nb_nonzero).

Ayway, the problem with this one is that it requires a dynamic iterator (one that generates values on the fly, e.g. something reading _lines from a pipe) to hold on to the next value between "while iter" _and "iter.next()".

Hmm, that depends on how you look at it: I was thinking in terms of reading from a file -- feof() is true as soon as the end of file is reached. The same could be done for iterators.

We might also consider a mixed approach:

#5: __iter = .iterator() while __iter: try: = __iter.next() except ExhaustedIterator: break

Some iterators may want to signal the end of iteration using an exception, others via the truth text prior to calling .next(), e.g. a list iterator can easily implement the truth test variant, while an iterator with complex .next() processing might want to use the exception variant.

Another possibility would be using exception class objects as singleton indicators of "end of iteration":

#6: __iter = .iterator() while 1: try: rc = __iter.next() except ExhaustedIterator: break else: if rc is ExhaustedIterator: break = rc

> Another possibility would be using an iterator attribute > to get at the variable: > > #4: _> iter = .iterator() > while 1: _> if not iter.next(): > break _> = iter.value >

Uglier than any of the others. > You might want to check out the counterobject.c approach I used > to speed up the current for-loop in Python 1.5's ceval.c: > it's basically a mutable C integer which is used instead of > the current Python integer index. > The details can be found in my old patch: > > http://starship.python.net/crew/lemburg/mxPython-1.5.patch.gz Ah, yes, that's what I was thinking of. > """ Generic object iterators. [...] Thanks -- yes, that's what I was thinking of. Did you just whip this up?

The file says: Feb 2000... I don't remember what I wrote it for; it's in my lib/ dir meaning that it qualified as general purpose utility :-)

-- Marc-Andre Lemburg

Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/