[Python-Dev] Re: PEP 279 (original) (raw)

Raymond Hettinger python@rcn.com
Fri, 29 Mar 2002 01:25:33 -0500

----- Original Message ----- From: "Guido van Rossum" <guido@python.org> To: "Raymond Hettinger" <python@rcn.com> Cc: <python-dev@python.org> Sent: Thursday, March 28, 2002 4:55 PM Subject: PEP 279

PEP 279 proposes three separate things. Comments on each:

1. New builtin: indexed() I like the idea of having some way to iterate over a sequence and its index set in parallel. It's fine for this to be a builtin.

Great!

I don't like the name "indexed"; adjectives do not make good function names. Maybe iterindexed()?

Any reasonable name is fine with me. Originally, I proposed index() but was talked out of it.

I like itercount() or enumerate(). The reason is that this function can work with any iterable including those that do not have numeric indices (such as dictionaries). Counting or enumeration is what is really happening.

I don't like the start and stop arguments. If I saw code like

for i, j in iterindexed("abcdefghij", 5, 10): print i, j I would expect it to print 5 f 6 g 7 h 8 i 9 j while the spec in the PEP would print 5 a 6 b 7 c 8 d 9 e

After nearly forty reviewers, you were the first to see the potential confusion, but now I see it too. The illusion comes from the argument name, 'start'. Changing it to 'countfrom=0' eliminates the suggestion of sequence slicing.

So now we have: itercount(collection, countfrom=0, stop=None)

Very confusing. I propose to remove the start/stop arguments

If the last change weren't sufficient, I could give-up the stop argument leaving:

itercount(collection, countfrom=0)

def iterindexed(collection): i = 0 it = iter(collection) while 1: yield (i, it.next()) i += 1

This is good. It gives 98% of what I'm after and I would take this rather than nothing at all. Still, I think we should not give up the countfrom argument which handles the common case of enumerating from one:

for linenum, line in itercount( alinelist, 1 ): print 'Line %03d: %s' % (linenum, line)

In summary, here is my order of preference:

  1. itercount(collection, countfrom=0, stop=None) # best
  2. itercount(collection, countfrom=0) # almost excellent
  3. itercount(collection) # good enough

2. Generator comprehensions

I don't think it's worth the trouble. I expect it will take a lot of work to hack it into the code generator: it has to create a separate code object in order to be a generator. List comprehensions are inlined, so I expect that the generator comprehension code generator can't share much with the list comprehension code generator. And this for something that's not that common and easily done by writing a 2-line helper function. IOW the ROI isn't high enough.

If implementation effort were taken out of the equation, would the balance shift back to a +1 ? In other words, if I find a developer who wants this badly enough to work with me to get it implemented, do we have a go?

Several commenters wanted this one quite a bit. An excerpt: "This rules. You rock."

3. Generator exception passing This is where the PEP seems weakest. There's no real motivation ("This is a true deficiency" doesn't count :-). There's no hint as to how it should be implemented.

I need help from others on py-dev who can articulate the need clearly. For me, it's as plain as day and I don't know what to say to convey the message better than it is expressed in the PEP.

The example has a "return log" statement in the generator body which is currently illegal, and I can't figure out to where this value would be returned.

I'll fix the example. It shouldn't return anything. That line should have said 'writelog(log)'.

It was meant to be a generic and oversimplified example of clean-up code in a generator being used as a data consumer.

The example looks like it doesn't need a generator,

Unfortunately, a minimal example demonstrating what throw does, is also so minimal that it doesn't make a compelling case. Dr. Mertz's co-routine code does make a much better case, but it takes half an hour to study his example. He was one of the early advocates of this proposal because he could see that it simplified and clarified his code. Tiny examples aren't as compelling.

Also, the need is much stronger when taken in conjunction with the generator attributes or generator parameter passing. That proposal was carved out and is being moved to a separate PEP so that the alternatives can be fully explored.

Raymond Hettinger

P.S. Thanks GvR for taking the time to look at this one. I didn't expect to hear back for another couple of weeks :)