[Python-Dev] PEP 0424: A method for exposing a length hint (original) (raw)

Alexandre Zani alexandre.zani at gmail.com
Sun Jul 15 17:38:20 CEST 2012

On Sun, Jul 15, 2012 at 8:08 AM, Mark Shannon <mark at hotpy.org> wrote:

Brett Cannon wrote:

On Sun, Jul 15, 2012 at 10:39 AM, Mark Shannon <mark at hotpy.org_ _<mailto:mark at hotpy.org>> wrote: Nick Coghlan wrote: Right, I agree on the value in being able to return something to say "this cannot be converted to a concrete container". I still haven't seen a use case where the appropriate response to "I don't know" differs from the appropriate response to a hint of zero - that is, you don't preallocate, you just start iterating. There seem to be 5 possible classes values of lengthhint that an iterator object can provide: 1. Don't implement it at all. 2. Implement lengthhint() but don't want to return any value. Either raise an exception (TypeError) -- As suggested in the PEP. or return NotImplemented -- my preferred option. 3. Return a "don't know" value: Returning 0 would be fine for this, but the VM might want to respond differently to "don't know" and 0. lengthhint() == 0 container should be minimum size. lengthhint() == "unknown" container starts at default size. 4. Infinite iterator: Could return float('inf'), but given this is a "hint" then returning sys.maxsize or sys.maxsize + 1 might be OK. Alternatively raise an OverflowError I am really having a hard time differentiating infinity with "I don't know" since they are both accurate from the point of view of lengthhint and its typical purpose of allocation. You have no clue how many values will be grabbed from an infinite iterator, so it's the same as just not knowing upfront how long the iterator will be, infinite or not, and thus not worth distinguishing. 5. A meaningful length. No problem :) Also, what are the allowable return types? 1. int only 2. Any number (ie any type with a int() method)? 3. Or any integer-like object (ie a type with a index() method)? My suggestion: a) Don't want to return any value or "don't know": return NotImplemented b) For infinite iterators: raise an OverflowError c) All other cases: return an int or a type with a index() method. I'm fine with (a), drop (b), and for (c) use what we allow for len() since, as Nick's operator.lengthhint pseudo-code suggests, people will call this as a fallback if len isn't defined. So how does an iterator express infinite length? What should happen if I am silly enough to do this: list(itertools.count()) This will fail; it should fail quickly. Cheers, Mark.

Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/alexandre.zani%40gmail.com

The PEP so far says: "It may raise a TypeError if a specific instance cannot have its length estimated." In many ways, "I don't know" is the same as this "specific instance cannot have its length estimated". Why not just raise a TypeError?

Also, regarding the code Nick posted above, I'm a little concerned about calling len as the first thing to try. That means that if I implement both len and len_hint (perhaps because len is very expensive) len_hint will never be used. It's relatively easy to say:

try: hint = len_hint(l) except TypeError: hint = len(l)

More information about the Python-Dev mailing list