[Python-Dev] Don't set local variable in a list comprehension or generator (original) (raw)

Guido van Rossum guido at python.org
Thu May 19 19:56:23 CEST 2011

On Wed, May 18, 2011 at 2:34 PM, Victor Stinner <victor.stinner at haypocalc.com> wrote:

Le mercredi 18 mai 2011 à 16:19 +0200, Nadeem Vawda a écrit :

I'm not sure why you would encounter code like that in the first place. Well, I found the STOREFAST/LOADFAST "issue" while trying to optimize the this module which reimplements rot13 using a dict in Python 3: d = {} for c in (65, 97):  for i in range(26):  d[chr(i+c)] = chr((i+13) % 26 + c) I tried: d = {chr(i+c): chr((i+13) % 26 + c) for i in range(26) for c in (65, 97)} But it is slower whereas I read somewhere than generators are faster than loops.

I'm curious where you read that. The explicit loop should be faster or equally fast except when you can avoid a loop in bytecode by applying map() to a built-in function. However map() with a lambda is significantly slower. Maybe what you recall actually (correctly) said that a comprehension is faster than map+lambda?

By the way, (c for c in ...) is slower than [c for c in ...]. I suppose that a generator is slower because it exits/reenter into PyEvalEvalFrameEx() at each step, whereas [c for c ...] uses BUILDLIST in a dummy (but fast) loop.

Did you test this in Python 2 or 3? In 2 the genexpr is definitely slower than the comprehension; in 3 I'm not sure there's much difference any more.

(c for c in ...) and [c for c in ...] is stupid, but I used a simplified example to explain the problem. A more realistic example would be:

squares = (x*x for x in range(10000)) You don't really need the "x" variable, you just want the square. Another example is the syntax using a if the filter the data set: (x for x in ... if condition(x))

> I heard about optimization in the AST tree instead of working on the > bytecode. What is the status of this project?

Are you referring to issue11549? There was some related discussion [1] on python-dev about six weeks ago, but I haven't seen anything on the topic since then. Ah yes, it looks to be this issue. I didn't know that there was an issue.

Hm, probably.

-- --Guido van Rossum (python.org/~guido)

More information about the Python-Dev mailing list