Issue 3692: improper scope in list comprehension, when used in class declaration (original) (raw)

Issue3692

Created on 2008-08-26 23:20 by kaizhu, last changed 2022-04-11 14:56 by admin.

Messages (13)
msg72002 - (view)	Author: kai zhu (kaizhu)	Date: 2008-08-26 23:20
in 3rd line, list comprehension tries to access class_attribute1 as a global variable (code is valid in python 2.5) >>> class Foo(object): ... class_attribute1 = 1 ... class_attribute2 = [class_attribute1 for x in range(8)] ... Traceback (most recent call last): File "", line 1, in File "", line 3, in Foo File "", line 3, in NameError: global name 'class_attribute1' is not defined
msg72007 - (view)	Author: Daniel Diniz (ajaksu2) *	Date: 2008-08-27 04:54
I believe the problem is that list comprehensions in 3.0 have scope like that of genexprs in 2.5, but the change was deliberate (as it also avoids leaking of temp variables). Compare to 2.5: >>> class Foo(object): ... class_attribute1 = 1 ... class_attribute2 = (class_attribute1 for x in range(8)) ... >>> Foo.class_attribute2.next() Traceback (most recent call last): File "", line 1, in File "", line 3, in NameError: global name 'class_attribute1' is not defined
msg72011 - (view)	Author: Georg Brandl (georg.brandl) *	Date: 2008-08-27 07:32
This won't change -- in 3.0, list comprehensions and generator expressions are both implemented using a function, so it's like the following: class Foo: attribute1 = 1 def f(): return attribute1 attribute2 = f()
msg192360 - (view)	Author: John McDonald (John.McDonald)	Date: 2013-07-05 20:58
Could we possibly revisit this? This feels like an implementation detail, not something specified. Consider the different cases: Type "help", "copyright", "credits" or "license" for more information. >>> class A: ... b = 5 ... print(locals()) ... x = [i*i for i in range(b)] ... {'__qualname__': 'A', '__locals__': {...}, '__module__': '__main__', 'b': 5} This case works, and shows that 'b' is clearly in locals(). >>> class A: ... b = 5 ... c = b * b ... print(locals()) ... d = [] ... for i in range(b): ... d.append(i*i) ... print(locals()) ... e = [i*i for i in range(b) if i*i < c] ... {'__locals__': {...}, '__qualname__': 'A', 'b': 5, '__module__': '__main__', 'c': 25} {'__qualname__': 'A', 'b': 5, '__module__': '__main__', 'c': 25, 'i': 4, 'd': [0, 1, 4, 9, 16], '__locals__': {...}} Traceback (most recent call last): File "", line 1, in File "", line 9, in A File "", line 9, in NameError: global name 'c' is not defined Again, it feels really arbitrary that the variable can be used in certain places in the list comprehension, but not others. And of course, all of this works properly if you place the definitions either at global scope or within a function.
msg314011 - (view)	Author: Simon Charette (charettes) *	Date: 2018-03-17 19:13
I stumble upon this bug when porting a Python 2 codebase to 3 and suddenly got a NameError for the following code. class Foo: a = [1,2,3] b = [4,5,6] c = [x * y for x in a for y in b] NameError: name 'b' is not defined Not sure what could be done at this point but I thought I'd leave some feedback given I was surprised this would break given it works just fine if not defined at the class level. a = [1,2,3] b = [4,5,6] c = [x * y for x in a for y in b]
msg315764 - (view)	Author: Mariatta (Mariatta) *	Date: 2018-04-25 23:46
I'm re-opening this, since the behavior sounds like a bug to me. While I understand why it behaves the way it is now, but it seems wrong behavior. Perhaps we should look into finding a solution.
msg316114 - (view)	Author: Ivan Levkivskyi (levkivskyi) *	Date: 2018-05-03 09:41
Mariatta, > While I understand why it behaves the way it is now, but it seems wrong behavior. Perhaps we should look into finding a solution. I am all in favour of dropping implicit function scope in comprehensions (mostly because of weirdness like described here, but it will also fix some `yield` issues and simplify some `await` aspects). But I was not able to convince others it worth the effort. Maybe we can make another attempt at discussing this?
msg316116 - (view)	Author: Ivan Levkivskyi (levkivskyi) *	Date: 2018-05-03 09:49
See https://bugs.python.org/issue33346 for yet another example where implicit function scope complicates life.
msg316142 - (view)	Author: Guido van Rossum (gvanrossum) *	Date: 2018-05-03 22:46
There's a proposal for a fix currently in PEP 572. I'm linking to the commit here because we're likely to remove it from PEP 572, but it may be proposed as a separate PEP: https://github.com/python/peps/commit/71fea57f19231b9917ce6a558c0b763288385398 Bug or not this is not going to change without a PEP, so I'm not sure it's a good idea to keep this issue open.
msg316271 - (view)	Author: Ivan Levkivskyi (levkivskyi) *	Date: 2018-05-07 13:55
> Bug or not this is not going to change without a PEP, so I'm not sure it's a good idea to keep this issue open. I agree this requires a PEP. I would like to keep this open as a place for pre-PEP discussions among those interested (and as a remainder to maybe write such PEP).
msg316272 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2018-05-07 14:09
I don't think this will need drastic changes. Just setting some flags here or there for making comprehensions using LOAD_NAME instead of LOAD_GLOBAL. The fix should lie near the fix for .
msg316274 - (view)	Author: Guido van Rossum (gvanrossum) *	Date: 2018-05-07 15:30
> I don't think this will need drastic changes. IIRC LOAD_NAME loads from the *local* scope, which will be the synthetic function created for the comprehension (whose scope contains the loop control variables). We may need a new opcode similar to LOAD_GLOBAL that looks in two non-local directories rather than just one before falling back to builtins; and the function object would need to have a link to both the class dict and the global dict -- currently function objects gave a __globals__ link but there's no chaining. Or perhaps __globals__ could be set to a chainmap referencing the class dict and the globals?
msg321361 - (view)	Author: Serhiy Storchaka (serhiy.storchaka) *	Date: 2018-07-10 07:31
Indeed, this issue is more complex than it looked to me. Technically, LOAD_FAST, which is used for reading the loop control variables in comprehensions, uses the array f->f_localsplus, while LOAD_NAME uses f->f_locals and f->f_globals. When executing class bodies, f->f_localsplus is NULL, and if executing functions f->f_locals usually is a NULL or a cached copy of a dict created from f->f_localsplus. But we can create a hybrid frame for comprehensions in class body, which will contain independent f->f_localsplus and f->f_locals. References to the loop control variables and to the class variables will work as expected, just the meaning of locals() will be changed.