[Python-Dev] variable name resolution in exec is incorrect (original) (raw)

Thomas Wouters thomas at python.org
Wed May 26 15:02:41 CEST 2010

On Wed, May 26, 2010 at 11:48, Mark Dickinson <dickinsm at gmail.com> wrote:

On Wed, May 26, 2010 at 10:15 AM, Colin H <hawkett at gmail.com> wrote: > issue991196 was closed being described as intentional. I've added > a comment in that issue which argues that this is a serious bug (also > aserted by a previous commenter - Armin Rigo), because it creates a > unique, undocumented, oddly behaving scope that doesn't apply closures > correctly. At the very least I think this should be acknowledged as a > plain old bug (rather than a feature), and then a discussion about > whether it will be fixed or not.

Here's a quick recap of the issue so that people don't have to go searching through the bug archive. In Python 2.x, we get the following behaviour: _>>> code = """_ ... y = 3 ... def f(): ... return y ... f() ... """ >>> exec code in {} # works fine >>> exec code in {}, {} # dies with a NameError Traceback (most recent call last): File "", line 1, in File "", line 4, in File "", line 3, in f NameError: global name 'y' is not defined The issue is whether the second example should work, given that two different dictionaries have been passed. The cause of the NameError can be seen by looking at the bytecode: y is bound using STORENAME, which stores y into the locals dictionary (which here is not the same as the globals dictionary) but the attempt to retrieve the value of y uses LOADGLOBAL, which only looks in the globals. >>> co = compile(code, 'mycode', 'exec') >>> dis.dis(co) 1 0 LOADCONST 0 (3) 3 STORENAME 0 (y) 2 6 LOADCONST 1 (<code object f at_ _0xa22b40, file "mycode", line 2>) 9 MAKEFUNCTION 0 12 STORENAME 1 (f) 4 15 LOADNAME 1 (f) 18 CALLFUNCTION 0 21 POPTOP 22 LOADCONST 2 (None) 25 RETURNVALUE >>> dis.dis(co.coconsts[1]) # disassembly of 'f' 3 0 LOADGLOBAL 0 (y) 3 RETURNVALUE This is a long way from my area of expertise (I'm commenting here because it was me who sent Colin here in the first place), and it's not clear to me whether this is a bug, and if it is a bug, how it could be resolved. What would the impact be of having the compiler produce 'LOADNAME' rather than 'LOADGLOBAL' here?

It wouldn't matter. The 'f' function only knows about its own namespace (separate from the surrounding code's local namespace), and the global namespace. LOAD_NAME is only different from LOAD_GLOBAL in that it looks in the local namespace first, but in this case the local namespace contains nothing. Here's what happens:

'exec code in d1, d2' executes code with 'd1' as locals and 'd2' as globals. Assignment always operates on the local namespace (barring the 'global' declaration.) The function definition also assigns to the local namespace, but the created function knows nothing about that local namespace -- it only cares about its own namespace and the global namespace, 'd1'.

'exec code in d1' does the same thing as 'exec code in d1, d1': it uses the same dict for the locals and the globals. The execution of the code doesn't change -- the assignment to 'y' still assigns to the locals, and the 'f' function still looks it up in globals, but now they are the same dict. Using the same dict for locals and globals is how modules work, as well.

The main confusion here is the fact that 'exec' doesn't generate closures. (Nobody was ever confused about this behaviour back in Python 2.0-and-earlier! :-) The reason for that is the disconnect between the compiler and the exec statement: the compiler sees no enclosing function, so it doesn't generate a closure. The exec statement, because it gets two different namespaces, then executes it like a function, with a distinct local namespace.

-- Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20100526/0c536c93/attachment-0001.html>

More information about the Python-Dev mailing list