[Python-Dev] LOAD_NAME & classes (original) (raw)

Guido van Rossum guido@python.org
Tue, 23 Apr 2002 08:35:24 -0400

[Tim]

There's a point to that too: uniqueness also imposes costs on newbies and/or newcomers. Across the world of programming languages now, dynamic scoping and lexical scoping are "almost entirely it". [...] Nor its advantages, including better error detection, and ease of transferring hard-won knowledge among other lexically scoped languages.

But Python is unique in that it doesn't require declarations. (I've got to admit that the Perl example saddened me. But then in Perl, local variables are a recent invention. :-)

We've found before that this can go against what's common knowledge for other language (e.g. integer division).

[...]

Does it scale?

x = 0 def f(i): if i & 4: x = 10 def g(i): if i & 2: x = 20 def h(i): if i & 1: x = 30 print x h(i) g(i) f(3) I can look at that today and predict with confidence that h() will either print 30 (if and only if i is odd), or raise an exception. This is from purely local analysis of h's body -- it doesn't matter that it's nested, and it's irrelvant what the enclosing functions look like or do. That's a great aid to writing correct code. If the value of x h sees may come from h, or from g, or from f, or from the module scope instead, depending on i's specific value at the time f is called, there's a lot more to think about.

Yup. But it also requires intricate knowledge of Python's rules, which are different than any other language's rules. You simply can't have a variable declaration inside an if statement in other languages that extends to the entire function body -- either the scope would be limited to the block it's in, or the syntax wouldn't allow it.

Python's original semantic model on the other hand, and the model that's still used for globals at the global level, gives a clear explanation: a namespace is implemented as a dictionary, and name lookup searches a pre-set sequence of namespaces until it finds a hit. The lexical scoping rule determines how namespaces are combined. Doing the lookup at runtime is easier to understand than doing it at compile time -- even if the compile version might catch more bugs. But I'm repeating myself; I already said that in my previous message.

I could keep local+global straight in pre-1.0 Python, although I never got used to the inability to write nested functions that could refer to each other (perhaps you've forgotten how many times you had to explain that one, and how difficult it was to get across?).

No; apart from you, most people were happy with the rule "nested functions don't work".

Now that Python has full-blown nested scopes, the namespace interactions are potentially much more convoluted, and the "purely local analysis" shortcut made possible by everyone else's notion of lexical scoping becomes correspondingly more valuable.

I don't know. Full-blown nested scopes make namespace interactions more convoluted no matter what set of rules we pick. An alternative implementation model (with associated subtly different semantics semantics) would have been to create an explicit list of the dicts involved in the name resolution for a particular function invocation; we rejected that model because we wanted this to be (nearly) as fast as locals, so we moved more of the analysis to compile time.

But by doing so, we introduced more of a dependency on the programmer's ability to understand what happens at compile time, and that breaks the "only runtime exists" illusion.

In PEP 267 Jeremy is exploring how to optimize access to globals without changing the rules. The change to LOAD_FAST that I considered before would have optimized access to locals without changing the rules, and I still regret that I didn't think of that when I created LOAD_FAST (even though you disagree): the "only runtime" rule is helpful for a large class of programmers, not only newbies, and I'm not sure that adding more and more cruft from truly compiled languages to Python's semantics is a good idea. (Adding compiler technology that doesn't change the rules is fine, of course, if it helps optimizations or better diagnostics.)

> ... > Um, that's not what I'd call dynamic scoping. It's dynamic lookup.

I know -- the problem is that you're the only one in the world making this distinction, and that makes it hard to maintain over time.

You can say that, but that doesn't make it so, and it doesn't convince me. The three-scope was gospel in the Python world, and many people actively disliked adding nested scopes (some still do).

If it had some killer advantage ... but it doesn't seem to. When Python switched to "strict local" names before 1.0, I don't recall anyone complaining -- if there was a real advantage to dynamic lookup at the local scope, it appeared to have escaped Python's users . I'll grant that it did make exec and "import *" more predictable in corner cases.

Well, we gave them a big reason not to complain: this was the singlemost biggest speedup in Python's history. But the rules were definitely harder to explain, because for the first time we had to explain a second compiler pass.

> It's trouble for a compiler that wants to optimize builtins, but the > semantic model is nice and simple and easy to explain with the "only > runtime" rule.

Dynamic scoping is also easy to explain, but it doesn't scale. I'm afraid dynamic lookup doesn't scale either. You should have stuck with Python's original two-level namespace, you know <0.9 wink>.

We need more than a single example to decide which rules bites worse for large programs. Deep nesting is not common; long functions are. And there the common annoyance is that a change in line 150 can break the code in line 2 of the function.

--Guido van Rossum (home page: http://www.python.org/~guido/)