Pep 572 update by gvanrossum · Pull Request #654 · python/peps (original) (raw)

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation30 Commits17 Checks0 Files changed

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

@Rosuav, @tim-one: On the plane home from PyCon I did some major surgery on PEP 572. The tl;dr is that I removed the changes to comprehensions and added Tim's proposal for making := inside a comprehension write the target in the containing scope. There's a bit more, below are the bullets from my local commits.

- Remove changes to comprehension scope
- Make := in comprehensions assign to containing non-comprehension scope
- Clarify binding precedence (tighter than comma, not at same level as =)
- Remove mention of more complex targets in the future
- Explicitly disallow toplevel :=
- Rewrite section on differences with =, enumerating all of them
- Remove "here's how this could be written without :=" from examples

- Tweak first paragraph of "Syntax and semantics"
- Add "Exceptional cases" (various explicit prohibitions)
- Clarify that lambda is a containing scope
- Clarify that := and = just don't mix
- Added "Open questions" section
- Added two new rejected alternatives: "Allowing commas to the right"
  and "Always requiring parentheses"
- Minor edits

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The outer parens don't really clarify much; perhaps point := (x, y) would be more indicative?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you're right. These critics want the following to be equivalent:

(point := (x, y))
(point := x, y)

The rest of this section is my refutation of that position. I'll fix it.

Do you want discussion here or on the ML?

One point that I find curious is that top-level assignment with := is forbidden due to the similarity with =, but that you're explicitly allowed to use NAME := expr as a positional parameter to a function, which is similarly similar to keyword args. The entire justification for this is the style recommendation of whether there are spaces surrounding the = or :=. That seems a little odd - that common style conventions (even those codified into PEP 8) should govern syntactic legality.

Personally, I would prefer to remove the special cases and just say that := assignments are allowed anywhere, including top level. It's perfectly legal to write f() if cond else g() at top-level, and we don't see heaps of code using that in place of if cond: f() else: g().

The grammar of Python is already a scarily complicated beast to look into. Adopting all of these special cases will make it even more so. Are you happy with the future of Python being "anyone can tinker with the stdlib, someone who knows C can tinker with the implementation of CPython, and only a core dev with at least twenty years' experience can look at the grammar"? Okay, so that's a bit of hyperbole, but only a bit. Every change is going to add some complexity, but IMO the goal should be to add as little as possible.

WRT co-authorship: with the edits you've made here, you absolutely merit putting your own name on it. Tim Peters less obviously, but I definitely wouldn't object to him being a co-author if you and/or he feel it's appropriate.

Do you want discussion here or on the ML?

Here please. The list has become too dysfunctional. I am even thinking that we may recommend to have all discussions in a dedicated tracker or issue.

One point that I find curious is that top-level assignment with := is forbidden due to the similarity with =, but that you're explicitly allowed to use NAME := expr as a positional parameter to a function, which is similarly similar to keyword args. The entire justification for this is the style recommendation of whether there are spaces surrounding the = or :=. That seems a little odd - that common style conventions (even those codified into PEP 8) should govern syntactic legality.

The justification for disallowing top level := is mostly political -- some people are very concerned about whether := will become the new spelling for =, and I don't want to encourage that worry. The same concern doesn't apply to keyword arguments.

Personally, I would prefer to remove the special cases and just say that := assignments are allowed anywhere, including top level. It's perfectly legal to write f() if cond else g() at top-level, and we don't see heaps of code using that in place of if cond: f() else: g().

Yeah, but as I said that would weaken support for the proposal.

The grammar of Python is already a scarily complicated beast to look into. Adopting all of these special cases will make it even more so. Are you happy with the future of Python being "anyone can tinker with the stdlib, someone who knows C can tinker with the implementation of CPython, and only a core dev with at least twenty years' experience can look at the grammar"? Okay, so that's a bit of hyperbole, but only a bit. Every change is going to add some complexity, but IMO the goal should be to add as little as possible.

Most of these special cases will have to checked in the second stage.

WRT co-authorship: with the edits you've made here, you absolutely merit putting your own name on it. Tim Peters less obviously, but I definitely wouldn't object to him being a co-author if you and/or he feel it's appropriate.

But if I'm co-author it's less clear that I can also approve (judge and jury and so on). Tim contributed the special case for := in comprehensions, a lot of finer points, and the ideas that real code is important.

The justification for disallowing top level := is mostly political
... Yeah, but as I said that would weaken support for the proposal.

Fair enough.

But if I'm co-author it's less clear that I can also approve (judge and jury and so on).

Good point. I'll leave it up to your discretion; at this point, it's another political decision really. You have the power to decide, regardless of authorship status; and you absolutely have merited it, by any logical measurement. Deciding to forego that authorship is your call.

Tim as co-author is fine in my view.

I should clarify some things: I don't have git. I don't know how to use git. I barely know enough about github to enter this comment. And I don't have a C compiler (my MSDN subscription "got lost" somehow when they migrated accounts on their side, and I gave up after some hours of trying to get that straightened out). That's why all I do now is type at mailing lists ;-) Presumably all of that could be repaired with reasonable effort, but there are so many other things I like doing that's not likley to rise to the top of my list soon.

That's why, e.g., when I suggested specific changes for the Reference Manual, I posted the new text rather than generate a patch. It's about all I can do for now.

Since I want the PEP to succeed, whether I'm added as co-author is a purely political issue to me: would that help, or hurt, the PEP's chances? I couldn't care less whether my name is on it either way, and I'm currently unable to generate or test patches either way either.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That wasn't my intent (I intended "in the parent block, period") - but this variation makes more sense to me - thanks! So long as expressions can't contain statements, it makes things "just work" more often, at the cost of making it impossible for inner comprehensions to rebind outer comprehensions' for target names. That's no real loss.

If, however, Python grows to support "expressions" that can contain arbitrary statements, then it also makes it impossible for comprehensions to rebind other kinds of local names too in outer comprehensions.

As is, this example is annoying ;-)

def f():
    global j
    [[(j := j) for i in range(5)] for j in range(5)]

In j := j, is the left j global and the right j local to the outer listcomp? We already have (or should have) a rule that makes it an error for a name to be both a for and an := target at the same level, but the example dodges that rule. In the "always in the parent block" version, both instances of j refer to the outer listcomp's j.

"Left is global, right is local to containing listcomp" is fine by me in practice, as would be just about any other resolution.. But it merits some thought.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I forgot about the bit where we agreed that you can't rebind the loop control variable of a comprehension using :=. Let's extend that so that you can't rebind the loop control variable of any containing comprehension. That would make this example invalid, which seems better than allowing the example to dodge the rule.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That works for me. There's a formal property I'm keen to preserve: that at any given comprehension level, all instances of a := target resolve to the same scope. So if any other seeming counterexamples come up, I expect I'd rather say "you can't do that" than pick one more-than-less arbitrarily.

OK, I've sent some updates:

• Correct/clarify "commas to the right" section
• Add Tim and Guido as authors
• Update abstract to mention :=
• Rule out targets conflicting with comprehension loop control

@tim-one If you have time and energy, it would be great if you could provide some motivating examples from your own life.

There are some TODOs in the text, and there's a section of open issues (with some overlap with the TODOs). Would be great to see opinions on these.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR adds multiple authors so this paragraph would be outdated.

I am even thinking that we may recommend to have all discussions in a dedicated tracker or issue.

Good idea.

Hey @stevendaprano are you interested in reviewing this update to PEP 572? (I'm not going to ask Nick since he's vowed to write a PEP about giving.)

[Guido]

@tim-one If you have time and energy, it would be great if you could provide some motivating examples from your own life.

I've been updating a mini-essay on the topic all along; maybe you could just slam in the following as an appendix ;-)

timcode.txt

I should be able to look at it over the next couple of days.

Thanks Tim, I added that as an appendix.

(Note that GitHub's "gvanrossum added some commits 2 days ago " is a lie -- most of those commits are from two days ago, but "Add timcode.txt as Appendix A" is from just now.)

I have another example that quite impressed me at the time. But I never posted it because nobody (except Mark Dickinson) would find it obvious on first or second reading ;-) Nevertheless, there may be some use in adding an example that isn't mindless ;-) Somebody else can judge that:

Where all variables are positive integers, and a is at least as large as the n'th root of x, this algorithm returns the floor of the n'th root of x (and roughly doubling the number of accurate bits per iteration):

while a > d := x // a**(n-1):
    a = ((n-1)*a + d) // n
return a

It's not obvious why that works, but is no more obvious in the "loop and a half" form. It's hard to prove correctness without building on the right insight (the "arithmetic mean - geometric mean inequality"), and knowing some non-trivial things about how nested floor functions behave. That is, the challenges are in the math, not really in the coding.

If you do know all that, then the assignment-expression form is easily read as "while the current guess is too large, get a smaller guess", where the "too large?" test and the new guess share an expensive sub-expression.

To my eyes, the original form is harder to understand:

while True:
    d = x // a**(n-1)
    if a <= d:
        break
    a = ((n-1)*a + d) // n
return a

• Remove changes to comprehension scope
• Make := in comprehensions assign to containing non-comprehension scope
• Clarify binding precedence (tighter than comma, not at same level as =)
• Remove mention of more complex targets in the future
• Explicitly disallow toplevel :=
• Rewrite section on differences with =, enumerating all of them
• Remove "here's how this could be written without :=" from examples

• Tweak first paragraph of "Syntax and semantics"
• Add "Exception cases" (various explicit prohibitions)
• Clarify that lambda is a containing scope
• Clarify that := and = just don't mix
• Added "Open questions" section
• Added two new rejected alternatives: "Allowing commas to the right" and "Always requiring parentheses"
• Minor edits

A very nice set of changes. I'm glad to see the changes to class scopes are gone: that ought to be a separate PEP.

I especially liked your examples of using := to capture witnesses for any/all. There have been a few requests for them to return the truthy/falsey value, and this is a reasonable way to get that effect.

A couple of other observations/comments...

You have prohibited using assignment expressions in function defaults:

def foo(answer=p := 42):  # INVALID

I don't know how I feel about this, but I recall at least one question of Python-List asking why they can't do this:

def foo(answer=42, value=answer+1):

It was some years ago and I can't find the post now. But perhaps we might say this is an obscure enough usage that we don't care to support it.

Regarding the Open Questions, I agree with all of the current answers with the possible exception of the last (function defaults) where I'm on the fence.

Chris, Guido, would you like me to write up a summary of the "Modern language design survey" thread for inclusion in the PEP?

Yes please! That'd make a great section IMO.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Peters'/Peters's/ Sorry, but that's English ;-)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added comments to two lines suggesting grammatical changes. I'm not sure why github is making me type this too ;-)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/become/became/

I've assigned my competing PEP a number (577) and merged it now. PR where I did that: #660

I found the scoping discussion with Tim very helpful, to the point where I became a convert to allowing inline assignment as a variant of augmented assignment, and structured the PEP primarily around allowing the existing augmented assignment operators to be used as expressions. Adding NAME := EXPR then becomes primarily about avoiding folks going to silly lengths to emulate it with one of the other operators.

While I've borrowed much of Tim's scoping proposal, what I have in PEP 577 isn't exactly the same as what you have here:

1. No implicit declarations of augmented assignment targets at function scope (you get TargetNameError instead). In combination with a suggested style rule for linters and type checkers, this takes the place of the general prohibition on top level usage of NAME := EXPR.
2. Lambda expressions get ignored when calculating augmented assignment targets the same way other expression level scopes do. This allows constructs that currently require 3 keywords (def, nonlocal, return) and three repetitions of a variable name to be replaced with a single keyword (lambda) and a single inline assignment expression.

(Note: now that Guido has pointed me to this PR, I won't post PEP 577 to python-ideas until you're happy with the changes you've made to PEP 572 and have merged them).

@Rosuav @tim-one Any further updates or can I push this? (I don't intend to post to python-ideas yet, but I'd rather not sit on such a momentous update forever. Let's iterate.)

@gvanrossum +1 . Process question: if I'd like to make small changes later (I installed git and will eventually figure out how to use it ;-) ), would I push them to this branch (pep-572-update) or create a new branch?

Create a new branch and make a new PR. This branch is over and done with now that it's been merged! Good luck learning git.

I'm one who thinks this should be available where statements are used today. But since the reason to use it there would be to de-emphasize the variable (limit the scope conceptually), making it (literally) a parenthetical comment is fine.

I'm not happy about the number (or even existence) of exceptions/special cases listed, but I think that in practice that will be a problem only for implementors, not users, so ... basically, this comment is a reminder that not everyone is upset by the PEP.