[Python-Dev] PEP 492 vs. PEP 3152, new round (original) (raw)

Nathaniel Smith njs at pobox.com
Fri May 1 05:30:05 CEST 2015

On Apr 30, 2015 1:57 AM, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote:

Nathaniel Smith wrote:

Even if we put aside our trained intuitions about arithmetic, I think it's correct to say that the way unary minus is parsed is: everything to the right of it that has a tighter precedence gets collected up and parsed as an expression, and then it takes that expression as its argument. Tighter or equal, actually: '--a' is allowed. This explains why Yury's syntax disallows 'await -f'. The 'await' operator requires something after it, but there's nothing between it and the following '-', which binds less tightly. So it's understandable, but you have to think a bit harder. Why do we have to think harder? I suspect it's because the notion of precedence is normally introduced to resolve ambiguities. Knowing that infix '*' has higher precedence than infix '+' tells us that 'a + b * c' is parsed as 'a + (b * c)' and not '(a + b) * c'. Similarly, knowing that infix '.' has higher precedence than prefix '-' tells us that '-a.b' is parsed as '-(a.b)' rather than '(-a).b'. However, giving prefix 'await' higher precedence than prefix '-' doesn't serve to resolve any ambiguity. '- await f' is parsed as '-(await f)' either way, and 'await f + g' is parsed as '(await f) + g' either way. So when we see 'await -f', we think we already know what it means. There is only one possible order for the operations, so it doesn't look as though precedence comes into it at all, and we don't consider it when judging whether it's a valid expression.

The other reason this threw me is that I've recently been spending time with a shunting yard parser, and in shunting yard parsers unary prefix operators just work in the expected way (their precedence only affects their interaction with later binary operators; a chain of unaries is always allowed). It's just a limitation of the parser generator tech that python uses that it can't handle unary operators in the natural fashion. (OTOH it can handle lots of cases that shunting yard parsers can't -- I'm not criticizing python's choice of parser.) Once I read the new "documentation grammar" this became much clearer.

What's the conclusion from all this? I think it's that using precedence purely to disallow certain constructs, rather than to resolve ambiguities, leads to a grammar with less-than-intuitive characteristics.

The actual effect of making "await" a different precedence is to resolve the ambiguity in await x ** 2

If await acted like -, then this would be await (x ** 2) But with the proposed grammar, it's instead (await x) ** 2 Which is probably correct, and produces the IMHO rather nice invariant that "await" binds more tightly than arithmetic in general (instead of having to say that it binds more tightly than arithmetic except in this one corner case...).

But then given the limitations of Python's parser plus the desire to disambiguate the expression above in the given way, it becomes an arguably regrettable, yet inevitable, consequence that await -fut await +fut await ~fut become parse errors.

AFAICT these and the ** case are the only expressions where there's any difference between Yury's proposed grammar and your proposal of treating await like unary minus.

-n -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20150430/ed299072/attachment.html>

More information about the Python-Dev mailing list