Non-terminals, e.g. expr, have diverged between parser and macro matcher · Issue #86730 · rust-lang/rust (original) (raw)

Spawned off of PR #84364, based on discussion from lang team meeting (minutes, youtube).

The meaning of the non-terminal "expr" according to the rustc parser has changed (or would like to change) from its meaning according to the current rustc macro matcher.

Concretely:

The inline_const feature has already added a production roughly of the form EXPR ::= const { EXPR }
The let_chains feature may want (in the future) to add a production roughly of the form COND_EXPR ::= let PAT = EXPR, where COND_EXPR can occur as a sub-expression in the condition that drives an if expression or while statement.

In both cases, for backwards compatibility, the expr fragment specifier does not accept these new productions. Examples of the kinds of problems this causes follow:

In other words, today you simply cannot write a macro fragment specifier denoting expressions that will match const { EXPR } or let PAT = EXPR, other than something like $e:tt, which would also match arbitrary token-trees, which won't catch errors as effectively as a more precise fragment specifier.

So, the main question this raises is: How should we resolve this divergence between the parser and macro-rules matcher?

E.g. we could add a new fragment specifier (e.g. expr2024) that covers the new forms and call it a day.
Or we could, at an edition boundary, change the meaning of the expr fragment specifier to cover the new forms. (This would probably need to be coupled with an addition of a new fragment specifier that would capture the old semantics.)
In any case, we need a clear summary of the grammar changes, so that we can understand what expr matches now that it did not before.
I think any change we make here would be well-served to be documented by writing down the actual grammar being implemented, or rather, the changes to the grammar.
We also may need a crater run.