Stabilize macros in some more positions by petrochenkov · Pull Request #63931 · rust-lang/rust (original) (raw)

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

jhpratt, mateusmedeiros, abonander, peterjoel, taiki-e, est31, theduke, Robbepop, fogti, DutchGhost, and 5 more reacted with thumbs up emoji leo60228, DutchGhost, lloydmeta, ErichDonGubler, lambda-fairy, dnrusakov, mateusmedeiros, and GabrielMajeri reacted with hooray emoji

This comment has been minimized.

Centril added needs-fcp

This change is insta-stable, so needs a completed FCP to proceed.

relnotes

Marks issues that should be documented in the release notes of the next release.

T-lang

Relevant to the language team, which will review and decide on the PR/issue.

labels

Aug 27, 2019

This comment has been minimized.

Stabilization report

All the features below target Rust 1.39 (November 7th 2019).
Tests can be found in test files touched by this PR.

Fn-like macros and attribute macros in extern blocks

Macro attributes are now supported on items in extern blocks, the behavior is the same as for attributes on free/trait/impl items available on stable.

The feature name is #![feature(macros_in_extern)].
The feature was implemented in April 2018 in #49350 to fix issue #48747.
It didn't go through an RFC because the feature seemed to be a trivial extension consistent with already supported macros in item and item-like positions.

Fn-like procedural macros in type positions

Proc macros can now be used in type positions (type A = foo!();), the behavior is the same as for macro_rules macros in the same position, which are available on stable.
The difference is that proc macros on stable use call-site hygiene, while macro_rules use mixed call-site/def-site hygiene, but that difference only affects local variables, labels and $crate so it doesn't apply to types (or at least applies to types no more than to item-position macros, which are available on stable).

The feature is also a part of the polyphyletic #![feature(proc_macro_hygiene)].
The feature was implemented together with all other proc macros somewhere in 2017-2018.
It wasn't stabilized yet because it somehow got lost during the Macro 1.2 stabilization wave, and nobody actively pushed it after that.

@Centril

it would be good to elaborate re. hygiene, how this may e.g. interact with expressions in type contexts, and give a recap (because issues re. hygiene isn't the best known subject...) of the issues wit proc macros expanding to expressions and why it isn't an issue here.

Proc macros in expressions and patterns have mostly philosophical issues - they can 1) consume local variables from their environment code and 2) can produce local variables consumable from their environment code.
Given that proc macros on stable can only use "unhygienic" Span::call_site hygiene, and unhygienic local variables are bad, proc macros in expression and patterns are prohibited in general.
Those philosophical issues do not apply to types, they can neither produce nor consume local variables.

The technical issue with expressions and patterns is that local variables stress the hygiene algorithm more and are more likely to hit some corner cases that may certainly exist because the hygiene algorithm does a few questionable things and is not fully understood.
That said, you still can work that feature gate around with stable item macros (https://crates.io/crates/proc-macro-hack), but some hope exists that proc-macro-hack is still a more limited exposure than arbitrary stable expression/pattern macros would be.

As such, I would like us to consider these separately in 3 different stabilization PRs with associated reports & FCPs.

The features are pretty trivial (that's why they were selected for immediate stabilization in the first place), I can split the PR if the lang team wants, but that would mostly create extra work for everyone involved, IMO.

The features are pretty trivial (that's why they were selected for immediate stabilization in the first place), I can split the PR if the lang team wants, but that would mostly create extra work for everyone involved, IMO.

Fair enough; the types bit is the only bit I currently have worries about (see below).

Those philosophical issues do not apply to types, they can neither produce nor consume local variables.

How does this affect interactions with VLAs in the style of #48055 or run-time-value-dependent typing more generally? For example, let x: type_mac!() = ...; looks like it could "consume" a local let n if unhygienic and producing [u8; dyn n] or some such?

r? @Centril (r=me on the code itself once FCP completes)

@Centril

How does this affect interactions with VLAs in the style of #48055 or run-time-value-dependent typing more generally? For example, let x: type_mac!() = ...; looks like it could "consume" a local let n if unhygienic and producing [u8; dyn n] or some such?

To clarify, Span::call_site hygiene generally works for local variables (I haven't seen bug reports for it at least), and already available on stable after jumping through a couple of hoops, e.g.

#[proc_macro] fn item_macro(input) { "fn item() { ${input} let y = x; // OK
}" }

item_macro!(let x = 10);

As I understand, the nrc's desire was to limit it (#52121 (comment)) rather than prohibit completely.
So further minor leaks like [u8; dyn n] should be ok.

First, sorry about the delay here but I think I need to understand better...

As I understand, the nrc's desire was to limit it (#52121 (comment)) rather than prohibit completely.

cc @nrc, @pnkfelix (maybe back from vacation soon?), @samth, and @rust-lang/lang

So further minor leaks like [u8; dyn n] should be ok.

So I feel a distinct lack of understanding re. what minor constitutes here. Would e.g. [u8; dyn { n = 3; n }] change anything?

As I understand, the nrc's desire was to limit it (#52121 (comment)) rather than prohibit completely.
So further minor leaks like [u8; dyn n] should be ok.

To clarify, I don't have a specific objection, just a general level of uncertainty about our macro hygiene in general. I'd prefer to not stabilise anything until we resolve that uncertainty, but that might have happened since I was last involved (or might be unrealistic to expect).

So I feel a distinct lack of understanding re. what minor constitutes here. Would e.g. [u8; dyn { n = 3; n }] change anything?

Minor as in amount of code it's used in, so it less likely to hit corner cases statistically.
(n = 3 doesn't change anything.)
Anyway, we can feature gate dyn n in expansions specifically, if it's ever implemented.

[T; dyn expr] in types isn't at all a thing that I've ever seen proposed.

The VLA(-like) proposal was for [expr; dyn expr] evaluating to a [T] value.

Runtime expressions in types amount to dependent typing, which we can all agree is far away, if ever, and for the time being these proc macros don't really have issues like that.

That said, the interesting case IMO is this:

macro_rules! foo { ($x:ident, $e:expr) => ([(); { let $x = 0; let x = 1; $e }]) }

macro_rules! bar { ($x:ident, $e:expr) => ([(); { let $x = 0; let $x = 1; $e }]) }

const _: foo!(x, x + 1) = [()]; const _: bar!(x, x + 1) = [(), ()];

If you were to implement either foo or bar as proc macros, would they both behave like bar?

However, even if that is the case, since the expression using x must be passed into the macro for the distinction to matter, the proc macro could very well "scavenge" for the right identifier in the input to produce the unhygienic behavior anyway.

So it does help that types are "closed expressions", wrt local bindings in scope.

I've just noticed that this PR stabilizes the next code as well:

#[my_attr] mod m { mod n; // Out-of-line module, how should my_attr see it in its input? }

So, we additionally need to feature gate out-of-line modules in attr/derive input.

This may affect stable in corner cases:

#[my_attr] fn m() { #[path = "zzz.rs"] mod n; }

UPDATE: Done in #64273.

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

The RFC will be merged soon.

petrochenkov added S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

and removed S-waiting-on-team

Status: Awaiting decision from the relevant subteam (see the T- label).

labels

Sep 30, 2019

Add some tests for macros in extern blocks, remove duplicate tests

@Centril
Rebased, the stabilization version is bumped to 1.40.

📌 Commit 5ae38bb has been approved by Centril

bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Sep 30, 2019

Centril added a commit to Centril/rust that referenced this pull request

Oct 1, 2019

Centril added a commit to Centril/rust that referenced this pull request

Oct 1, 2019

bors added a commit that referenced this pull request

Oct 1, 2019

Rollup of 10 pull requests

Successful merges:

#63674 (syntax: Support modern attribute syntax in the meta matcher)
#63931 (Stabilize macros in some more positions)
#64887 (syntax: recover trailing | in or-patterns)
#64895 (async/await: improve not-send errors)
#64896 (Remove legacy grammar)
#64907 (A small amount of tidying-up factored out from PR #64648)
#64928 (Add tests for some issues)
#64930 (Silence unreachable code lint from await desugaring)
#64935 (Improve code clarity)
#64937 (Deduplicate closure type errors)

Failed merges:

r? @ghost

ehuss mentioned this pull request

Oct 1, 2019

Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request

Dec 29, 2019

JohnTitor added a commit to JohnTitor/rust that referenced this pull request

Dec 30, 2019

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request

Jan 6, 2020

Version 1.40.0 (2019-12-19)

Language

You can now use tuple structs and tuple enum variant's constructors in const contexts. e.g.

pub struct Point(i32, i32);

const ORIGIN: Point = {
    let constructor = Point;

    constructor(0, 0)
};

You can now mark structs, enums, and enum variants with the #[non_exhaustive] attribute to indicate that there may be variants or fields added in the future. For example this requires adding a wild-card branch (_ => {}) to any match statements on a non-exhaustive enum. (RFC 2008)
You can now use function-like procedural macros in extern blocks and in type positions. e.g. type Generated = macro!();
Function-like and attribute procedural macros can now emit macro_rules! items, so you can now have your macros generate macros.
The meta pattern matcher in macro_rules! now correctly matches the modern attribute syntax. For example (#[$m:meta]) now matches #[attr], #[attr{tokens}], #[attr[tokens]], and #[attr(tokens)].

Compiler

* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support.

Libraries

The is_power_of_two method on unsigned numeric types is now a const function.

Stabilized APIs

Cargo

Misc

You can now specify the #[cfg(doctest)] attribute to include an item only when running documentation tests with rustdoc.

Compatibility Notes

As previously announced, any previous NLL warnings in the 2015 edition are now hard errors.
The include! macro will now warn if it failed to include the entire file. The include! macro unintentionally only includes the first expression in a file, and this can be unintuitive. This will become either a hard error in a future release, or the behavior may be fixed to include all expressions as expected.
Using #[inline] on function prototypes and consts now emits a warning under unused_attribute lint. Using #[inline] anywhere else inside traits or extern blocks now correctly emits a hard error.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this pull request

Jan 14, 2020

Version 1.40.0 (2019-12-19)

Language

You can now use tuple structs and tuple enum variant's constructors in const contexts. e.g.

pub struct Point(i32, i32);

const ORIGIN: Point = {
    let constructor = Point;

    constructor(0, 0)
};

You can now mark structs, enums, and enum variants with the #[non_exhaustive] attribute to indicate that there may be variants or fields added in the future. For example this requires adding a wild-card branch (_ => {}) to any match statements on a non-exhaustive enum. (RFC 2008)
You can now use function-like procedural macros in extern blocks and in type positions. e.g. type Generated = macro!();
Function-like and attribute procedural macros can now emit macro_rules! items, so you can now have your macros generate macros.
The meta pattern matcher in macro_rules! now correctly matches the modern attribute syntax. For example (#[$m:meta]) now matches #[attr], #[attr{tokens}], #[attr[tokens]], and #[attr(tokens)].

Compiler

* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support.

Libraries

The is_power_of_two method on unsigned numeric types is now a const function.

Stabilized APIs

Cargo

Misc

You can now specify the #[cfg(doctest)] attribute to include an item only when running documentation tests with rustdoc.

Compatibility Notes

As previously announced, any previous NLL warnings in the 2015 edition are now hard errors.
The include! macro will now warn if it failed to include the entire file. The include! macro unintentionally only includes the first expression in a file, and this can be unintuitive. This will become either a hard error in a future release, or the behavior may be fixed to include all expressions as expected.
Using #[inline] on function prototypes and consts now emits a warning under unused_attribute lint. Using #[inline] anywhere else inside traits or extern blocks now correctly emits a hard error.

Labels

disposition-merge

This issue / PR is in PFCP or FCP with a disposition to merge it.

finished-final-comment-period

The final comment period is finished for this PR / Issue.

relnotes

Marks issues that should be documented in the release notes of the next release.

S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

T-lang

Relevant to the language team, which will review and decide on the PR/issue.