Inconsistency between Display and IntoIterator for a TokenStream containing a module (original) (raw)

The following crate reproduces the issue. Given an attribute macro on a mod item like #[...] mod module;, iterating over the tokens of the TokenStream produces mod module; but invoking to_string() on the TokenStream produces mod module { /* the contents */ }. I don't know which one is correct but they should be consistent.

@jseyfried

Cargo.toml

[package] name = "repro" version = "0.0.0" publish = false

[lib] proc-macro = true

src/main.rs

#![feature(proc_macro)]

extern crate repro;

#[repro::print] mod module;

fn main() {}

src/module.rs

src/lib.rs

#![feature(proc_macro)]

extern crate proc_macro; use proc_macro::TokenStream;

#[proc_macro_attribute] pub fn print(_args: TokenStream, input: TokenStream) -> TokenStream { println!("{}", input); for tt in input { println!("{:?}", tt.kind); } TokenStream::empty() }

Output

mod module {
    pub struct S;
}
Term(Term(mod))
Term(Term(module))
Op(';', Alone)