Refactor the parser to consume token trees by jseyfried · Pull Request #39118 · rust-lang/rust (original) (raw)

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation5 Commits8 Checks0 Files changed

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

This is groundwork for efficiently parsing attribute proc macro invocations, bang macro invocations, and TokenStream-based attributes and fragment matchers.

This improves parsing performance by 8-15% and expansion performance by 0-5% on a sampling of the compiler's crates.

r? @nrc

This looks great. I'm a little concerned about the commit removing the lookahead buffer - is that because we never use lookahead now?

Oh, right, you can look ahead in the Tok stream without it, I guess?

📌 Commit 0b9e26f has been approved by nrc

Yeah, we look ahead in parser.tts directly.

alexcrichton added a commit to alexcrichton/rust that referenced this pull request

Jan 20, 2017

…r=nrc

Refactor the parser to consume token trees

This is groundwork for efficiently parsing attribute proc macro invocations, bang macro invocations, and TokenStream-based attributes and fragment matchers.

This improves parsing performance by 8-15% and expansion performance by 0-5% on a sampling of the compiler's crates.

r? @nrc

bors added a commit that referenced this pull request

Jan 20, 2017

Rollup of 28 pull requests

• Successful merges: #38603, #38761, #38842, #38847, #38955, #38966, #39062, #39068, #39077, #39111, #39112, #39114, #39118, #39120, #39132, #39135, #39138, #39142, #39143, #39146, #39157, #39166, #39167, #39168, #39179, #39184, #39195, #39197
• Failed merges: #39060, #39145

bors added a commit that referenced this pull request

Jan 21, 2017

Rollup of 28 pull requests

• Successful merges: #38603, #38761, #38842, #38847, #38955, #38966, #39062, #39068, #39077, #39111, #39112, #39114, #39118, #39120, #39132, #39135, #39138, #39142, #39143, #39146, #39157, #39166, #39167, #39168, #39179, #39184, #39195, #39197
• Failed merges: #39060, #39145