Store a Symbol in InternedString by Zoxc · Pull Request #46972 · rust-lang/rust (original) (raw)
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Conversation24 Commits2 Checks0 Files changed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})
r? @pnkfelix
(rust_highfive has picked a reviewer for you, use r? to override)
r? @jseyfried (I don't think we want these changes but we probably need a wider discussion)
EDIT: InternedString, before this PR, might be unsound, wrt placing it in a thread_local.
However, that wouldn't be the case if the interner is owned outside of the threads using it.
☔ The latest upstream changes (presumably #46899) made this pull request unmergeable. Please resolve the merge conflicts.
WIP: Store a Symbol in InternedString Store a Symbol in InternedString
☔ The latest upstream changes (presumably #46803) made this pull request unmergeable. Please resolve the merge conflicts.
bors added a commit that referenced this pull request
Store a Symbol in InternedString
(Also, Note that @jseyfried has not yet weighed in on whether they want these changes.)
review ping @jseyfried ! pinging you on IRC too! or not because i don't see you :)
Relevant to the compiler team, which will review and decide on the PR/issue.
label
Review ping for you @jseyfried! Also cc @rust-lang/compiler.
Can somebody summarize (a) what is being done here so I don't have to read the diffs and (b) why I should care?
@nikomatsakis This is reverting an API change @jseyfried (I believe) did.
Right now we deref InternedString to str (sound because InternedString is not Send/Sync).
@Zoxc moves us backwards to an with_str style API with this PR (which makes InternedString largely redundant with Symbol - IMO InternedString should be e.g. SymbolContents anyway).
What's the point of InternedString as a separate type if it's equivalent to Symbol now?
So far I didn't look carefully at @Zoxc's multithreading-related PRs and mostly seen them as something quite unfortunate, but maybe necessary after all.
Could someone write a design summary about how Symbols are supposed to be organized in multithreaded rustc?
Is there going to be a global interner? Or many thread-local interners with string duplication? Or both?
What's the point of InternedString as a separate type if it's equivalent to Symbol now?
It has Ord and Hash implementations based on strings instead of an unstable integer like Symbol.
Is there going to be a global interner? Or many thread-local interners with string duplication? Or both?
My branch has a interner per compilation session which is behind a lock. There is no parallelism during parsing there though, so all symbols are identical to what happens on master.
This seems to be slowing things down quite a bit (as per the perf run above). Is this a soundness fix? Is there an alternative implementation that allows us to keep using direct string references?
I believe we can just require that the interner outlives the threads that use it.
@michaelwoerister I suspect the slowdown is due to the usages of to_string() here. Most of these are because we want to match the interned string on constant strings and there's some return inside so we don't want a closure. We should instead pre-intern these constant strings using a macro. This avoids the string allocation and the string comparisons.
@eddyb I think that spawning a thread for the interner should work. Not terribly elegant though.
@Zoxc I mean you already have the threads, you just have to own the interner outside of them.
We should instead pre-intern these constant strings using a macro. This avoids the string allocation and the string comparisons.
It seems pretty important to resolve this, no?
Thanks for the PR, @Zoxc ! Since we haven't heard from you in a few weeks, I'm going to close this for now. Please feel free to reopen once you address the merge conflicts and the last round of feedback!
Status: Inactive and waiting on the author. This is often applied to closed PRs.
and removed S-waiting-on-author
Status: This is awaiting some action (such as code changes or more information) from the author.
labels
This was referenced
Apr 9, 2018
kennytm added a commit to kennytm/rust that referenced this pull request
…woerister
Rename InternedString to LocalInternedString and introduce a new thread-safe InternedString
This is an allocation-free alternative to rust-lang#46972.
alexcrichton added a commit to alexcrichton/rust that referenced this pull request
bors added a commit that referenced this pull request
Labels
Status: Inactive and waiting on the author. This is often applied to closed PRs.
Relevant to the compiler team, which will review and decide on the PR/issue.