Lazy-chunk Symbol interner by Mark-Simulacrum · Pull Request #136401 · rust-lang/rust (original) (raw)
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})
Status: Awaiting review from the assignee but also interested parties.
Relevant to the compiler team, which will review and decide on the PR/issue.
labels
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-page Symbol interner
This is a simple and only semi-correct impl that works well only on 64-bit platforms where we can lazily allocate zeroed pages (i.e., not actually use up memory for zeroed pages), meaning that we can grab a 4GB chunk of memory for all interned strings and then just offset allocate into that.
This already fixes unsoundness in the Symbol::as_str by leaking that 4gb memory chunk, but could also be faster (TBD).
A rewrite supporting ~any platform should be pretty straightforward by sharding the allocation and manually lazily allocating it in chunks.
r? @ghost -- this impl can't land due to being non-workable on Windows, opening for initial perf run
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-page Symbol interner
This is a simple and only semi-correct impl that works well only on 64-bit platforms where we can lazily allocate zeroed pages (i.e., not actually use up memory for zeroed pages), meaning that we can grab a 4GB chunk of memory for all interned strings and then just offset allocate into that.
This already fixes unsoundness in the Symbol::as_str by leaking that 4gb memory chunk, but could also be faster (TBD).
A rewrite supporting ~any platform should be pretty straightforward by sharding the allocation and manually lazily allocating it in chunks.
r? @ghost -- this impl can't land due to being non-workable on Windows, opening for initial perf run
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-page Symbol interner
This is a simple and only semi-correct impl that works well only on 64-bit platforms where we can lazily allocate zeroed pages (i.e., not actually use up memory for zeroed pages), meaning that we can grab a 4GB chunk of memory for all interned strings and then just offset allocate into that.
This already fixes unsoundness in the Symbol::as_str by leaking that 4gb memory chunk, but could also be faster (TBD).
A rewrite supporting ~any platform should be pretty straightforward by sharding the allocation and manually lazily allocating it in chunks.
r? @ghost -- this impl can't land due to being non-workable on Windows, opening for initial perf run
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-chunk Symbol interner
This fixes the unsoundness in Symbol::as_str, while also making it a bit faster (replacing locking with a single atomic load).
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-chunk Symbol interner
This fixes the unsoundness in Symbol::as_str, while also making it a bit faster (replacing locking with a single atomic load).
Status: This is awaiting some action (such as code changes or more information) from the author.
and removed S-waiting-on-review
Status: Awaiting review from the assignee but also interested parties.
labels
This fixes unsoundness in the Symbol::as_str by leaking the chunks (via the static memory).
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-chunk Symbol interner
This fixes the unsoundness in Symbol::as_str, while also making it a bit faster (replacing locking with a single atomic load).