Lazy-chunk Symbol interner by Mark-Simulacrum · Pull Request #136401 · rust-lang/rust (original) (raw)
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})
This fixes the unsoundness in Symbol::as_str, while also making it a bit faster (replacing locking with a single atomic load).
Status: Awaiting review from the assignee but also interested parties.
Relevant to the compiler team, which will review and decide on the PR/issue.
labels
These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.
If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.
This comment has been minimized.
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-page Symbol interner
This is a simple and only semi-correct impl that works well only on 64-bit platforms where we can lazily allocate zeroed pages (i.e., not actually use up memory for zeroed pages), meaning that we can grab a 4GB chunk of memory for all interned strings and then just offset allocate into that.
This already fixes unsoundness in the Symbol::as_str by leaking that 4gb memory chunk, but could also be faster (TBD).
A rewrite supporting ~any platform should be pretty straightforward by sharding the allocation and manually lazily allocating it in chunks.
r? @ghost -- this impl can't land due to being non-workable on Windows, opening for initial perf run
This comment has been minimized.
☀️ Try build successful - checks-actions
Build commit: 085fad8 (085fad86adba2eafccd19c18a6c7ed5d3fbecbab)
This comment has been minimized.
Finished benchmarking commit (085fad8): comparison URL.
Overall result: ❌✅ regressions and improvements - please read the text below
Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.
Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.
@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression
Instruction count
This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 0.2% | [0.2%, 0.2%] | 1 |
| Regressions ❌ (secondary) | 0.4% | [0.2%, 0.8%] | 7 |
| Improvements ✅ (primary) | -0.5% | [-0.8%, -0.2%] | 4 |
| Improvements ✅ (secondary) | -0.3% | [-0.6%, -0.1%] | 20 |
| All ❌✅ (primary) | -0.4% | [-0.8%, 0.2%] | 5 |
Max RSS (memory usage)
Results (primary -0.2%, secondary -3.0%)
This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 3.5% | [3.2%, 3.8%] | 3 |
| Regressions ❌ (secondary) | 2.4% | [1.7%, 3.4%] | 3 |
| Improvements ✅ (primary) | -2.1% | [-3.5%, -1.0%] | 6 |
| Improvements ✅ (secondary) | -3.7% | [-10.1%, -0.8%] | 24 |
| All ❌✅ (primary) | -0.2% | [-3.5%, 3.8%] | 9 |
Cycles
Results (secondary 1.4%)
This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | - | - | 0 |
| Regressions ❌ (secondary) | 2.9% | [2.3%, 3.3%] | 6 |
| Improvements ✅ (primary) | - | - | 0 |
| Improvements ✅ (secondary) | -3.2% | [-3.2%, -3.1%] | 2 |
| All ❌✅ (primary) | - | - | 0 |
Binary size
This benchmark run did not return any relevant results for this metric.
Bootstrap: 777.851s -> 776.705s (-0.15%)
Artifact size: 328.84 MiB -> 328.88 MiB (0.01%)
This comment has been minimized.
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-page Symbol interner
This is a simple and only semi-correct impl that works well only on 64-bit platforms where we can lazily allocate zeroed pages (i.e., not actually use up memory for zeroed pages), meaning that we can grab a 4GB chunk of memory for all interned strings and then just offset allocate into that.
This already fixes unsoundness in the Symbol::as_str by leaking that 4gb memory chunk, but could also be faster (TBD).
A rewrite supporting ~any platform should be pretty straightforward by sharding the allocation and manually lazily allocating it in chunks.
r? @ghost -- this impl can't land due to being non-workable on Windows, opening for initial perf run
☀️ Try build successful - checks-actions
Build commit: bad4b51 (bad4b5105a2fc0c2d704f0d02aac6fd34ec7b4a9)
This comment has been minimized.
Finished benchmarking commit (bad4b51): comparison URL.
Overall result: ❌✅ regressions and improvements - please read the text below
Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.
Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.
@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression
Instruction count
This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 0.7% | [0.2%, 3.9%] | 223 |
| Regressions ❌ (secondary) | 1.2% | [0.1%, 3.9%] | 142 |
| Improvements ✅ (primary) | - | - | 0 |
| Improvements ✅ (secondary) | -0.3% | [-0.4%, -0.2%] | 14 |
| All ❌✅ (primary) | 0.7% | [0.2%, 3.9%] | 223 |
Max RSS (memory usage)
Results (primary -0.2%, secondary -3.2%)
This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 3.2% | [2.8%, 3.5%] | 2 |
| Regressions ❌ (secondary) | 3.0% | [2.1%, 3.7%] | 4 |
| Improvements ✅ (primary) | -1.6% | [-2.5%, -0.6%] | 5 |
| Improvements ✅ (secondary) | -4.4% | [-10.1%, -1.0%] | 21 |
| All ❌✅ (primary) | -0.2% | [-2.5%, 3.5%] | 7 |
Cycles
Results (primary 2.0%, secondary 1.3%)
This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 2.0% | [0.9%, 3.1%] | 25 |
| Regressions ❌ (secondary) | 2.9% | [1.8%, 4.3%] | 34 |
| Improvements ✅ (primary) | - | - | 0 |
| Improvements ✅ (secondary) | -2.9% | [-3.3%, -2.1%] | 13 |
| All ❌✅ (primary) | 2.0% | [0.9%, 3.1%] | 25 |
Binary size
This benchmark run did not return any relevant results for this metric.
Bootstrap: 777.275s -> 777.884s (0.08%)
Artifact size: 328.69 MiB -> 328.73 MiB (0.01%)
This comment has been minimized.
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-page Symbol interner
This is a simple and only semi-correct impl that works well only on 64-bit platforms where we can lazily allocate zeroed pages (i.e., not actually use up memory for zeroed pages), meaning that we can grab a 4GB chunk of memory for all interned strings and then just offset allocate into that.
This already fixes unsoundness in the Symbol::as_str by leaking that 4gb memory chunk, but could also be faster (TBD).
A rewrite supporting ~any platform should be pretty straightforward by sharding the allocation and manually lazily allocating it in chunks.
r? @ghost -- this impl can't land due to being non-workable on Windows, opening for initial perf run
☀️ Try build successful - checks-actions
Build commit: ad08842 (ad088427a5424ba3008dd911900b7aff41db7be2)
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-chunk Symbol interner
This fixes the unsoundness in Symbol::as_str, while also making it a bit faster (replacing locking with a single atomic load).
☀️ Try build successful - checks-actions
Build commit: d500a34 (d500a3496173ce0986f80d00bcdd0866741b1ed6)
This comment has been minimized.
Finished benchmarking commit (d500a34): comparison URL.
Overall result: ❌✅ regressions and improvements - please read the text below
Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.
Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.
@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression
Instruction count
This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 0.4% | [0.1%, 0.9%] | 40 |
| Regressions ❌ (secondary) | 0.5% | [0.2%, 1.0%] | 44 |
| Improvements ✅ (primary) | -2.6% | [-2.6%, -2.6%] | 1 |
| Improvements ✅ (secondary) | -0.2% | [-0.2%, -0.2%] | 1 |
| All ❌✅ (primary) | 0.3% | [-2.6%, 0.9%] | 41 |
Max RSS (memory usage)
Results (primary 0.7%, secondary -0.7%)
This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 3.3% | [2.2%, 6.9%] | 6 |
| Regressions ❌ (secondary) | 3.0% | [2.6%, 4.7%] | 9 |
| Improvements ✅ (primary) | -1.6% | [-2.9%, -0.5%] | 7 |
| Improvements ✅ (secondary) | -3.2% | [-7.8%, -0.8%] | 13 |
| All ❌✅ (primary) | 0.7% | [-2.9%, 6.9%] | 13 |
Cycles
Results (primary -2.4%, secondary -2.8%)
This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | - | - | 0 |
| Regressions ❌ (secondary) | - | - | 0 |
| Improvements ✅ (primary) | -2.4% | [-2.4%, -2.4%] | 1 |
| Improvements ✅ (secondary) | -2.8% | [-2.8%, -2.8%] | 1 |
| All ❌✅ (primary) | -2.4% | [-2.4%, -2.4%] | 1 |
Binary size
This benchmark run did not return any relevant results for this metric.
Bootstrap: 779.019s -> 777.464s (-0.20%)
Artifact size: 365.14 MiB -> 365.15 MiB (0.00%)
This comment has been minimized.
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-chunk Symbol interner
This fixes the unsoundness in Symbol::as_str, while also making it a bit faster (replacing locking with a single atomic load).
This comment has been minimized.
This comment has been minimized.
Status: This is awaiting some action (such as code changes or more information) from the author.
and removed S-waiting-on-review
Status: Awaiting review from the assignee but also interested parties.
labels
This fixes unsoundness in the Symbol::as_str by leaking the chunks (via the static memory).
This comment has been minimized.
bors added a commit to rust-lang-ci/rust that referenced this pull request
…try>
Lazy-chunk Symbol interner
This fixes the unsoundness in Symbol::as_str, while also making it a bit faster (replacing locking with a single atomic load).
☀️ Try build successful - checks-actions
Build commit: a310201 (a310201bb079a11b845f5302af2fcacd5ceef80f)
This comment has been minimized.
Finished benchmarking commit (a310201): comparison URL.
Overall result: ❌✅ regressions and improvements - please read the text below
Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.
Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.
@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression
Instruction count
This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 0.3% | [0.2%, 0.4%] | 4 |
| Regressions ❌ (secondary) | 0.3% | [0.2%, 0.4%] | 9 |
| Improvements ✅ (primary) | -0.6% | [-2.3%, -0.2%] | 6 |
| Improvements ✅ (secondary) | -0.4% | [-1.1%, -0.2%] | 12 |
| All ❌✅ (primary) | -0.2% | [-2.3%, 0.4%] | 10 |
Max RSS (memory usage)
Results (primary -0.6%, secondary -1.8%)
This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | 2.0% | [0.5%, 3.1%] | 4 |
| Regressions ❌ (secondary) | 3.8% | [2.5%, 4.4%] | 6 |
| Improvements ✅ (primary) | -2.7% | [-6.1%, -1.0%] | 5 |
| Improvements ✅ (secondary) | -3.4% | [-9.6%, -1.2%] | 21 |
| All ❌✅ (primary) | -0.6% | [-6.1%, 3.1%] | 9 |
Cycles
Results (primary -2.2%, secondary 8.9%)
This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.
| mean | range | count | |
|---|---|---|---|
| Regressions ❌ (primary) | - | - | 0 |
| Regressions ❌ (secondary) | 8.9% | [8.2%, 9.6%] | 2 |
| Improvements ✅ (primary) | -2.2% | [-2.2%, -2.2%] | 1 |
| Improvements ✅ (secondary) | - | - | 0 |
| All ❌✅ (primary) | -2.2% | [-2.2%, -2.2%] | 1 |
Binary size
This benchmark run did not return any relevant results for this metric.
Bootstrap: 771.649s -> 771.642s (-0.00%)
Artifact size: 365.07 MiB -> 365.08 MiB (0.00%)
Labels
Performance regression.
Status: This is awaiting some action (such as code changes or more information) from the author.
Relevant to the compiler team, which will review and decide on the PR/issue.