Rewrite handling of universe-leaking placeholder regions into outlives constraints by amandasystems · Pull Request #123720 · rust-lang/rust (original) (raw)

rustbot added S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

T-compiler

Relevant to the compiler team, which will review and decide on the PR/issue.

labels

Apr 10, 2024

amandasystems changed the title[WIP] Rewrite handling of universe-leaking placeholder regions into outlives constraints Rewrite handling of universe-leaking placeholder regions into outlives constraints

Apr 12, 2024

rustbot added S-waiting-on-author

Status: This is awaiting some action (such as code changes or more information) from the author.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Apr 12, 2024

bors added a commit to rust-lang-ci/rust that referenced this pull request

Apr 12, 2024

…=

Rewrite handling of universe-leaking placeholder regions into outlives constraints

This commit prepares for Polonius by moving handling of leak check/universe errors out of the inference step by rewriting any universe error into an outlives-static constraint.

This variant is a work in progress but seems to pass most tests.

Note that a few debug assertions no longer hold; a few extra eyes on those changes are appreciated!

bors added S-waiting-on-bors

Status: Waiting on bors to run and complete tests. Bors will change the label on completion.

and removed S-waiting-on-review

Status: Awaiting review from the assignee but also interested parties.

labels

Jul 2, 2024

lqd mentioned this pull request

Sep 1, 2024

21 tasks

bors added a commit to rust-lang-ci/rust that referenced this pull request

Sep 27, 2024

…mpletely, r=

[WIP] Remove placeholders completely

This PR does shotgun surgery on borrowck to remove all special handling of placeholders, completely replacing them with a preprocessing step that rewrites placeholder leaks into constraints, removing constraint propagation of placeholders and all logic used to detect placeholder violations during error reporting. This finishes what rust-lang#123720 started.

The new method works like this:

1. during SCC construction, some information about SCC membership and reachability is retained
2. just after SCC construction, a constraint r - (from: to_invalid) - > 'static is added when r is the representative of an SCC and
   1. that SCC either has had its universe shrunk because it reaches a region with a smaller one (in which case to_invalid is the smallest-universed region it reaches), or if it reaches a region with a too large universe that isn't part of the SCC (in which case to_invalid is the region with a too large universe). In either case, from is also r.
3. some region reaches in r's SCC reaches another placeholder, reached, in which case the added constraint is r -> (reaches: reached) 'static. Through clever choice of defaults (chosing minimum elements), reached will be r if at all possible.

When tracing errors for diagnostics one of these special constraints along a path are treated much like a HTTP redirect: if we are explaining from: to and reach an edge with reaches: invalid we stop the search and start following reaches: invalid instead. When doing this the implicit edges x: 'static for every region x are ignored, since the search would otherwise be able to cheat by going through 'static and re-find the same edge again.

A bunch of optimisations are possible:

• Conservatively add constraints, e.g. one per SCC. May worsen error tracing!
• as a final pass, allow fusing the annotations for the SCC after adding the extra constraints to remove unnecessary information and save memory. This could be done cheaply since we already iterate over the entire SCC.
• currently, if constraints are added the entire set of SCCs are recomputed. This is of course rather wasteful, and we could maybe do better.

There are a bunch of rather nice bonuses:

• We now don't need to expose region indices in MirTypeckRegionConstraints to the entire crate. The only entry point is placeholder_region() so correctness of the indices is now guaranteed
• A lot of things that were previously iterations over lists is now a single lookup
• The constraint graph search functions are simple and at least one of them can now take a proper region as target rather than a predicate function. The only case that needs the predicate argument to find_constraint_path_to() is find_sub_region_live_at(), which may or may not be possible to work around.

r​? nikomatsakis

bors added a commit to rust-lang-ci/rust that referenced this pull request

Oct 1, 2024

…mpletely, r=

[WIP] Remove placeholders completely

This PR does shotgun surgery on borrowck to remove all special handling of placeholders, completely replacing them with a preprocessing step that rewrites placeholder leaks into constraints, removing constraint propagation of placeholders and all logic used to detect placeholder violations during error reporting. This finishes what rust-lang#123720 started.

The new method works like this:

1. during SCC construction, some information about SCC membership and reachability is retained
2. just after SCC construction, a constraint r - (from: to_invalid) - > 'static is added when r is the representative of an SCC and
   1. that SCC either has had its universe shrunk because it reaches a region with a smaller one (in which case to_invalid is the smallest-universed region it reaches), or if it reaches a region with a too large universe that isn't part of the SCC (in which case to_invalid is the region with a too large universe). In either case, from is also r.
3. some region reaches in r's SCC reaches another placeholder, reached, in which case the added constraint is r -> (reaches: reached) 'static. Through clever choice of defaults (chosing minimum elements), reached will be r if at all possible.

When tracing errors for diagnostics one of these special constraints along a path are treated much like a HTTP redirect: if we are explaining from: to and reach an edge with reaches: invalid we stop the search and start following reaches: invalid instead. When doing this the implicit edges x: 'static for every region x are ignored, since the search would otherwise be able to cheat by going through 'static and re-find the same edge again.

A bunch of optimisations are possible:

• Conservatively add constraints, e.g. one per SCC. May worsen error tracing!
• as a final pass, allow fusing the annotations for the SCC after adding the extra constraints to remove unnecessary information and save memory. This could be done cheaply since we already iterate over the entire set of SCCs.
• currently, if constraints are added the entire set of SCCs are recomputed. This is of course rather wasteful, and we could do better. Especially since SCCs are added in dependency order. This would require a fully separate SCC module since the dynamic SCC combo we'd need now shares almost no properties with regular SCC computation. Given that this is meant to be a temporary work-around, that seems like too much work.

There are a bunch of rather nice bonuses:

• We now don't need to expose region indices in MirTypeckRegionConstraints to the entire crate. The only entry point is placeholder_region() so correctness of the indices is now guaranteed
• A lot of things that were previously iterations over lists is now a single lookup
• The constraint graph search functions are simple and at least one of them can now take a proper region as target rather than a predicate function. The only case that needs the predicate argument to find_constraint_path_to() is find_sub_region_live_at(), which may or may not be possible to work around.

r​? nikomatsakis

bors added a commit to rust-lang-ci/rust that referenced this pull request

Oct 4, 2024

…mpletely, r=

[WIP] Remove placeholders completely

This PR does shotgun surgery on borrowck to remove all special handling of placeholders, completely replacing them with a preprocessing step that rewrites placeholder leaks into constraints, removing constraint propagation of placeholders and all logic used to detect placeholder violations during error reporting. This finishes what rust-lang#123720 started.

The new method works like this:

1. during SCC construction, some information about SCC membership and reachability is retained
2. just after SCC construction, a constraint r - (from: to_invalid) - > 'static is added when r is the representative of an SCC and
   1. that SCC either has had its universe shrunk because it reaches a region with a smaller one (in which case to_invalid is the smallest-universed region it reaches), or if it reaches a region with a too large universe that isn't part of the SCC (in which case to_invalid is the region with a too large universe). In either case, from is also r.
3. some region reaches in r's SCC reaches another placeholder, reached, in which case the added constraint is r -> (reaches: reached) 'static. Through clever choice of defaults (chosing minimum elements), reached will be r if at all possible.

When tracing errors for diagnostics one of these special constraints along a path are treated much like a HTTP redirect: if we are explaining from: to and reach an edge with reaches: invalid we stop the search and start following reaches: invalid instead. When doing this the implicit edges x: 'static for every region x are ignored, since the search would otherwise be able to cheat by going through 'static and re-find the same edge again.

A bunch of optimisations are possible:

• Conservatively add constraints, e.g. one per SCC. May worsen error tracing!
• as a final pass, allow fusing the annotations for the SCC after adding the extra constraints to remove unnecessary information and save memory. This could be done cheaply since we already iterate over the entire set of SCCs.
• currently, if constraints are added the entire set of SCCs are recomputed. This is of course rather wasteful, and we could do better. Especially since SCCs are added in dependency order. This would require a fully separate SCC module since the dynamic SCC combo we'd need now shares almost no properties with regular SCC computation. Given that this is meant to be a temporary work-around, that seems like too much work.

There are a bunch of rather nice bonuses:

• We now don't need to expose region indices in MirTypeckRegionConstraints to the entire crate. The only entry point is placeholder_region() so correctness of the indices is now guaranteed
• A lot of things that were previously iterations over lists is now a single lookup
• The constraint graph search functions are simple and at least one of them can now take a proper region as target rather than a predicate function. The only case that needs the predicate argument to find_constraint_path_to() is find_sub_region_live_at(), which may or may not be possible to work around.

r​? nikomatsakis

bors added a commit to rust-lang-ci/rust that referenced this pull request

Oct 13, 2024

…mpletely, r=

[WIP] Remove placeholders completely

This PR does shotgun surgery on borrowck to remove all special handling of placeholders, completely replacing them with a preprocessing step that rewrites placeholder leaks into constraints, removing constraint propagation of placeholders and all logic used to detect placeholder violations during error reporting. This finishes what rust-lang#123720 started.

The new method works like this:

1. during SCC construction, some information about SCC membership and reachability is retained
2. just after SCC construction, a constraint r - (from: to_invalid) - > 'static is added when r is the representative of an SCC and
   1. that SCC either has had its universe shrunk because it reaches a region with a smaller one (in which case to_invalid is the smallest-universed region it reaches), or if it reaches a region with a too large universe that isn't part of the SCC (in which case to_invalid is the region with a too large universe). In either case, from is also r.
3. some region reaches in r's SCC reaches another placeholder, reached, in which case the added constraint is r -> (reaches: reached) 'static. Through clever choice of defaults (chosing minimum elements), reached will be r if at all possible.

When tracing errors for diagnostics one of these special constraints along a path are treated much like a HTTP redirect: if we are explaining from: to and reach an edge with reaches: invalid we stop the search and start following reaches: invalid instead. When doing this the implicit edges x: 'static for every region x are ignored, since the search would otherwise be able to cheat by going through 'static and re-find the same edge again.

A bunch of optimisations are possible:

• Conservatively add constraints, e.g. one per SCC. May worsen error tracing!
• as a final pass, allow fusing the annotations for the SCC after adding the extra constraints to remove unnecessary information and save memory. This could be done cheaply since we already iterate over the entire set of SCCs.
• currently, if constraints are added the entire set of SCCs are recomputed. This is of course rather wasteful, and we could do better. Especially since SCCs are added in dependency order. This would require a fully separate SCC module since the dynamic SCC combo we'd need now shares almost no properties with regular SCC computation. Given that this is meant to be a temporary work-around, that seems like too much work.

There are a bunch of rather nice bonuses:

• We now don't need to expose region indices in MirTypeckRegionConstraints to the entire crate. The only entry point is placeholder_region() so correctness of the indices is now guaranteed
• A lot of things that were previously iterations over lists is now a single lookup
• The constraint graph search functions are simple and at least one of them can now take a proper region as target rather than a predicate function. The only case that needs the predicate argument to find_constraint_path_to() is find_sub_region_live_at(), which may or may not be possible to work around.

r​? nikomatsakis

bors added a commit to rust-lang-ci/rust that referenced this pull request

Nov 6, 2024

…mpletely, r=

[WIP] Remove placeholders completely

This PR does shotgun surgery on borrowck to remove all special handling of placeholders, completely replacing them with a preprocessing step that rewrites placeholder leaks into constraints, removing constraint propagation of placeholders and all logic used to detect placeholder violations during error reporting. This finishes what rust-lang#123720 started.

The new method works like this:

1. during SCC construction, some information about SCC membership and reachability is retained
2. just after SCC construction, a constraint r - (from: to_invalid) - > 'static is added when r is the representative of an SCC and
   1. that SCC either has had its universe shrunk because it reaches a region with a smaller one (in which case to_invalid is the smallest-universed region it reaches), or if it reaches a region with a too large universe that isn't part of the SCC (in which case to_invalid is the region with a too large universe). In either case, from is also r.
3. some region reaches in r's SCC reaches another placeholder, reached, in which case the added constraint is r -> (reaches: reached) 'static. Through clever choice of defaults (chosing minimum elements), reached will be r if at all possible.

When tracing errors for diagnostics one of these special constraints along a path are treated much like a HTTP redirect: if we are explaining from: to and reach an edge with reaches: invalid we stop the search and start following reaches: invalid instead. When doing this the implicit edges x: 'static for every region x are ignored, since the search would otherwise be able to cheat by going through 'static and re-find the same edge again.

A bunch of optimisations are possible:

• Conservatively add constraints, e.g. one per SCC. May worsen error tracing!
• as a final pass, allow fusing the annotations for the SCC after adding the extra constraints to remove unnecessary information and save memory. This could be done cheaply since we already iterate over the entire set of SCCs.
• currently, if constraints are added the entire set of SCCs are recomputed. This is of course rather wasteful, and we could do better. Especially since SCCs are added in dependency order. This would require a fully separate SCC module since the dynamic SCC combo we'd need now shares almost no properties with regular SCC computation. Given that this is meant to be a temporary work-around, that seems like too much work.

There are a bunch of rather nice bonuses:

• We now don't need to expose region indices in MirTypeckRegionConstraints to the entire crate. The only entry point is placeholder_region() so correctness of the indices is now guaranteed
• A lot of things that were previously iterations over lists is now a single lookup
• The constraint graph search functions are simple and at least one of them can now take a proper region as target rather than a predicate function. The only case that needs the predicate argument to find_constraint_path_to() is find_sub_region_live_at(), which may or may not be possible to work around.

r​? nikomatsakis

bors added a commit to rust-lang-ci/rust that referenced this pull request

Nov 15, 2024

…mpletely, r=

[WIP] Remove placeholders completely

This PR does shotgun surgery on borrowck to remove all special handling of placeholders, completely replacing them with a preprocessing step that rewrites placeholder leaks into constraints, removing constraint propagation of placeholders and all logic used to detect placeholder violations during error reporting. This finishes what rust-lang#123720 started.

The new method works like this:

1. during SCC construction, some information about SCC membership and reachability is retained
2. just after SCC construction, a constraint r - (from: to_invalid) - > 'static is added when r is the representative of an SCC and
   1. that SCC either has had its universe shrunk because it reaches a region with a smaller one (in which case to_invalid is the smallest-universed region it reaches), or if it reaches a region with a too large universe that isn't part of the SCC (in which case to_invalid is the region with a too large universe). In either case, from is also r.
3. some region reaches in r's SCC reaches another placeholder, reached, in which case the added constraint is r -> (reaches: reached) 'static. Through clever choice of defaults (chosing minimum elements), reached will be r if at all possible.

When tracing errors for diagnostics one of these special constraints along a path are treated much like a HTTP redirect: if we are explaining from: to and reach an edge with reaches: invalid we stop the search and start following reaches: invalid instead. When doing this the implicit edges x: 'static for every region x are ignored, since the search would otherwise be able to cheat by going through 'static and re-find the same edge again.

A bunch of optimisations are possible:

• Conservatively add constraints, e.g. one per SCC. May worsen error tracing!
• as a final pass, allow fusing the annotations for the SCC after adding the extra constraints to remove unnecessary information and save memory. This could be done cheaply since we already iterate over the entire set of SCCs.
• currently, if constraints are added the entire set of SCCs are recomputed. This is of course rather wasteful, and we could do better. Especially since SCCs are added in dependency order. This would require a fully separate SCC module since the dynamic SCC combo we'd need now shares almost no properties with regular SCC computation. Given that this is meant to be a temporary work-around, that seems like too much work.

There are a bunch of rather nice bonuses:

• We now don't need to expose region indices in MirTypeckRegionConstraints to the entire crate. The only entry point is placeholder_region() so correctness of the indices is now guaranteed
• A lot of things that were previously iterations over lists is now a single lookup
• The constraint graph search functions are simple and at least one of them can now take a proper region as target rather than a predicate function. The only case that needs the predicate argument to find_constraint_path_to() is find_sub_region_live_at(), which may or may not be possible to work around.

r​? nikomatsakis

bors added a commit to rust-lang-ci/rust that referenced this pull request

Feb 28, 2025

…mpletely, r=

[WIP] Remove placeholders completely

This PR does shotgun surgery on borrowck to remove all special handling of placeholders, completely replacing them with a preprocessing step that rewrites placeholder leaks into constraints, removing constraint propagation of placeholders and most of the logic used to detect placeholder violations during error reporting. This finishes what rust-lang#123720 started. Due to the already sprawling scope of this PR, not all the breaks are clean. In particular, some of the error reporting code can almost certainly be further simplified.

The new method works like this:

1. during SCC construction, some information about SCC membership and reachability is retained
2. just after SCC construction, a constraint r - (from: to_invalid) - > 'static is added when r is the representative of an SCC and
   1. that SCC either has had its universe shrunk because it reaches a region with a smaller one (in which case to_invalid is the smallest-universed region it reaches), or if it reaches a region with a too large universe that isn't part of the SCC (in which case to_invalid is the region with a too large universe). In either case, from is also r.
3. some region reaches in r's SCC reaches another placeholder, reached, in which case the added constraint is r -> (reaches: reached) 'static. Through clever choice of defaults (chosing minimum elements), reached will be r if at all possible.

When tracing errors for diagnostics one of these special constraints along a path are treated much like a HTTP redirect: if we are explaining from: to and reach an edge with reaches: invalid we stop the search and start following reaches: invalid instead. When doing this the implicit edges x: 'static for every region x are ignored, since the search would otherwise be able to cheat by going through 'static and re-find the same edge again.

Type-tests are also rewritten to account for placeholder issues. In particular, if a bound implies : 'static, this is flagged using a new variant, and if a test is guaranteed to always fail (e.g. if an equals bound reaches different placeholders), it is replaced with a bound that is always unsatisfied.

A bunch of optimisations are possible:

• Conservatively add constraints, e.g. one per SCC. May worsen error tracing!
• as a final pass, allow fusing the annotations for the SCC after adding the extra constraints to remove unnecessary information and save memory. This could be done cheaply since we already iterate over the entire set of SCCs.
• currently, if constraints are added the entire set of SCCs are recomputed. This is of course rather wasteful, and we could do better. Especially since SCCs are added in dependency order. This would require a fully separate SCC module since the dynamic SCC combo we'd need now shares almost no properties with regular SCC computation. Given that this is meant to be a temporary work-around, that seems like too much work.

There are a bunch of rather nice bonuses:

• We now don't need to expose region indices in MirTypeckRegionConstraints to the entire crate. The only entry point is placeholder_region() so correctness of the indices is now guaranteed
• A lot of things that were previously iterations over lists is now a single lookup
• The constraint graph search functions are simple and at least one of them can now take a proper region as target rather than a predicate function. The only case that needs the predicate argument to find_constraint_path_to() is find_sub_region_live_at(), which may or may not be possible to work around.

r​? nikomatsakis

bors added a commit to rust-lang-ci/rust that referenced this pull request

Mar 7, 2025

…mpletely, r=

[WIP] Remove placeholders completely

This PR does shotgun surgery on borrowck to remove all special handling of placeholders, completely replacing them with a preprocessing step that rewrites placeholder leaks into constraints, removing constraint propagation of placeholders and most of the logic used to detect placeholder violations during error reporting. This finishes what rust-lang#123720 started. Due to the already sprawling scope of this PR, not all the breaks are clean. In particular, some of the error reporting code can almost certainly be further simplified.

The new method works like this:

1. during SCC construction, some information about SCC membership and reachability is retained
2. just after SCC construction, a constraint r - (from: to_invalid) - > 'static is added when r is the representative of an SCC and
   1. that SCC either has had its universe shrunk because it reaches a region with a smaller one (in which case to_invalid is the smallest-universed region it reaches), or if it reaches a region with a too large universe that isn't part of the SCC (in which case to_invalid is the region with a too large universe). In either case, from is also r.
3. some region reaches in r's SCC reaches another placeholder, reached, in which case the added constraint is r -> (reaches: reached) 'static. Through clever choice of defaults (chosing minimum elements), reached will be r if at all possible.

When tracing errors for diagnostics one of these special constraints along a path are treated much like a HTTP redirect: if we are explaining from: to and reach an edge with reaches: invalid we stop the search and start following reaches: invalid instead. When doing this the implicit edges x: 'static for every region x are ignored, since the search would otherwise be able to cheat by going through 'static and re-find the same edge again.

Type-tests are also rewritten to account for placeholder issues. In particular, if a bound implies : 'static, this is flagged using a new variant, and if a test is guaranteed to always fail (e.g. if an equals bound reaches different placeholders), it is replaced with a bound that is always unsatisfied.

A bunch of optimisations are possible:

• Conservatively add constraints, e.g. one per SCC. May worsen error tracing!
• as a final pass, allow fusing the annotations for the SCC after adding the extra constraints to remove unnecessary information and save memory. This could be done cheaply since we already iterate over the entire set of SCCs.
• currently, if constraints are added the entire set of SCCs are recomputed. This is of course rather wasteful, and we could do better. Especially since SCCs are added in dependency order. This would require a fully separate SCC module since the dynamic SCC combo we'd need now shares almost no properties with regular SCC computation. Given that this is meant to be a temporary work-around, that seems like too much work.

There are a bunch of rather nice bonuses:

• We now don't need to expose region indices in MirTypeckRegionConstraints to the entire crate. The only entry point is placeholder_region() so correctness of the indices is now guaranteed
• A lot of things that were previously iterations over lists is now a single lookup
• The constraint graph search functions are simple and at least one of them can now take a proper region as target rather than a predicate function. The only case that needs the predicate argument to find_constraint_path_to() is find_sub_region_live_at(), which may or may not be possible to work around.

r​? nikomatsakis