Stabilize associated type position impl Trait (ATPIT) by traviscross · Pull Request #120700 · rust-lang/rust (original) (raw)

Stabilization report

Summary

We are stabilizing #![feature(impl_trait_in_assoc_type)], commonly called either "associated type position impl Trait" (ATPIT) or "impl Trait in associated type" (ITIAT).

Among other things, this feature allows async blocks and async fn to be used in more places (instead of writing implementations of Future by hand), filling gaps in the story of async Rust.

Stabilizing ATPIT helps the many Rust users who have well known use cases, including, e.g., the Tower library (in particular, e.g., its Service trait) and its extensive ecosystem.

The theme of this stabilization report is simplicity. Through much work, we've found that we can stabilize a subset of RFC 2515 that solves the most demanded use cases while enabling a simple and robust implementation in the compiler, supporting efficient implementations in other tooling, and answering all previously-open language design questions in a principled way.

This is a partial stabilization of RFC 2515 and #63063.

What is stabilized

Summary of stabilization

We now allow impl Trait to appear in type position in associated type definitions within trait implementations.

For the first time, we can now use async blocks to implement IntoFuture without boxing. E.g.:

use core::future::{Future, IntoFuture};

struct AlwaysAsync42; impl IntoFuture for AlwaysAsync42 { type Output = impl Iterator<Item = impl Fn() -> u8>; type IntoFuture = impl Future<Output = Self::Output>; fn into_future(self) -> Self::IntoFuture { async { core::iter::repeat(|| 42) }} }

Just as with RPIT, impl Trait may appear syntactically multiple times within the type, and any item that does not register a hidden type for the opaque witnesses it as an opaque type. No items outside of the impl block may register hidden types for impl Trait opaques defined within. The rules for which items within the impl block may do so are a simple extension of the RPIT rules and are described below.

Opaqueness

When we say that an item can only use a type opaquely, what we mean is that item can only use the type via the interfaces defined by the traits that the type is declared to implement and those of the leaked auto traits (as with RPIT).

A type that can only be used opaquely by some items is known as an opaque type, or, in context, as simply an opaque. The concrete type or type constructor that underlies the opaque is known as the hidden type, or, in context, as the hidden.

When an item chooses a concrete hidden type to underlie some opaque, we say that it has registered a hidden for that opaque, or equivalently, that it has "defined the hidden type of the opaque", or, more loosely and in context, has "defined the opaque type".

We describe below which items may and must register a hidden type for a given opaque. Other items may only use such types opaquely.

Parallel to RPIT

ATPIT is the extension of RPIT to associated type definitions and trait implementations. Everything that is true about RPIT opaque types is true about ATPIT opaque types modulo that:

Syntactic reachability rule

The rule to decide whether an item may and must register a hidden type for an opaque is strictly syntactic and strictly local to the impl block. It's an extension from considering just the signature, as with RPIT, to also considering the definitions of associated types in the same impl.

Intuitively, we collect all impl Trait types within the same trait impl that are syntactically reachable from the signature of an item, considering only that item's signature and the associated type definitions within the impl block. We don't recurse into ADTs (e.g. the fields of a struct).

More precisely, the rule is as follows:

To determine the set of impl Trait opaque types for which an item may and must register a hidden type, from the signature (including from any where clauses), we:

  1. Collect all types and generic type arguments1 without normalization, and for each, we:
    • a. Collect RPIT-like ("existential") uses of impl Trait into the set of opaque types for which this item may and must register a hidden type.2
    • b. Recurse syntactically (i.e. normalize one step) into the definition of associated types from the same trait when all of the generic arguments (including Self) match, and from there, we repeat step 1.

Note that RPIT already performs steps 1 and 1a. The stabilization of ATPIT adds only step 1b.

Example of syntactic reachability rule

Following the rules for syntactic reachability, this works as we would expect:

use core::future::{Future, IntoFuture};

struct AlwaysAsync42; impl IntoFuture for AlwaysAsync42 { type Output = Box<dyn Iterator<Item = impl Fn() -> u8>>; type IntoFuture = impl Future<Output = ::Output>; fn into_future(self) -> Self::IntoFuture { async { Box::new(core::iter::repeat(|| 42u8)) as Box<dyn Iterator<Item = _>> }} }

May define == must define rule

If an impl Trait opaque type is syntactically reachable from the signature of an item according to the syntactic reachability rule, then a hidden type may and must be registered by that item for the opaque.

Items not satisfying this predicate may not register a hidden type for the opaque.

Sibling only rule

Only the syntactic items that are direct children of the impl block may register hidden types for an impl Trait opaque type in an associated type. Nested syntactic items within those items may not do so. As with RPIT, closures and async blocks, which are in some sense items but are not syntactic ones and which share the generics and where clauses of their parent, may register hidden types.

Coherence rule

Coherence checking is the process by which we ensure that no two trait impls may overlap. To decide that, we must have a rule for whether any two types may be equal, for the purposes of coherence.

During coherence checking, when an opaque type is related with another type we assume that the two types may be equal unless that other type does not fulfill any of the item bounds of the opaque.

Design principles

The design described in this document for ATPIT adheres to the following principles.

Convenience and minimal overhead for the common cases

We believe that impl Trait syntax in return position and in associated type position is for convenience and should have minimal overhead to use.

The design proposed for stabilization here solves common and important use cases well, conveniently, and with minimal overhead, similar to RPIT, by e.g. leaning on the syntactic reachability rule. Other use cases may prefer to wait for full type alias impl Trait.

Local reasoning

We believe that it should be easy to determine whether an item may and must register a hidden type of an impl Trait opaque type.

In certain edge cases, whether or not an item may register a hidden type for an opaque can affect method selection and type inference. It should therefore be straightforward for the user to look syntactically at the impl block only to determine whether or not an item may register a hidden type for any opaque. This is what the syntactic reachability rule achieves.

Crisp behavior

We believe that the behavior of impl Trait should be very crisp; if an item may register the hidden type for an opaque, then within that item the type should act exactly like an inference variable (existential quantification). If it cannot, then within that item the type should act like a generic type (universal quantification).

If items were allowed to register hidden types without being required to do so, then it is believed to be either difficult or impossible to maintain this kind of crispness in all circumstances. Consequently, this design adopts the "may define == must define" rule to preserve this crisp behavior.

Motivation

We long ago stabilized async fn and async { .. } blocks as these make writing async Rust more pleasant than having to implement Future everywhere by hand.

However, there's been a lingering problem with this story. The type of the futures returned by async fn and async { .. } blocks cannot be named, and we often need to name types to do useful things in Rust. This means that we can't use async everywhere that we might want to use it, and it means that if our dependencies do use it, that can create problems for us that we can't fix ourselves.

It's for this reason that using RPIT in public APIs has long been considered an anti-pattern. Using the recently-stabilized RPITIT and AFIT in public APIs carries these same pitfalls while adding a new one: the inability for callers to set bounds on these types.

It is these problems, in the context of interfaces defined as traits, that ATPIT addresses.

Using async in more places

Today, if we want to implement IntoFuture for a type so that it can be awaited using .await, we have no choice but to implement Future::poll for some wrapper type by hand so that it can be named in the associated type of IntoFuture.

With ATPIT, for the first time, we can use async { .. } blocks to implement IntoFuture. E.g.:

use core::future::{Future, IntoFuture};

struct AlwaysAsync42; impl IntoFuture for AlwaysAsync42 { type Output = impl Iterator<Item = impl Fn() -> u8>; type IntoFuture = impl Future<Output = Self::Output>; fn into_future(self) -> Self::IntoFuture { async { core::iter::repeat(|| 42) }} }

Naming types, expressing bounds, object-safety, and precise capturing

If someone were writing a Service-like trait today, now that AFIT and RPITIT are stable, that person may think to write it as follows so that async blocks or async fn could be used in the impl:

#![allow(async_fn_in_trait)]

pub trait Service { type Response; async fn call(&self) -> Self::Response; }

However, as compared with using associated types, that would create four problems for users of the trait:

  1. The trait would now not be object-safe.
  2. The type of the returned Future can't be named so as e.g. to store it in a struct.
  3. The type of the returned Future can't be named so as to set bounds on it, e.g. to require a Send, FusedFuture, or TryFuture bound.
  4. The type of the returned Future captures a lifetime we may not want to capture.

In the future, there may be other and better solutions to some of these problems (those solutions may in fact desugar essentially to ATPIT). But today, without ATPIT, trait authors face a dilemma. They must either accept all of these drawbacks or, alternatively, must accept that implementors of the trait will not be able to use async blocks and will have to write manual implementations of Future.

ATPIT offers us a way out of this dilemma. Trait authors can allow implementors to use async blocks (and conceivably, in the future, async fn) to implement the trait while preserving the object safety of the trait, allow users to name the type so as to store it and set bounds, and express precise capturing of type and lifetime generic parameters.

Open items

"Must define before use"

Add Projection/NormalizesTo goals for opaque types

Update the Rust Reference

Acknowledgments

The stabilization of ATPIT would not have been possible without, in particular, the ongoing and outstanding work of @oli-obk, who has been quietly pushing forward on all aspects of type alias impl Trait for years. Thanks are also due, in no small part, to @compiler-errors for pushing forward both on this work directly and on critical foundations which have made this work possible. Similarly, we can't say enough positive things about the work that @lcnr has been and is doing on the new trait solver; that work has shaped this proposal and is also what gives us confidence about the ability to support and extend this feature into the long term.

Separately, the author of this stabilization report thanks @oli-obk, @compiler-errors, @tmandry, and @nikomatsakis for their personal support on this work.

Thanks are due to the types team generally for helping develop the "Mini-TAIT" proposal that preceded this work. And thanks are of course due to the authors of RFC 1522, RFC 1951, RFC 2071, and RFC 2515, @Kimundi, @aturon, @cramertj, and @varkor, and to all those who contributed usefully to those designs and discussions.

Thanks to @nikomatsakis for setting out the design principles articulated in this document, and to @tmandry, @oli-obk, and @compiler-errors for reviewing drafts. All errors and omissions remain those of the author alone.

Footnotes

  1. ADTs with generic parameters, non-unit tuples, arrays, slices, pointers, references, function pointers, and impl Trait and dyn Trait types with generic parameters or associated types are type constructors. Traits with generic parameters or associated types, including the Fn* traits, are bounds constructors. When we say, "generic type arguments" above, we mean types that are used to parameterize these type constructors or bounds constructors, including for associated types. When we collect generic type arguments, we do so syntactically (i.e. without normalization and looking only at what is written) and recursively (i.e. within arbitrary levels of syntactic composition). ↩
  2. There are existing restrictions on where an RPIT impl Trait may appear within a type and within a signature. Those restrictions remain and are not being changed by this stabilization. ↩