Add core::stream::from_iter by yoshuawuyts · Pull Request #81797 · rust-lang/rust (original) (raw)

Tracking issue: #81798

This_ PR implements std::stream::from_iter, as outlined in the "Converting an Iterator to a Stream" section of the Stream RFC. This function enables converting an Iterator to a Stream by wrapping each item in the iterator with a Poll::Ready instance.

r? @tmandry

cc/ @rust-lang/libs @rust-lang/wg-async-foundations

Example

Being able to convert from an iterator into a stream is useful when refactoring from iterative loops into a more functional adapter-based style. This is fairly common when using more complex filter / map / find chains. In its basic form this conversion looks like this:

before

let mut output = vec![]; for item in my_vec { let out = do_io(item).await?; output.push(out); }

after

use std::stream;

let output = stream::from_iter(my_vec.iter()) .map(async |item| do_io(item).await) .collect()?;

Having a way to convert an Iterator to a Stream is essential in enabling this flow.

Implementation Notes

This PR makes use of unsafe {} to pin an item. Currently we're having conversations on the libs stream in Zulip how to bring pin-project in as a dependency to core so we can omit the unsafe {}.

This PR also includes a documentation block which references Stream::next which currently doesn't exist in the stdlib (originally included in the RFC and PR, but later omitted because of an unresolved issue). stream::from_iter can't stabilize before Stream does, and there's still a chance we may stabilize Stream with a next method. So this PR includes documentation referencing that method, which we can remove as part of stabilization if by any chance we don't have Stream::next.

Alternatives Considered

impl IntoStream for T: IntoIterator

An obvious question would be whether we could make it so every iterator can automatically be converted into a stream by calling into_stream on it. The answer is: "perhaps, but it could cause type issues". Types like std::collections may want to opt to create manual implementations for IntoStream and IntoIter, which wouldn't be possible if it was implemented through a catch-all trait.

Possibly an alternative such as impl IntoStream for T: Iterator could work, but it feels somewhat restrictive. In the end, converting an iterator to a stream is likely to be a bit of a niche case. And even then, adding a standalone function to convert an Iterator into a Stream would not be mutually exclusive with a blanket implementation.

Naming

The exact name can be debated in the period before stabilization. But I've chosen stream::from_iter rather than stream::iter because we are creating a stream from an iterator rather than iterating a stream. We also expect to add a stream counterpart to iter::from_fn later on (blocked on async closures), and having stream::from_fn and stream::from_iter would feel like a consistent pair. It also has prior art in async_std::stream::from_iter.

Future Directions

Stream conversions for collections

This is a building block towards implementing stream/stream_mut/into_stream methods for std::collections, std::vec, and more. This would allow even quicker refactorings from using loops to using iterator adapters by omitting the import altogether:

before

use std::stream;

let output = stream::from_iter(my_vec.iter()) .map(async |item| do_io(item).await) .collect()?;

after

let output = my_vec .stream() .map(async |item| do_io(item).await) .collect()?;