High level plan for the book · Issue #224 · rust-lang/async-book (original) (raw)

I'm dumping some thoughts about goals and plans for the updated book. Let's discuss!

cc @rust-lang/wg-async

Audience

I would like to cater to a pretty full range of audiences - from first time async programmers to advanced users who are implementing their own runtimes and contributing to the work of the async WG. Obviously we can't cover everything of interest to the latter group though.

Some audience considerations I think are worth calling out:

• I'm going to assume the reader is a competent Rust programmer and not teach any Rust other than the async parts
  • I'll assume I don't need to teach Rust's non-async concurrency primitives, libs, etc., but at least for the beginner-focussed sections will not assume a deep understanding of concurrent programming.
• Beginners may have no async programming experience or may have async experience in other languages. There are some important points which are especially important to the latter group. I'll aim to isolate these to make them easier to find and also so as not to interupt the narrative flow too much (since these points are likely to be important to all readers that means some duplication, but I think the different emphasis will be useful).
• At the intermediate to advanced level, I think there is a distinction between readers who want to deep vs broad. E.g., readers who want to implement a runtime or are just curious will want many of the 'implementation details' of async programming, whereas readers who want to use async Rust to solve difficult problems might be more likely to broaden their knowledge to include tools for debugging, third party crates, or design patterns.

Goals

• Provide an approachable and accessible introduction to async programming using good default choices and high-level patterns.
• Extend that introduction to cover the core parts of async programming without requiring the reader to understand the implementation.
• Cover advanced topics in detail with a preference for solutions which are possible today rather than what might happen in the future or what can't be done.
• Make the detailed topics accessible and discoverable by providing multiple entry points.

Structure

I'm pretty sure that we should have:

• Part 1: a tutorial style guide from zero to early intermediate level. Covers everything needed to be a competent programmer using async Rust. Would be practical, mostly use async/await, use Tokio, and avoid implementation details. I think it will need to cover using Futures and alternate runtimes. I'm not sure whether we need to cover implementing futures, but I fear that we might have to in order to cover async iterators.
• Part 2: advanced topics (which would also cover a lot of intermediate topics). Each chapter would be quite self-contained and stand-alone. Not designed to be read in order.
• Several ways to access the above information, for example, a detailed index, FAQs, a glossary with references, a topic index, etc.

The things I'm not at all sure about include:

• What should be in part 2 (although I have a bunch of ideas, and this we can definitely iterate on, so I don't think we need to plan that up front)
• Would case studies be useful?
• Many of the advanced topics I've thought of are actually footguns which really everyone should know about even if they aren't likely to come across any particular one on any given day. I'm not sure how to direct beginner readers to these sections without making a long, boring book which covers too much. (I.e., I think readers may skim over chapters in part 2 because they don't sound important, but actually they might be!)
• Should part 2 be divided into sections which are useful for implementers and sections for users? This sounds like a useful distinction to make, however, there are many topics which are important to both, e.g., around cancellation safety there are definitely issues which users should know about, but also this is a topic which is important for implementers since it affects much ongoing design work. Possibly there should be two sections (one for users, one for implementers)? Also there is some overlap between the groups in people who implement a runtime but not work in rustc or std, etc.
• How much we should reflect the current status and things which might work in the future (especially in the near future, e.g., things that are on nightly or which have an interim solution.

Part 1

My very rough idea for topics for part 1 is (there would be a separate intro/front matter; the list is not one bullet per chapter/section):

• Concurrency, parallelism, processes, threads, green threads, and async tasks
  • cooperative multitasking
• async and await
  • async main
    * cf explicit init
  • lifetimes and borrowing
  • errors
  • tasks, spawn, join
    * (tasks vs futures) vs threads
    * we'll need to explain the abstract concept of a future here, but won't go into detail until later
  • testing
  • async traits
  • async blocks and async closures
• IO and blocking, long-running compute
  • async IO traits
• Monitoring, debugging, tracing, profiling
• channels and locking
• concurrency techniques
  • select, race, etc.
• timers and other utilities
• futures
  • libs and combinators, etc, futures crate
  • IntoFuture
  • only progress when polled
• destruction and cleaning up
• role of the runtime and runtime issues
  • work stealing, thread pools, and bounds on futures (this might be better in the 'alternate runtimes section of part 2)
  • possibility of using different runtimes
• streams/async iterators

Part 2

Some of the topics I think should be covered include (not in order, necessarily, though I don't intend for there to be a narrative order, there should be some organisation):

• implementing futures and streams
• Alternate runtimes
• Implementing your own runtime
  • understanding the runtime model
    * reactors/executors/event loops
    * more detail on tasks
  • executors, wakers, and other types
• async in sync, sync in async, etc
• completion IO and io_uring
• design patterns and idioms
  • 'fluent builder'
  • structured concurrency (maybe deserves its own section)
  • multiple runtimes or multiple executors from the same runtime
• cancellation
• starvation
• buffering and zero-copy
• Pinning
  • pin project, pin project lite crates
• Compare to other languages' solutions in detail
• How async/await is implemented in the compiler as a state machine