reduce the amount of panics in {TokenStream, Literal}::from_str calls by cyrgani · Pull Request #147859 · rust-lang/rust (original) (raw)

Before this PR, calling TokenStream::from_str or Literal::from_str with an invalid argument would always cause a compile error, even if the TokenStream is not used afterwards at all.
This PR changes this so it returns a LexError instead in some cases.

This is very theoretically a breaking change, but the doc comment on the impl already says

/// NOTE: some errors may cause panics instead of returning `LexError`. We reserve the right to
/// change these errors into `LexError`s later.

Fixes some cases of #58736.