Nondeterminism encountered in metadata for doc_link_resolutions (original) (raw)

Fuchsia has a test that checks whether rustc output is deterministic, and we find that it fairly reliably reorders bits of the metadata output for doc_link_resolutions for one particular crate in our build.

I determined this by picking out some changes of d5bd169 to add RUSTC_FILE_ENCODER_PANIC_AT_OFFSET and examining the backtrace at an offset that differs in one failure in the hexdump below. I also grepped for the strings that appear in all of the differences.

   4: <rustc_serialize::opaque::FileEncoder>::write_all
             at /code/rust/compiler/rustc_serialize/src/opaque.rs:162:13
   5: <rustc_serialize::opaque::FileEncoder as rustc_serialize::serialize::Encoder>::emit_raw_bytes
             at /code/rust/compiler/rustc_serialize/src/opaque.rs:279:9
   6: <rustc_metadata::rmeta::encoder::EncodeContext as rustc_serialize::serialize::Encoder>::emit_raw_bytes
             at /code/rust/compiler/rustc_metadata/src/rmeta/encoder.rs:79:25
   7: <rustc_metadata::rmeta::encoder::EncodeContext as rustc_serialize::serialize::Encoder>::emit_str
             at /code/rust/compiler/rustc_serialize/src/serialize.rs:69:9
   8: <rustc_metadata::rmeta::encoder::EncodeContext as rustc_span::SpanEncoder>::encode_symbol
             at /code/rust/compiler/rustc_metadata/src/rmeta/encoder.rs:211:26
   9: <rustc_span::symbol::Symbol as rustc_serialize::serialize::Encodable<rustc_metadata::rmeta::encoder::EncodeContext>>::encode
             at /code/rust/compiler/rustc_span/src/lib.rs:1185:9
  10: <(rustc_span::symbol::Symbol, rustc_hir::def::Namespace) as rustc_serialize::serialize::Encodable<rustc_metadata::rmeta::encoder::EncodeContext>>::encode
             at /code/rust/compiler/rustc_serialize/src/serialize.rs:424:19
  11: <std::collections::hash::map::HashMap<(rustc_span::symbol::Symbol, rustc_hir::def::Namespace), core::option::Option<rustc_hir::def::Res<rustc_ast::node_id::NodeId>>, core::hash::BuildHasherDefault<rustc_hash::FxHasher>> as rustc_serialize::serialize::Encodable<rustc_metadata::rmeta::encoder::EncodeContext>>::encode
             at /code/rust/compiler/rustc_serialize/src/serialize.rs:598:13
  12: <rustc_data_structures::unord::UnordMap<(rustc_span::symbol::Symbol, rustc_hir::def::Namespace), core::option::Option<rustc_hir::def::Res<rustc_ast::node_id::NodeId>>> as rustc_serialize::serialize::Encodable<rustc_metadata::rmeta::encoder::EncodeContext>>::encode
             at /code/rust/compiler/rustc_data_structures/src/unord.rs:420:39
  13: <rustc_metadata::rmeta::encoder::EncodeContext>:🦥:<rustc_data_structures::unord::UnordMap<(rustc_span::symbol::Symbol, rustc_hir::def::Namespace), core::option::Option<rustc_hir::def::Res<rustc_ast::node_id::NodeId>>>, &rustc_data_structures::unord::UnordMap<(rustc_span::symbol::Symbol, rustc_hir::def::Namespace), core::option::Option<rustc_hir::def::Res<rustc_ast::node_id::NodeId>>>>
             at /code/rust/compiler/rustc_metadata/src/rmeta/encoder.rs:466:9
  14: <rustc_metadata::rmeta::encoder::EncodeContext>::encode_def_ids
             at /code/rust/compiler/rustc_metadata/src/rmeta/encoder.rs:1516:13
  15: <rustc_metadata::rmeta::encoder::EncodeContext>::encode_crate_root::{closure#16}
             at /code/rust/compiler/rustc_metadata/src/rmeta/encoder.rs:639:33
  16: <rustc_metadata::rmeta::encoder::EncodeContext>::encode_crate_root
             at /code/rust/compiler/rustc_metadata/src/rmeta/encoder.rs:598:27
  17: rustc_metadata::rmeta::encoder::encode_metadata
             at /code/rust/compiler/rustc_metadata/src/rmeta/encoder.rs:2266:16
  18: rustc_metadata::fs::encode_and_write_metadata
             at /code/rust/compiler/rustc_metadata/src/fs.rs:65:13
  19: rustc_interface::passes::start_codegen
             at /code/rust/compiler/rustc_interface/src/passes.rs:929:44

Lines are from compiler commit 7f2fc33. Source code of the crate is available here.
Full backtrace

In the following diff, the first version (red) is more common.

--- obj/src/starnix/kernel/libstarnix_core.rlib.llvm-objdump.local 2024-04-22 16:14:34.523061172 +0000 +++ obj/src/starnix/kernel/libstarnix_core.rlib.llvm-objdump.remote 2024-04-22 16:14:51.439162227 +0000 @@ -355421,55 +355421,55 @@ 56c560 52d526d5 1ddc7edc 2ba5a001 e0a001a3 R.&...~.+....... 56c570 a001e315 cb268525 a646930f d336a775 .....&.%.F...6.u 56c580 c4558a13 b908ee24 c562c72a c4601502 .U.....$.b.*.`..