rust (original) (raw)

Currently, it's very challenging to perform a sandboxed opt-dist
bootstrap because the tool requires rustc-perf to be present, but
there is no proper management/tracking of it. Instead, a specific commit
is hardcoded where it is needed, and a non-checksummed zip is fetched
ad-hoc. This happens in two places:

src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile:

ENV PERF_COMMIT 4f313add609f43e928e98132358e8426ed3969ae RUN curl -LS -o perf.zip https://ci-mirrors.rust-lang.org/rustc/rustc-perf-$PERF_COMMIT.zip &&
unzip perf.zip &&
mv rustc-perf-$PERF_COMMIT rustc-perf &&
rm perf.zip

src/tools/opt-dist/src/main.rs

// FIXME: add some mechanism for synchronization of this commit SHA with // Linux (which builds rustc-perf in a Dockerfile) // rustc-perf version from 2023-10-22 const PERF_COMMIT: &str = "4f313add609f43e928e98132358e8426ed3969ae";

let url = format!("https://ci-mirrors.rust-lang.org/rustc/rustc-perf-{PERF_COMMIT}.zip"); let client = reqwest::blocking::Client::builder() .timeout(Duration::from_secs(60 * 2)) .connect_timeout(Duration::from_secs(60 * 2)) .build()?; let response = retry_action( || Ok(client.get(&url).send()?.error_for_status()?.bytes()?.to_vec()), "Download rustc-perf archive", 5, )?;

This causes a few issues:

Maintainers need to be careful to bump PERF_COMMIT in both places
every time
In order to run opt-dist in a sandbox, you need to provide your own
rustc-perf (feat(tools/opt-dist): allow local builds to specify a rustc-perf checkout #125125), but to
figure out which commit to provide you need to grep the Dockerfile
Even if you manage to provide the correct rustc-perf, its
dependencies are not included in the vendor/ dir created during
dist, so it will fail to build from the published source tarballs
It is hard to provide any level of automation around updating the
rustc-perf in use, leading to staleness

Fundamentally, this means rustc-src tarballs no longer contain
everything you need to bootstrap Rust, and packagers hoping to leverage
opt-dist need to go out of their way to keep track of this "hidden"
dependency on rustc-perf.

This change adds rustc-perf as a git submodule, pinned to the current
PERF_COMMIT 4f313ad. Subsequent
commits ensure the submodule is initialized when necessary, and make use
of it in opt-dist.