Fetch GitHub commits by long hash more efficiently by dtolnay · Pull Request #10079 · rust-lang/cargo (original) (raw)

Tested with the following Cargo.toml:

[package] name = "repro" version = "0.0.0" edition = "2021" publish = false

[dependencies] cargo = { git = "https://github.com/rust-lang/cargo", rev = "b30694b4d9b29141298870b7993e9aee10940524" }

$ rm -rf ~/.cargo/git/db/cargo-* ~/.cargo/git/checkouts/cargo-* $ time $CARGO generate-lockfile $ du -shc ~/.cargo/git/db/cargo-* ~/.cargo/git/checkouts/cargo-*

Using current cargo from the most recent nightly, the generate-lockfile command downloads 69704 git objects in 7.0 seconds, consuming 41 MB on disk.

Using cargo built from this PR by cargo build --release, the same command downloads 21481 objects in 2.2 seconds, consuming 17 MB on disk.

Once libgit2 is able to do shallow clones (libgit2/libgit2#3058) this can be even more of a speedup. Using command-line git (which does not use libgit2) and time git fetch --depth=1 https://github.com/rust-lang/cargo b30694b4d9b29141298870b7993e9aee10940524 indicates that it downloads just 262 objects in 1.1 seconds.