`cargo test` does not report failures when a test exits the process · Issue #87323 · rust-lang/rust (original) (raw)

Problem
I'm running cargo test with tests that fail, and it exits with code 0 and does not gather and report errors.

Steps

git clone https://github.com/apollographql/rover
git checkout avery/repro-tests
Notice there is a test at the root crate that has a single call to panic!("It fails")
Run cargo test, you should notice the FAILED test only if you scroll up to look for it, the message at the end makes you think the tests pass.
Run echo $? and see output 0

Possible Solution(s)
No idea why this is happening and... it's quite alarming!

If I run the same tests through my IDE, it makes a call to cargo test --package rover --lib -- test command::supergraph::config::tests::it_can_fail_a_test --exact --nocapture < which does fail properly!

Notes

$ cargo version cargo 1.53.0 (4369396ce 2021-04-27) $ rustc -V
rustc 1.53.0 (53cb7b09b 2021-06-17) $ rustup version rustup 1.24.3 (ce5817a94 2021-05-31) info: This is the version for the rustup toolchain manager, not the rustc compiler. info: The currently active rustc version is rustc 1.53.0 (53cb7b09b 2021-06-17)

This seems to be a cross-platform bug, as our CI runner does not detect issues with any of the following targets:

x86_64-unknown-linux-musl,
x86_64-unknown-linux-gnu,
x86_64-pc-windows-msvc,
x86_64-apple-darwin,

This is a workspace using the 2018 edition, and seems to happen on test runs with both v1 and v2 of the resolver. It also happens whether you're testing a single package with -p, the whole workspace with --workspace, or just cargo test by itself.

cargo test does not report failures when a test exits the process · Issue #87323 · rust-lang/rust (original) (raw)

`cargo test` does not report failures when a test exits the process · Issue #87323 · rust-lang/rust (original) (raw)