bpo-30764: regrtest: add --fail-env-changed option by vstinner · Pull Request #2402 · python/cpython (original) (raw)

vstinner added a commit that referenced this pull request

bpo-30523: regrtest --list-cases --match (#2401)
regrtest --list-cases now supports --match and --match-file options. Example: ./python -m test --list-cases -m FileTests test_os
--list-cases now also sets support.verbose to False to prevent messages to stdout when loading test modules.
Add support._match_test() private function. (cherry picked from commit ace56d5)
bpo-30764: regrtest: add --fail-env-changed option (#2402)
bpo-30764: regrtest: change exit code on failure
Exit code 2 if failed tests ("bad")
Exit code 3 if interrupted
bpo-30764: regrtest: add --fail-env-changed option

If the option is set, mark a test as failed if it alters the environment, for example if it creates a file without removing it.

(cherry picked from commit 63f54c6)

bpo-30776: reduce regrtest -R false positives (#2422)
Change the regrtest --huntrleaks checker to decide if a test file leaks or not. Require that each run leaks at least 1 reference.
Warmup runs are now completely ignored: ignored in the checker test and not used anymore to compute the sum.
Add an unit test for a reference leak.

Example of reference differences previously considered a failure (leak) and now considered as success (success, no leak):

[3, 0, 0]
[0, 1, 0]
[8, -8, 1]

(cherry picked from commit 48b5c42)