[2.7] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master by vstinner · Pull Request #2444 · python/cpython (original) (raw)

• regrtest --list-cases now supports --match and --match-file options. Example: ./python -m test --list-cases -m FileTests test_os
• --list-cases now also sets support.verbose to False to prevent messages to stdout when loading test modules.
• Add support._match_test() private function. (cherry picked from commit ace56d5)

(cherry picked from commit 36946c0)

• bpo-30764: regrtest: change exit code on failure

• Exit code 2 if failed tests ("bad")

• Exit code 3 if interrupted

• bpo-30764: regrtest: add --fail-env-changed option

If the option is set, mark a test as failed if it alters the environment, for example if it creates a file without removing it.

(cherry picked from commit 63f54c6) (cherry picked from commit 1f33857)

• Change the regrtest --huntrleaks checker to decide if a test file leaks or not. Require that each run leaks at least 1 reference.
• Warmup runs are now completely ignored: ignored in the checker test and not used anymore to compute the sum.
• Add an unit test for a reference leak.

Example of reference differences previously considered a failure (leak) and now considered as success (success, no leak):

[3, 0, 0]
[0, 1, 0]
[8, -8, 1]

(cherry picked from commit 48b5c42)

(cherry picked from commit e0f8b43)