| msg138101 - (view) |
Author: Éric Araujo (eric.araujo) *  |
Date: 2011-06-10 16:19 |
| When regrtest tells me that os.environ was modified by test_packaging, it’s very hard to find which test is responsible. The -vv output is hard to read: it prints four dictionaries, which I have to visually delineate, copy, paste, pretty-print and diff to find what’s changed. Then I edit the source code of regrtest so that it fails earlier, or I add a debug print in the packaging.util code that touches os.environ. I’d like regrtest to tell me what exactly was changed, and where. For the first request, I think each checked object could use a type-specific method, for example _compare_dict would see if any value was changed and if keys are different, and print only that info. |
|
|
| msg138109 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2011-06-10 16:52 |
| Improvements are welcome, the whole thing was a QnD hack that has been evolving over time...maybe there's some way to reuse unittest's assertEqual machinery? I also wonder if the whole environment-changed thing should be moved into unittest so it could wrap around individual test methods. |
|
|
| msg138110 - (view) |
Author: Éric Araujo (eric.araujo) * |
Date: 2011-06-10 16:56 |
| > maybe there's some way to reuse unittest's assertEqual machinery? Hum, I think the diff functionality is cleanly separated in a (private) method, so we could get diff for free. > I also wonder if the whole environment-changed thing should be moved > into unittest so it could wrap around individual test methods. Yeah, I was thinking about a WatchfulTestRunner that would perform the same checks. It annoys me to have two diverging sets of features to run tests: either checks with regrtest, or failfast, discovery and granularity with unittest. |
|
|
| msg138378 - (view) |
Author: Michael Foord (michael.foord) * |
Date: 2011-06-15 14:42 |
| I don't think this is something that belongs in unittest - it's not something particularly useful (or at least particularly requested) outside of the python test suite. No reason that a WatchfulTestRunner couldn't live in regrtest. |
|
|
| msg138381 - (view) |
Author: R. David Murray (r.david.murray) * |
Date: 2011-06-15 15:22 |
| Sounds like a plan. |
|
|
| msg138673 - (view) |
Author: STINNER Victor (vstinner) * |
Date: 2011-06-19 23:15 |
| > I’d like regrtest to tell me what exactly was changed, and where. regrtests has many tests (you give some examples: os.environ, sys.path), run all tests after calling a single test function would make regrtest slower. We can add an option (e.g. --strict?) to run these sanity tests after calling a single test function. I would also be nice to have this option for the refleak checks: run refleak checks on each function, instead of doing the checks on the whole file. |
|
|
| msg144276 - (view) |
Author: Roundup Robot (python-dev)  |
Date: 2011-09-19 14:10 |
| New changeset 9c61f46ea6b4 by Éric Araujo in branch 'default': Make regrtest look at internal dicts in sysconfig. http://hg.python.org/cpython/rev/9c61f46ea6b4 |
|
|
| msg145110 - (view) |
Author: Éric Araujo (eric.araujo) * |
Date: 2011-10-07 17:26 |
| Idea: when a regrtest check fails, the test is run again but a WatchfulMixin is injected into the test case’s bases, so that setUp and tearDown call the right methods. This would be more efficient than always running with a watchful test runner. |
|
|
| msg220508 - (view) |
Author: Mark Lawrence (BreamoreBoy) * |
Date: 2014-06-13 22:26 |
| @Éric do you intend following up on this? |
|
|