[Python-Dev] unittest: shortDescription, _TextTestResult and other issues (original) (raw)

Michael Foord mfoord at python.org
Tue Feb 9 19:49:40 CET 2010

I missed another minor issue. In the interests of completeness...

You currently have to subclass TextTestRunner (and override _makeResult) for it to use a custom TestResult. Implementing a custom test result is one of extensibility points of unittest, so I propose adding an optional argument to TextTestRunner allowing you to pass in a result class (or other callable) that _makeResult will use to create the result object. Should be uncontroversial. :-)

http://bugs.python.org/issue7893

Michael

On 09/02/2010 16:40, Michael Foord wrote:

Hello all,

I've been looking at outstanding unittest issues as part of my preparation for my PyCon talk. There are a couple of changes (minor) I'd like to make that I thought I ought to run past Python-Dev first. If I don't get any responses then I'll just do it, so you have been warned. :-) The great google merge into unittest happened at PyCon last year [1]. This included a change to TestCase.shortDescription() so that it would always include the test name, whereas previously it would return the test docstring or None. The problem this change solved was that tests with a docstring would not have their name (test class and method name) reported during the test run. Unfortunately the change broke part of twisted test running. Reported as issue 7588: http://bugs.python.org/issue7588 It seems to me that the same effect (always reporting test name) can be achieved in TextTestResult.getDescription(). I propose to revert the change to TestCase.shortDescription() (which has both a horrible name and a horrible implementation and should probably be renamed getDocstring so that what it does is obvious but never mind) and put the change into TextTestResult. It annoys me that TextTestResult is private, as you will almost certainly want to use it or subclass it when implementing custom test systems. I am going to rename it TextTestResult, alias the old name and document the old name as being deprecated. Another issue that I would like to address, but there are various possible approaches, is issue 7559: http://bugs.python.org/issue7559 Currently loadTestsFromName catches ImportError and rethrows as AttributeError. This is horrible (it obscures the original error) but there are backwards compatibility issues with fixing it. There are three possible approaches: 1) Leave it (the default) 2) Only throw an AttributeError if the import fails due to the name being invalid (the module not existing) otherwise allow the error through. (A minor but less serious change in behavior). 3) A new method that turns failures into pseudo-tests that fail with the original error when run. Possibly deprecating loadTestsFromName I favour option 3, but can't think of a good replacement name. :-) Comments welcomed. Despite deprecating (in the documentation - no actual deprecations warnings I believe) a lot of the duplicate ways of doing things (assert* favoured over fail* and assertEqual over assertEquals) we didn't include deprecating assert in favour of assertTrue. I would like to add that to the documentation. After 3.2 is out I would like to clean up the documentation, removing mention of the deprecated methods from the main documentation into a separate 'deprecated methods' section. They currently make the documentation very untidy. The unittest page should probably be split into several pages anyway and needs improving. Other outstanding minor issues: Allow dotted names for test discovery http://bugs.python.org/issue7780 - I intend to implement this as described in the last comment A 'checkorder' optional argument (defaulting to True) for assertSequenceEqual http://bugs.python.org/issue7832 - needs patch _The breaking of unittest caused by splitting unittesst into a package needs fixing. The fix needs to work when Python is run without frames support (IronPython). http://bugs.python.org/issue7815 - needs patch _Allow a unittest (or similar) decorator for user implemented assert functions http://bugs.python.org/issue1705520 - needs patch Allow modules to define testsuite callable. http://bugs.python.org/issue7501 - I propose to close as rejected. Use loadtests instead. Display time taken by individual tests when in verbose mode. http://bugs.python.org/issue4080 - anyone any opinions? Allow automatic formatting of arguments in assert* failure messages. http://bugs.python.org/issue6966 - I propose to close as rejected removeTest() method on TestSuite http://bugs.python.org/issue1778410 - anyone any opinions? expect methods (delayed fail) http://bugs.python.org/issue3615 - any opinions? Personally I think that the TestCase API is big enough already

All the best, Michael Foord [1] Mostly in revision 7-837. http://svn.python.org/view?view=rev&revision=70837

-- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog

READ CAREFULLY. By accepting and reading this email you agree, on behalf of your employer, to release me from all obligations and waivers arising from any and all NON-NEGOTIATED agreements, licenses, terms-of-service, shrinkwrap, clickwrap, browsewrap, confidentiality, non-disclosure, non-compete and acceptable use policies (”BOGUS AGREEMENTS”) that I have entered into with your employer, its partners, licensors, agents and assigns, in perpetuity, without prejudice to my ongoing rights and privileges. You further represent that you have the authority to release me from any BOGUS AGREEMENTS on behalf of your employer.

More information about the Python-Dev mailing list