[Python-Dev] Unittest/doctest formatting differences in 2.7a1? (original) (raw)

Olemis Lang olemis at gmail.com
Wed Dec 9 18:14:24 CET 2009

On Wed, Dec 9, 2009 at 11:34 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

On 09/12/2009 16:27, Lennart Regebro wrote:

I just ran the tests for zope.testing on Python 2.7, and the results are not good. It seems that there are multiple minor difference in the output formatting of the testresults between 2.7 and previous 2.x versions. The result is that all the tests that test testing (zope.testing includes a testrunner) fails. Is these changes necessary? It's going to be hell to test any form of testrunner under both 2.6 and 2.7 if the formatting of test results have changed.

This is fuzzy . It is necessary to be more precise and describe what happens in detail.

Besides, I think that it's also necessary to mention (briefly) how tests look like . Perhaps this is a situation where many fragile tests [1]_ have been written and something

Relying on specific formatting for test results sounds like you are relying on an implementation detail

This makes sense (~65% , and not because it's partially wrong but because of subjective interpretations and criteria) . Perhaps the best way would be to analyze contents of test results . The nature of test runners implies that many features (e.g. formatting used to display test results) are not standard . OTOH , the contents in instances of TestResult are (should be more) standard, thereby it's (probably) better to analyze the contents recorded in there (using doctest or unittest, or ... ;o)

PS: I'm talking loud with my eyes closed , just a few preliminary thoughts while waiting for the details ;o)

Some of the failure reporting in unittest has improved in Python 2.7 - are you feeding the output of unittest back into doctest... ?

... and I'm assuming that this is what's been done, isn't it ?

.. [1] Fragile tests (http://xunitpatterns.com/Fragile%20Test.html)

-- Regards,

Olemis.

Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/

Featured article:

More information about the Python-Dev mailing list