[Python-Dev] Unittest/doctest formatting differences in 2.7a1? (original) (raw)

exarkun at twistedmatrix.com exarkun at twistedmatrix.com
Fri Dec 11 21:36:38 CET 2009

On 9 Dec, 06:09 pm, fuzzyman at voidspace.org.uk wrote:

On 09/12/2009 18:02, exarkun at twistedmatrix.com wrote:

On 05:11 pm, lregebro at jarn.com wrote:

On Wed, Dec 9, 2009 at 17:34, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

Can you be more specific?

Only with an insane amount of work. I'll hold that off for a while. I don't know if this is related at all (and I guess we won't until Lennart can be more specific :), but here are some Twisted unit test failures which are probably due to unittest changes in 2.7: =============================================================================== [FAIL]: twisted.trial.test.testloader.LoaderTest.testsortCases Traceback (most recent call last): File "/home/buildslave/pybot/trunk.glass-x86/build/Twisted/twisted/trial/test/testloader.py", line 167, in testsortCases [test.testMethodName for test in suite.tests]) twisted.trial.unittest.FailTest: not equal: a = ['testb', 'testc', 'testa'] b = ['testc', 'testb', 'testa'] Is the order significant? Can you just compare sorted versions of the lists instead? If the order is significant I would be interested to know which change caused this.

It looks like a change to shortDescription may be responsible for all the failures mentioned here.

The order is significant (it's a test for how tests are ordered...). The sorting code (which wasn't super awesome, I'll admit) it uses was broken by the change in the return value of shortDescription, something which is much more obvious in some of the other failures.

[snip]

a = 'skip1 (twisted.trial.test.testtests.SkippingTests)' b = 'skip1' These two test failures are due to the change in repr of something I guess? Is a or b the original output?

b is the original, a is the produced value. Here's a simpler example:

Python 2.6.4 (r264:75706, Nov 2 2009, 14:38:03) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from twisted.test.testabstract import AddressTests AddressTests('testdecimalDotted').shortDescription() 'test_decimalDotted'

Python 2.7a0 (trunk:76325M, Nov 16 2009, 09:50:40) [GCC 4.4.1] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from twisted.test.testabstract import AddressTests AddressTests('testdecimalDotted').shortDescription() 'test_decimalDotted (twisted.test.test_abstract.AddressTests)'

Aside from compatibility issues, this seems like a not-too-ideal change in behavior for a method named "shortDescription", at least to me. Going out on a limb, I'll guess that it was made in order to provide a different user-facing experience in the stdlib's test runner. If that's right, then I think it would make sense to revert shortDescription back to the previous behavior and either introduce a new method which returns this string or just put the logic all into the display code instead.

I'm not opposed to the improvement of unittest (or any part of Python). Perhaps more of the improvements can be provided in new APIs rather than by changing the behavior of existing APIs, though. Well, introducing lots of new APIs has its own problems of course.

It seems to me that it typically has fewer problems.

Hearing about difficulties changes cause is good though (the reason for alphas I guess) and if it is possible to work out why things are broken then maybe we can still have the new functionality without the breakage.

Of course. Also, I should have pointed this out in my previous email, this information about failures is always easily available, at least for Twisted (and at most for any project interested in participating in the project), on the community buildbots page:

http://www.python.org/dev/buildbot/community/trunk/

So anyone who cares to can check to see if their changes have broken things right away, instead of only finding out 6 or 12 or 18 months later. :)

The problem with Lennart's report is that it is just "things are broken" without much clue. I am sympathetic to this (working out exactly what is broken in someone else's code can be a painful chore) but I'm not sure what can be done about it without more detail.

Certainly. Perhaps Zope would like to add something to the community builders page.

Thanks,

Jean-Paul

More information about the Python-Dev mailing list