[Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 (original) (raw)

Fredrik Lundh fredrik at pythonware.com
Sun Jan 8 15:57:06 CET 2006

Neal Norwitz wrote:

In the past, we haven't checked in tests which are known to be broken. There are several good reasons for this. I would prefer you, 1) also fix the code so the test doesn't fail, 2) revert the change (there's still a bug report open, right?), or 3) generalize tests for known bugs.

I strongly prefer #1, but have been thinking about adding #3. There are many open bug reports that fall into two broad categories: incorrect behaviour and crashers. I've been thinking about adding two tests which incorporate these bugs as a way of consolidating where the known problems are. Also, it's great when we have test cases that can be moved to the proper place once the fix has been checked in.

many test frameworks support "expected failures" for this purpose. how hard would it be to add a

unittest.FailingTestCase

class that runs a TestCase, catches any errors in it, and signals an error ("test foo passed unexpectedly") if it runs cleanly ?

More information about the Python-Dev mailing list