[Python-Dev] Checking in a broken test was: Re: [Python-checkins] r41940 (original) (raw)

Tim Peters tim.peters at gmail.com
Sun Jan 8 00:24:06 CET 2006

[Neal Norwitz]

... In the past, we haven't checked in tests which are known to be broken.

It's an absolute rule that you never check in a change (whether a test or anything else) that causes regretst.py -uall to fail. Even if it passes on your development box, but fails on someone else's box, it's fair game to revert that change.

Since this particular change caused -uall to fail on all boxes, I had no qualms about reverting rev 41940.

Reinhold, if you're working on a fix for the problem too, feel free to check that test in on a branch instead (short-lived branches are easy & fast under SVN). Branches aren't required to pass any tests. The test on the branch can be merged into the trunk later, after a fix has also been checked in on that branch.

...

There are many open bug reports that fall into two broad categories: incorrect behaviour and crashers. I've been thinking about adding two tests which incorporate these bugs as a way of consolidating where the known problems are. Also, it's great when we have test cases that can be moved to the proper place once the fix has been checked in.

I'm proposing something like add two files to Lib/test: outstandingbugs.py and outstandingcrashes.py. Both would be normal test files with info about the bug report and the code that causes problems. This test in testcompiler should be moved to outstandingbugs.py.

That would be fine provided that the failing tests aren't normally run via -uall.

More information about the Python-Dev mailing list