[Python-Dev] unittest Suggestions (original) (raw)

Sebastian Rittau srittau at jroger.in-berlin.de
Tue Aug 12 16:30:59 CEST 2008

[I just saw the other post about unit testing, while I was writing this. A strange conincidence.]

Hello,

since this is the first time I post to this list, I'd like to introduce myself briefly. As you can see my name is Sebastian. I am working as a software developer both as a professional and as a hobbyist. During the years I've used many programming languages, but in the last few years I've fallen in love with Python. (Blame Martin v. Löwis for that - I started using Python after participating in a seminar held by him at university.)

I'd like to propose a few changes to the unittest module. These changes are supposed to ease usage, enable future changes and enhancements, and bring unittest more in line with other modern xUnit variants (such as JUnit 4 and NUnit). I have implemented most of those changes in a custom library that hacks around the current unittest behaviour. I've published the relevant package at [1]. All proposed changes are (mostly) backward compatible.

Change unittest into a package

Currently unittest is a module. Changing it into a package would enable us to separate concerns and move code out into separate modules. For example the unittest.asserts proposal below requires this change. If a mocking framework would be included in the Python standard library at some point, it could go into unittest.mock etc.

@test, @before, and @after decorators

Currently unittest uses "magic" names starting with "test_" for detecting the test methods in a test case class. Following the "explicit is better than implicit" guideline, I propose to add a @test decorator to explicitly mark test methods:

class MyTest(TestCase):

  @test
  def attributes(self):
      """Do stuff."""

I've been bitten in the past by naming methods test_xyz that were not supposed to be called by the test runner. For now those methods still need to be recognized, since they are widely used. But enabling (and recommending) an alternative is a first step.

The decorator syntax makes is possible to introduce alternative decorators as well, like @disabled_test. In the example package I included a @performance_test decorator that takes a time argument where the test fails if it takes more than a given amount of time.

Similarily I recommend to introduce @before and @after decorators to supplement the setUp and tearDown methods. The decorators were named after the decorators in JUnit 4, although I wonder whether @setup and @teardown would be a better name. @before and @after methods are called in any order, but @before methods in a super class are guaranteed to be called before @before methods in a derived class, while it's vice versa with @after methods. Introducing these has the added benefit of being able to get rid of the mixedCase setUp and tearDown methods in my own code.

unittest.asserts

Currently all assertions reside as methods on the TestCase class. This means that all testing functions, including utility functions, need to be on a TestCase-derived class. This seems wrong to me, especially since it combines different concerns on a class: Testing the functionality of a SUT, running a test suite, and providing test functions. This proposal will not try to separate the first two concerns, but only the third one. In the long term I think it's worthwhile to make tests work on any class, not only on classes derived from TestCase, but as I said, this is not part of this proposal.

JUnit 4 moved the assertions to a static class in a separate module. In Python I'd propose to use functions in a separate module. A typical test suite would then look like this:

from unittest import TestCase from unittest.asserts import *

class FooTest(TestCase):

  """Tests for the Foo class."""

  @before
  def setup(self):
      self._sut = create_an_object_to_test()

  @test
  def construct__some_attribute(self):
      assert_equals("foobar", self._sut.some_attribute)

  ...

Again, as a side effect this removes inconsistent naming (mixedCase and three different version of each test) from my tests. There is one regression, though. Currently it's possible to change the assertion exception (which defaults to AssertionError) by setting an attribute of the TestCase instance. I am not sure how relevant that is, though. I've never used this feature myself, not even when testing the unittest module and its extensions, and this approach wouldn't work anyway, if you use classes with utility assertions etc.

Anyway, I am happy about any comments.

[1] http://www.rittau.biz/~srittau/unittest, though I will move it to http://www.rittau.org/python/unittest, once I'm home.

More information about the Python-Dev mailing list