[Python-Dev] Fwd: PEP: Frequently-requested additional features for the unittest module (original) (raw)

Olemis Lang olemis at gmail.com
Thu Aug 14 17:34:47 CEST 2008

Abstract

This PEP proposes frequently-requested additions to the standard library unittest module that are natural extensions of its existing functionality.

I had not seen this proposal... I will post a few comments.

Motivation

The unittest module is functionally complete. Nevertheless, many users request and devise extensions to perform functions that are so common they deserve to be in the standard module.

Specification

New condition tests

The following test methods will be added to the TestCase class. The function signature is part of this specification. The body is to be treated as a reference implementation only; any functionally identical implementation also meets this specification.

...

The following test methods are also added. Their implementation in each case is simply the logical inverse of a corresponding method above.

My question here is... should assert* (fail*) methods be included in a separate class or module? I think this is a good idea (at least in the long term) Take a look at the following thread [1] for further details

Simple invocation of test collection

The following new functionality will be added to the unittest module. The function signature for run_tests is part of this specification; the implementation is to be considered a reference implementation only.

::

def run_tests(
    tests,
    loader_class=TestLoader, suite_class=TestSuite,
    runner_class=TextTestRunner):
    """ Run a collection of tests with a test runner

        :param tests:
            A sequence of objects that can contain test cases:
            modules, `TestSuite` instances, or `TestCase`
            subclasses.
        :param loader_class:
            The type of test loader to use for collecting tests
            from the `tests` collection.
        :param suite_class:
            The type of test suite to use for accumulating the
            collected test cases.
        :param runner_class:
            The type of test runner to instantiate for running the
            collected test cases.
        :return:
            None.

        """

I suppose that the reason for this function to exist is to ease test code (provide a shortcut or something), reuse some test code, and still allow different loading, running strategies. However, I dont think that passing (loader / suite / runner) classes in to this function is a good idea... IMO objects should be considered instead...

Why? Mainly because the code assumes that all loaders are stateless. This may be true for unittest's and perhaps others, but this might not always be true. An example of statefull test loaders might be found in [2] (presented in [3]), and perhaps there other examples like this (even though I don't know them). This kind of code might restrain the use of this kind of loaders, and therefore might not be as reusable as should be.

Nonetheless, you might think this is a very peculiar case and decide that this comment is irrelevant, but as far as I know, semantics of unittest TestLoader do not enforce that its subclasses ought to be stateless (although, I repeat, unittest test loaders are stateless).

The same might be said for test runners... and they are more likely to be stateful.

    def iter_testcases_recursively(collection, loader):
        # Flatten and iterate over collection, generating
        # instances of TestCase
    loader = loader_class()
    suite = suite_class()
    for testcase in iter_testcases_recursively(tests, loader):
        suite.add_tests(testcase)
    runner = runner_class()
    runner.run(suite)

Besides... why is it necessary to flatten a hierarchy of test suites containing test cases? Why do you need to put them into another suite if the test loader already does this for us? Isn't it an extra overhead? Also there is a class attribute (suiteClass) in TestLoader which allows doing something like this.

[1] [Python-Dev] unittest Suggestions (http://mail.python.org/pipermail/python-dev/2008-August/081763.html)

[2] Module dutest (https://sourceforge.net/project/showfiles.php?group_id=220287&abmode=1)

[3] [Python-Dev] An OO API for doctest / unittest integration... (http://mail.python.org/pipermail/python-dev/2008-August/081761.html)

-- Regards,

Olemis.

More information about the Python-Dev mailing list