zope.testrunner

This package provides a flexible test runner with layer support.

Detailed documentation is hosted at

Using zope.testrunner

Installation

Buildout-based projects

zope.testrunner is often used for projects that use buildout:

[buildout]
develop = .
parts = ... test ...

[test]
recipe = zc.recipe.testrunner
eggs = mypackage

The usual buildout process :

python bootstrap.py
bin/buildout

creates a bin/test script that will run the tests for mypackage.

Virtualenv-based projects

pip install zope.testrunner and you'll get a zope-testrunner script. Run your tests with :

zope-testrunner --test-path=path/to/your/source/tree

Your source code needs to be available for the testrunner to import, so you need to run python setup.py install or pip install -e . into the same virtualenv.

Some useful command-line options to get you started

-p show a progress indicator -v increase verbosity -c colorize the output -t test specify test names (one or more regexes) -m module specify test modules (one or more regexes) -s package specify test packages (one or more regexes) --list-tests show names of tests instead of running them -x stop on first error or failure -D, --pdb enable post-mortem debugging of test failures --xml path generate XML reports to be written at the given path --help show all command-line options (there are many more!)

For example :

bin/test -pvc -m test_foo -t TestBar

runs all TestBar tests from a module called test_foo.py.

Writing tests

zope.testrunner expects to find your tests inside your package directory, in a subpackage or module named tests. Test modules in a test subpackage should be named test*.py.

Tests themselves should be classes inheriting from unittest.TestCase, and if you wish to use doctests, please tell the test runner where to find them and what options to use for them in by supplying a function named test_suite.

Example:

import unittest
import doctest

class TestArithmetic(unittest.TestCase):

    def test_two_plus_two(self):
        self.assertEqual(2 + 2, 4)


def doctest_string_formatting():
    """Test Python string formatting

        >>> print('{} + {}'.format(2, 2))
        2 + 2

    """

def test_suite():
    return unittest.TestSuite([
        unittest.defaultTestLoader.loadTestsFromName(__name__),
        doctest.DocTestSuite(),
        doctest.DocFileSuite('../README.txt',
                             optionflags=doctest.ELLIPSIS),
    ])

Test grouping

In addition to per-package and per-module filtering, zope.testrunner has other mechanisms for grouping tests:

• layers allow you to have shared setup/teardown code to be used by a group of tests, that is executed only once, and not for each test. Layers are orthogonal to the usual package/module structure and are specified by setting the layer attribute on test suites.
• levels allow you to group slow-running tests and not run them by default. They're specified by setting the level attribute on test suites to an int.

Other features

zope.testrunner can profile your tests, measure test coverage, check for memory leaks, integrate with subunit, shuffle the test execution order, and run multiple tests in parallel.

zope.testrunner Changelog

6.6 (2024-10-16)

• Make tests compatible with Python 3.13 + add support for that version. (#181)
• Drop support for Python 3.7.

6.5 (2024-08-06)

• Remove setuptools fossils.
• unittest.TestCase.subTest support (#91).
• remove support for setup.py's test command. Support for this command has been dropped by modern setuptools versions and correspondingly has been removed from most zopefoundation packages; zope.testrunner now follows.
• setup.py's ftest command is now only supported when the used setuptools version still supports test.

6.4 (2024-02-27)

• Add PEP 420 support (implicit namespaces). (#160)

6.3.1 (2024-02-12)

• Fix XML tests when running in distribution resp. separately. (#163)

6.3 (2024-02-07)

• Exit cleanly when using the test runner --version argument. (#102)
• Add new `--xml