{"author":"Zope Foundation and Contributors","author_email":"zope-dev@zope.dev","bugtrack_url":null,"classifiers":["Development Status :: 5 - Production/Stable","Environment :: Console","Framework :: Zope :: 3","Intended Audience :: Developers","License :: OSI Approved :: Zope Public License","Operating System :: OS Independent","Programming Language :: Python","Programming Language :: Python :: 3","Programming Language :: Python :: 3.10","Programming Language :: Python :: 3.11","Programming Language :: Python :: 3.12","Programming Language :: Python :: 3.13","Programming Language :: Python :: 3.9","Programming Language :: Python :: Implementation :: CPython","Programming Language :: Python :: Implementation :: PyPy","Topic :: Software Development :: Libraries :: Python Modules","Topic :: Software Development :: Testing"],"description":"=================\n zope.testrunner\n=================\n\n.. image:: https://img.shields.io/pypi/v/zope.testrunner.svg\n :target: https://pypi.org/project/zope.testrunner/\n :alt: Latest release\n\n.. image:: https://img.shields.io/pypi/pyversions/zope.testrunner.svg\n :target: https://pypi.org/project/zope.testrunner/\n :alt: Supported Python versions\n\n.. image:: https://github.com/zopefoundation/zope.testrunner/actions/workflows/tests.yml/badge.svg\n :target: https://github.com/zopefoundation/zope.testrunner/actions/workflows/tests.yml\n\n.. image:: https://coveralls.io/repos/github/zopefoundation/zope.testrunner/badge.svg?branch=master\n :target: https://coveralls.io/github/zopefoundation/zope.testrunner?branch=master\n\n.. image:: https://readthedocs.org/projects/zopetestrunner/badge/?version=latest\n :target: https://zopetestrunner.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\n.. contents::\n\nThis package provides a flexible test runner with layer support.\n\nDetailed documentation is hosted at https://zopetestrunner.readthedocs.io\n\n\n=======================\n Using zope.testrunner\n=======================\n\n.. IMPORTANT: This document is included in the long_description for\n rendering on PyPI, so it cannot use Sphinx-only features.\n\nInstallation\n============\n\nBuildout-based projects\n-----------------------\n\nzope.testrunner is often used for projects that use buildout_::\n\n [buildout]\n develop = .\n parts = ... test ...\n\n [test]\n recipe = zc.recipe.testrunner\n eggs = mypackage\n\nThe usual buildout process ::\n\n python bootstrap.py\n bin/buildout\n\ncreates a ``bin/test`` script that will run the tests for *mypackage*.\n\n.. tip::\n\n zc.recipe.testrunner_ takes care to specify the right\n ``--test-path`` option in the generated script. You can add\n other options (such as ``--tests-pattern``) too; check\n zc.recipe.testrunner_'s documentation for details.\n\n\nVirtualenv-based projects\n-------------------------\n\n``pip install zope.testrunner`` and you'll get a ``zope-testrunner``\nscript. Run your tests with ::\n\n zope-testrunner --test-path=path/to/your/source/tree\n\nYour source code needs to be available for the testrunner to import,\nso you need to run ``python setup.py install`` or ``pip install -e\n.`` into the same virtualenv_.\n\n\nSome useful command-line options to get you started\n===================================================\n\n-p show a progress indicator\n-v increase verbosity\n-c colorize the output\n-t test specify test names (one or more regexes)\n-m module specify test modules (one or more regexes)\n-s package specify test packages (one or more regexes)\n--list-tests show names of tests instead of running them\n-x stop on first error or failure\n-D, --pdb enable post-mortem debugging of test failures\n--xml path generate XML reports to be written at the given path\n--help show *all* command-line options (there are many more!)\n\nFor example ::\n\n bin/test -pvc -m test_foo -t TestBar\n\nruns all TestBar tests from a module called test_foo.py.\n\nWriting tests\n=============\n\n``zope.testrunner`` expects to find your tests inside your package\ndirectory, in a subpackage or module named ``tests``. Test modules\nin a test subpackage should be named ``test*.py``.\n\n.. tip::\n\n You can change these assumptions with ``--tests-pattern`` and\n ``--test-file-pattern`` test runner options.\n\nTests themselves should be classes inheriting from\n``unittest.TestCase``, and if you wish to use doctests, please tell\nthe test runner where to find them and what options to use for them\nin by supplying a function named ``test_suite``.\n\nExample::\n\n import unittest\n import doctest\n\n class TestArithmetic(unittest.TestCase):\n\n def test_two_plus_two(self):\n self.assertEqual(2 + 2, 4)\n\n\n def doctest_string_formatting():\n \"\"\"Test Python string formatting\n\n >>> print('{} + {}'.format(2, 2))\n 2 + 2\n\n \"\"\"\n\n def test_suite():\n return unittest.TestSuite([\n unittest.defaultTestLoader.loadTestsFromName(__name__),\n doctest.DocTestSuite(),\n doctest.DocFileSuite('../README.txt',\n optionflags=doctest.ELLIPSIS),\n ])\n\n\n\nTest grouping\n=============\n\nIn addition to per-package and per-module filtering, zope.testrunner\nhas other mechanisms for grouping tests:\n\n* **layers** allow you to have shared setup/teardown code to be used\n by a group of tests, that is executed only once, and not for each\n test. Layers are orthogonal to the usual package/module structure\n and are specified by setting the ``layer`` attribute on test\n suites.\n\n* **levels** allow you to group slow-running tests and not run them\n by default. They're specified by setting the ``level`` attribute\n on test suites to an int.\n\n\nOther features\n==============\n\nzope.testrunner can profile your tests, measure test coverage,\ncheck for memory leaks, integrate with subunit_, shuffle the\ntest execution order, and run multiple tests in parallel.\n\n.. _buildout: https://buildout.readthedocs.io\n.. _virtualenv: https://virtualenv.pypa.io/\n.. _zc.recipe.testrunner: https://pypi.python.org/pypi/zc.recipe.testrunner\n.. _subunit: https://pypi.python.org/pypi/python-subunit\n\n\n===========================\n zope.testrunner Changelog\n===========================\n\n7.3 (2025-05-15)\n================\n\n- Improve ``@unittest.expectedFailure`` support, especially regarding\n post-mortem debugger. (`#196 `_)\n\n7.2 (2025-03-06)\n================\n\n- Re-add a single import of ``pkg_resources`` to avoid other import issues in\n mixed pip/buildout environments.\n (`#194 `_)\n\n\n7.1 (2025-03-05)\n================\n\n- Replace ``pkg_resources`` with ``importlib.metadata``.\n\n\n7.0 (2025-02-12)\n================\n\nBackwards-incompatible changes\n------------------------------\n\n- Remove ``setup.py ftest`` command.\n (`#178 `_)\n\n- Remove ``zope.testrunner.eggsupport``. It is no longer usable as of\n ``setuptools`` 72.0.0.\n (`#185 `_)\n\n\n6.7 (2025-02-07)\n================\n\n- Drop support for Python 3.8.\n\n- Add option ``--only-level=level`` to run tests only at the specified level.\n (`#188 `_)\n\n\n6.6.1 (2024-12-13)\n==================\n\n- Make signatures in ``tb_format`` Python 3.12+ compatible\n (`#186 `_)\n\n\n6.6 (2024-10-16)\n================\n\n- Make tests compatible with Python 3.13 + add support for that version.\n (`#181 `_)\n\n- Drop support for Python 3.7.\n\n\n6.5 (2024-08-06)\n================\n\n- Remove setuptools fossils.\n\n- ``unittest.TestCase.subTest`` support\n (`#91 `_).\n\n- remove support for ``setup.py``'s ``test`` command.\n Support for this command has been dropped by modern\n ``setuptools`` versions and correspondingly has been removed from\n most ``zopefoundation`` packages; ``zope.testrunner`` now follows.\n\n- ``setup.py``'s ``ftest`` command is now only supported\n when the used ``setuptools`` version still supports ``test``.\n\n\n6.4 (2024-02-27)\n================\n\n- Add PEP 420 support (implicit namespaces).\n (`#160 `_)\n\n\n6.3.1 (2024-02-12)\n==================\n\n- Fix XML tests when running in distribution resp. separately.\n (`#163 `_)\n\n\n6.3 (2024-02-07)\n================\n\n- Exit cleanly when using the test runner ``--version`` argument.\n (`#102 `_)\n\n- Add new ``--xml `` option to write JUnit-like XML reports.\n Code comes from ``collective.xmltestreport``, but be aware that here ``--xml``\n is not a boolean, but expects a path!\n (`#148 `_).\n\n- Add support for Python 3.13a3.\n\n\n6.2.1 (2023-12-22)\n==================\n\n- Work around Python 3.12.1+ no longer calling ``startTest`` for skipped tests\n (`#157 `_).\n\n\n6.2 (2023-11-08)\n================\n\n- Add support for Python 3.12.\n\n- Update code and tests to ``python-subunit >= 1.4.3`` thus requiring at least\n this version.\n\n\n6.1 (2023-08-26)\n================\n\n- Add preliminary support for Python 3.12b4.\n (`#149 `_)\n\n\n6.0 (2023-03-28)\n================\n\n- Drop support for Python 2.7, 3.5, 3.6.\n\n\n5.6 (2022-12-09)\n================\n\n- Add support for Python 3.11.\n\n- Inline a small part of ``random.Random.shuffle`` which was deprecated in\n Python 3.9 and removed in 3.11 (`#119\n `_).\n\n- Don't trigger post mortem debugger for skipped tests. ( `#141\n `_).\n\n\n5.5.1 (2022-09-07)\n==================\n\n- Fix: let ``--at-level=level`` with ``level <= 0`` run the tests\n at all levels (rather than at no level)\n `#138 `_.\n\n\n5.5 (2022-06-24)\n================\n\n- Use ``sys._current_frames`` (rather than ``threading.enumerate``)\n as base for new thread detection, fixes\n `#130 `_.\n\n- New option ``--gc-after-test``. It calls for a garbage collection\n after each test and can be used to track down ``ResourceWarning``s\n and cyclic garbage.\n With ``rv = gc.collect()``, ``!`` is output on verbosity level 1 when\n ``rv`` is non zero (i.e. when cyclic structures have been released),\n ``[``*rv*``]`` on higher verbosity levels and\n a detailed cyclic garbage analysis on verbosity level 4+.\n For details, see\n `#133 `_.\n\n- Allow the filename for the logging configuration to be specified\n via the envvar ``ZOPE_TESTRUNNER_LOG_INI``.\n If not defined, the configuration continues to be locked for\n in file ``log.ini`` of the current working directory.\n Remember the logging configuration file in envvar\n ``ZOPE_TESTRUNNER_LOG_INI`` to allow spawned child processes\n to recreate the logging configuration.\n For details, see\n `#134 `_.\n\n\n5.4.0 (2021-11-19)\n==================\n\n- Improve ``--help`` documentation for ``--package-path`` option\n (`#121 `_).\n\n- Do not disable existing loggers during logsupport initialization\n (`#120 `_).\n\n- Fix tests with testtools >= 2.5.0 (`#125\n `_).\n\n- Add support for Python 3.10.\n\n\n5.3.0 (2021-03-17)\n==================\n\n- Add support for Python 3.9.\n\n- Fix `package init file missing` warning\n (`#112 `_).\n\n- Make standard streams provide a `buffer` attribute on Python 3 when using\n `--buffer` or testing under subunit.\n\n\n5.2 (2020-06-29)\n================\n\n- Add support for Python 3.8.\n\n- When a layer is run in a subprocess, read its stderr in a thread to avoid\n a deadlock if its stderr output (containing failing and erroring test IDs)\n overflows the capacity of a pipe (`#105\n `_).\n\n\n5.1 (2019-10-19)\n================\n\n- Recover more gracefully when layer setUp or tearDown fails, producing\n useful subunit output.\n\n- Prevent a spurious warning from the ``--require-unique`` option if the\n ``--module`` option was not used.\n\n- Add optional buffering of standard output and standard error during tests,\n requested via the ``--buffer`` option or enabled by default for subunit.\n\n- Fix incorrect failure counts in per-layer summary output, broken in 4.0.1.\n\n\n5.0 (2019-03-19)\n================\n\n- Fix test failures and deprecation warnings occurring when using Python 3.8a1.\n (`#89 `_)\n\n- Drop support for Python 3.4.\n\n\n4.9.2 (2018-11-24)\n==================\n\n- Fix ``TypeError: a bytes-like object is required, not 'str'``\n running tests in parallel on Python 3. See `issue 80\n `_.\n\n\n4.9.1 (2018-11-21)\n==================\n\n- Fix AssertionError in _DummyThread.isAlive on Python 3 (`#81\n `_).\n\n\n4.9 (2018-10-05)\n================\n\n- Drop support for Python 3.3.\n\n- Add support for Python 3.7.\n\n- Enable test coverage reporting on coveralls.io and in tox.ini.\n\n- Host documentation at https://zopetestrunner.readthedocs.io\n\n- Remove untested support for the ``--pychecker`` option. See\n `issue 63 `_.\n\n- Update the command line interface to use ``argparse`` instead of\n ``optparse``. See `issue 61\n `_.\n\n- Use ipdb instead of pdb for post-mortem debugging if available\n (`#10 `_).\n\n- Add a --require-unique option to check for duplicate test IDs. See\n `LP #682771\n `_.\n\n- Reintroduce optional support for ``subunit``, now with support for both\n version 1 and version 2 of its protocol.\n\n- Handle string in exception values when formatting chained exceptions.\n (`#74 `_)\n\n\n4.8.1 (2017-11-12)\n==================\n\n- Enable ``DeprecationWarning`` earlier, when discovering test\n modules. This lets warnings that are raised on import (such as those\n produced by ``zope.deprecation.moved``) be reported. See `issue 57\n `_.\n\n\n4.8.0 (2017-11-10)\n==================\n\n- Automatically enable ``DeprecationWarning`` when running tests. This\n is recommended by the Python core developers and matches the\n behaviour of the ``unittest`` module. This can be overridden with\n Python command-line options (``-W``) or environment variables\n (``PYTHONWARNINGS``). See `issue 54\n `_.\n\n4.7.0 (2017-05-30)\n==================\n\n- Drop all support for ``subunit``.\n\n\n4.6.0 (2016-12-28)\n==================\n\n- Make the ``subunit`` support purely optional: applications which have\n been getting the dependencies via ``zope.testrunner`` should either add\n ``zope.testrunner[subunit]`` to their ``install_requires`` or else\n depend directly on ``python-subunit``.\n\n- New option ``--ignore-new-thread=`` to suppress \"New thread(s)\"\n warnings.\n\n- Support Python 3.6.\n\n\n4.5.1 (2016-06-20)\n==================\n\n- Fixed: Using the ``-j`` option to run tests in multiple processes\n caused tests that used the ``multiprocessing`` package to hang\n (because the testrunner replaced ``sys.stdin`` with an unclosable\n object).\n\n- Drop conditional dependency on ``unittest2`` (redundant after dropping\n support for Python 2.6).\n\n\n4.5.0 (2016-05-02)\n==================\n\n- Stop tests for all layers when test fails/errors when started with\n -x/--stop-on-error\n (`#37 `_).\n\n- Drop support for Python 2.6 and 3.2.\n\n\n4.4.10 (2015-11-10)\n===================\n\n- Add support for Python 3.5\n (`#31 `_).\n\n- Insert extra paths (from ``--path``) to the front of sys.argv\n (`#32 `_).\n\n\n4.4.9 (2015-05-21)\n==================\n\n- When using ``-j``, parallelize all the tests, including the first test layer\n (`#28 `_).\n\n\n4.4.8 (2015-05-01)\n==================\n\n- Support skipped tests in subunit output\n (`#25 `_).\n\n- More efficient test filtering\n (`#26 `_).\n\n\n4.4.7 (2015-04-02)\n==================\n\n- Work around a bug in PyPy3's curses module\n (`#24 `_).\n\n\n4.4.6 (2015-01-21)\n==================\n\n- Restore support for instance-based test layers that regressed in 4.4.5\n (`#20 `_).\n\n\n4.4.5 (2015-01-06)\n==================\n\n- Sort related layers close to each other to reduce the number of unnecessary\n teardowns (fixes `#14\n `_).\n\n- Run the unit test layer first (fixes `LP #497871\n `__).\n\n\n4.4.4 (2014-12-27)\n==================\n\n- When looking for the right location of test code, start with longest\n location paths first. This fixes problems with nested code locations.\n\n\n4.4.3 (2014-03-19)\n==================\n\n- Added support for Python 3.4.\n\n\n4.4.2 (2014-02-22)\n==================\n\n- Drop support for Python 3.1.\n\n- Fix post-mortem debugging when a non-printable exception happens\n (https://github.com/zopefoundation/zope.testrunner/issues/8).\n\n\n4.4.1 (2013-07-10)\n==================\n\n- Updated ``boostrap.py`` to version 2.2.\n\n- Fix nondeterministic test failures on Python 3.3\n\n- Tear down layers after ``post_mortem`` debugging is finished.\n\n- Fix tests that write to source directory, it might be read-only.\n\n\n4.4.0 (2013-06-06)\n==================\n\n- Fix tests selection when the negative \"!\" pattern is used several times\n (LP #1160965)\n\n- Moved tests into a 'tests' subpackage.\n\n- Made ``python -m zope.testrunner`` work again.\n\n- Support 'skip' feature of unittest2 (which became the new unittest in Python\n 2.7).\n\n- Better diagnostics when communication with subprocess fails\n (https://github.com/zopefoundation/zope.testrunner/issues/5).\n\n- Do not break subprocess execution when the test suite changes the working\n directory (https://github.com/zopefoundation/zope.testrunner/issues/6).\n\n- Count test module import errors as errors (LP #1026576).\n\n\n4.3.3 (2013-03-03)\n==================\n\n- Running layers in sub-processes did not use to work when run via\n ``python setup.py ftest`` since it tried to run setup.py with all the\n command line options. It now detects ``setup.py`` runs and we run the test\n runner directly.\n\n\n4.3.2 (2013-03-03)\n==================\n\n- Fix ``SkipLayers`` class in cases where the distribution specifies a\n ``test_suite`` value.\n\n\n4.3.1 (2013-03-02)\n==================\n\n- Fixed a bug in the `ftest` command and added a test.\n\n- Fixed a trivial test failure with Python 3 of the previous release.\n\n\n4.3.0 (2013-03-02)\n==================\n\n- Expose `ftest` distutils command via an entry point.\n\n- Added tests for ``zope.testrunner.eggsupport``.\n\n\n4.2.0 (2013-02-12)\n==================\n\n- Dropped use of 2to3, rewrote source code to be compatible with all Python\n versions. Introduced a dependency on `six`_.\n\n\n4.1.1 (2013-02-08)\n==================\n\n- Dropped use of zope.fixers (LP: #1118877).\n\n- Fixed tox test error reporting; fixed tests on Pythons 2.6, 3.1, 3.2, 3.3 and\n PyPy 1.9.\n\n- Fix --shuffle ordering on Python 3.2 to be the same as it was on older Python\n versions.\n\n- Fix --shuffle nondeterminism when multiple test layers are present.\n Note: this will likely change the order of tests for the same --shuffle-seed.\n\n- New option: --profile-directory. Use it in the test suite so that tests\n executed by detox in parallel don't conflict.\n\n- Use a temporary coverage directory in the test suite so that tests\n executed by detox in parallel don't conflict.\n\n- Fix --post-mortem (aka -D, --pdb) when a test module cannot be imported\n or is invalid (LP #1119363).\n\n\n4.1.0 (2013-02-07)\n==================\n\n- Replaced deprecated ``zope.interface.implements`` usage with equivalent\n ``zope.interface.implementer`` decorator.\n\n- Dropped support for Python 2.4 and 2.5.\n\n- Made StartUpFailure compatible with unittest.TextTestRunner() (LP #1118344).\n\n\n4.0.4 (2011-10-25)\n==================\n\n- Work around sporadic timing-related issues in the subprocess buffering\n tests. Thanks to Jonathan Ballet for the patch!\n\n\n4.0.3 (2011-03-17)\n==================\n\n- Added back support for Python <= 2.6 which was broken in 4.0.2.\n\n\n4.0.2 (2011-03-16)\n==================\n\n- Added back Python 3 support which was broken in 4.0.1.\n\n- Fixed `Unexpected success`_ support by implementing the whole concept.\n\n- Added support for the new __pycache__ directories in Python 3.2.\n\n\n4.0.1 (2011-02-21)\n==================\n\n- LP #719369: An `Unexpected success`_ (concept introduced in Python 2.7) is\n no longer handled as success but as failure. This is a workaround. The\n whole unexpected success concept might be implemented later.\n\n.. _`Unexpected success`: http://www.voidspace.org.uk/python/articles/unittest2.shtml#more-skipping\n\n\n4.0.0 (2010-10-19)\n==================\n\n- Show more information about layers whose setup fails (LP #638153).\n\n\n4.0.0b5 (2010-07-20)\n====================\n\n- Update fix for LP #221151 to a spelling compatible with Python 2.4.\n\n- Timestamps are now always included in subunit output (r114849).\n\n- LP #591309: fix a crash when subunit reports test failures containing\n UTF8-encoded data.\n\n\n4.0.0b4 (2010-06-23)\n====================\n\n- Package as a zipfile to work around Python 2.4 distutils bug (no\n feature changes or bugfixes in ``zope.testrunner`` itself).\n\n\n4.0.0b3 (2010-06-16)\n====================\n\n- LP #221151: keep ``unittest.TestCase.shortDescription`` happy by supplying\n a ``_testMethodDoc`` attribute.\n\n- LP #595052: keep the distribution installable under Python 2.4: its\n distutils appears to munge the empty ``__init__.py`` file in the\n ``foo.bar`` egg used for testing into a directory.\n\n- LP #580083: fix the ``bin/test`` script to run only tests from\n ``zope.testrunner``.\n\n- LP #579019: When layers were run in parallel, their tearDown was\n not called. Additionally, the first layer which was run in the main\n thread did not have its tearDown called either.\n\n\n4.0.0b2 (2010-05-03)\n====================\n\n- Having 'sampletests' in the MANIFEST.in gave warnings, but doesn't actually\n seem to include any more files, so I removed it.\n\n- Moved zope.testing.exceptions to zope.testrunner.exceptions. Now\n zope.testrunner no longer requires zope.testing except for when running\n its own tests.\n\n\n4.0.0b1 (2010-04-29)\n====================\n\n- Initial release of the testrunner from zope.testrunner as its own module.\n (Previously it was part of zope.testing.)\n\n\n.. _six: http://pypi.python.org/pypi/six\n","description_content_type":null,"docs_url":null,"download_url":null,"downloads":{"last_day":-1,"last_month":-1,"last_week":-1},"dynamic":null,"home_page":"https://github.com/zopefoundation/zope.testrunner","keywords":null,"license":"ZPL-2.1","license_expression":null,"license_files":null,"maintainer":null,"maintainer_email":null,"name":"zope.testrunner","package_url":"https://pypi.org/project/zope.testrunner/","platform":null,"project_url":"https://pypi.org/project/zope.testrunner/","project_urls":{"Homepage":"https://github.com/zopefoundation/zope.testrunner"},"provides_extra":["test","subunit","docs"],"release_url":"https://pypi.org/project/zope.testrunner/7.3/","requires_dist":["setuptools","zope.exceptions","zope.interface","zope.testing; extra == \"test\"","python-subunit>=1.4.3; extra == \"subunit\"","testtools>=0.9.30; extra == \"subunit\"","Sphinx; extra == \"docs\"","sphinxcontrib-programoutput; extra == \"docs\""],"requires_python":">=3.9","summary":"Zope testrunner script.","version":"7.3","yanked":false,"yanked_reason":null}

Dependency	Optional
setuptools	false
zope.exceptions	false
zope.interface	false
python-subunit	true
Sphinx	true
sphinxcontrib-programoutput	true
testtools	true
zope.testing	true

Package search

zope.testrunner

project_urls

Dependencies

All package metadata

Releases