{"author":null,"author_email":"Adam Johnson ","bugtrack_url":null,"classifiers":["Development Status :: 5 - Production/Stable","Framework :: Pytest","Intended Audience :: Developers","License :: OSI Approved :: MIT License","Natural Language :: English","Programming Language :: Python :: 3 :: Only","Programming Language :: Python :: 3.10","Programming Language :: Python :: 3.11","Programming Language :: Python :: 3.12","Programming Language :: Python :: 3.13","Programming Language :: Python :: 3.9","Typing :: Typed"],"description":"===============\npytest-randomly\n===============\n\n.. image:: https://img.shields.io/github/actions/workflow/status/pytest-dev/pytest-randomly/main.yml.svg?branch=main&style=for-the-badge\n :target: https://github.com/pytest-dev/pytest-randomly/actions?workflow=CI\n\n.. image:: https://img.shields.io/badge/Coverage-100%25-success?style=for-the-badge\n :target: https://github.com/pytest-dev/pytest-randomly/actions?workflow=CI\n\n.. image:: https://img.shields.io/pypi/v/pytest-randomly.svg?style=for-the-badge\n :target: https://pypi.org/project/pytest-randomly/\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg?style=for-the-badge\n :target: https://github.com/psf/black\n\n.. image:: https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white&style=for-the-badge\n :target: https://github.com/pre-commit/pre-commit\n :alt: pre-commit\n\nPytest plugin to randomly order tests and control ``random.seed``.\n\n----\n\n**Testing a Django project?**\nCheck out my book `Speed Up Your Django Tests `__ which covers loads of ways to write faster, more accurate tests.\n\n----\n\nFeatures\n========\n\nAll of these features are on by default but can be disabled with flags.\n\n* Randomly shuffles the order of test items. This is done first at the level of\n modules, then at the level of test classes (if you have them), then at the\n order of functions. This also works with things like doctests.\n\n* Resets the global ``random.seed()`` at the start of every test case and test\n to a fixed number - this defaults to ``time.time()`` from the start of your\n test run, but you can pass in ``--randomly-seed`` to repeat a\n randomness-induced failure.\n\n* If\n `factory boy `_\n is installed, its random state is reset at the start of every test. This\n allows for repeatable use of its random 'fuzzy' features.\n\n* If `faker `_ is installed, its random\n state is reset at the start of every test. This is also for repeatable fuzzy\n data in tests - factory boy uses faker for lots of data. This is also done\n if you're using the ``faker`` pytest fixture, by defining the ``faker_seed``\n fixture\n (`docs `__).\n\n* If\n `Model Bakery `_\n is installed, its random state is reset at the start of every test. This\n allows for repeatable use of its random fixture field values.\n\n* If `numpy `_ is installed, its legacy global random state in |numpy.random|__ is reset at the start of every test.\n\n .. |numpy.random| replace:: ``numpy.random``\n __ https://numpy.org/doc/stable/reference/random/index.html\n\n* If additional random generators are used, they can be registered under the\n ``pytest_randomly.random_seeder``\n `entry point `_ and\n will have their seed reset at the start of every test. Register a function\n that takes the current seed value.\n\n* Works with `pytest-xdist `__.\n\nAbout\n=====\n\nRandomness in testing can be quite powerful to discover hidden flaws in the\ntests themselves, as well as giving a little more coverage to your system.\n\nBy randomly ordering the tests, the risk of surprising inter-test dependencies\nis reduced - a technique used in many places, for example Google's C++ test\nrunner `googletest\n`_.\nResearch suggests that \"dependent tests do exist in practice\" and a random\norder of test executions can effectively detect such dependencies [1]_.\nAlternatively, a reverse order of test executions, as provided by `pytest-reverse\n`__, may find less dependent\ntests but can achieve a better benefit/cost ratio.\n\nBy resetting the random seed to a repeatable number for each test, tests can\ncreate data based on random numbers and yet remain repeatable, for example\nfactory boy's fuzzy values. This is good for ensuring that tests specify the\ndata they need and that the tested system is not affected by any data that is\nfilled in randomly due to not being specified.\n\nI have written a `blog post covering the history of\npytest-randomly `__,\nincluding how it started life as the nose plugin\n`nose-randomly `__.\n\nAdditionally, I appeared on the Test and Code podcast to `talk about\npytest-randomly `__.\n\nInstallation\n============\n\nInstall with:\n\n.. code-block:: bash\n\n python -m pip install pytest-randomly\n\nPython 3.9 to 3.13 supported.\n\nUsage\n=====\n\nPytest will automatically find the plugin and use it when you run ``pytest``.\nThe output will start with an extra line that tells you the random seed that is\nbeing used:\n\n.. code-block:: bash\n\n $ pytest\n ...\n platform darwin -- Python ...\n Using --randomly-seed=1553614239\n ...\n\nIf the tests fail due to ordering or randomly created data, you can restart\nthem with that seed using the flag as suggested:\n\n.. code-block:: bash\n\n pytest --randomly-seed=1234\n\nOr more conveniently, use the special value ``last``:\n\n.. code-block:: bash\n\n pytest --randomly-seed=last\n\n(This only works if pytest’s cacheprovider plugin has not been disabled.)\n\nSince the ordering is by module, then by class, you can debug inter-test\npollution failures by narrowing down which tests are being run to find the bad\ninteraction by rerunning just the module/class:\n\n.. code-block:: bash\n\n pytest --randomly-seed=1234 tests/module_that_failed/\n\nYou can disable behaviours you don't like with the following flags:\n\n* ``--randomly-dont-reset-seed`` - turn off the reset of ``random.seed()`` at\n the start of every test\n* ``--randomly-dont-reorganize`` - turn off the shuffling of the order of tests\n\nThe plugin appears to Pytest with the name 'randomly'. To disable it\naltogether, you can use the ``-p`` argument, for example:\n\n.. code-block:: sh\n\n pytest -p no:randomly\n\nAvoid reordering some tests\n---------------------------\n\nTo fix the order of some tests, use the ``pytest-order`` plugin.\nSee its `documentation section `__ on usage with pytest-randomly.\n\nEntry Point\n===========\n\nIf you're using a different randomness generator in your third party package,\nyou can register an entrypoint to be called every time ``pytest-randomly``\nreseeds. Implement the entrypoint ``pytest_randomly.random_seeder``, referring\nto a function/callable that takes one argument, the new seed (int).\n\nFor example in your ``setup.cfg``:\n\n.. code-block:: ini\n\n [options.entry_points]\n pytest_randomly.random_seeder =\n mypackage = mypackage.reseed\n\nThen implement ``reseed(new_seed)``.\n\nReferences\n==========\n\n.. [1] Sai Zhang, Darioush Jalali, Jochen Wuttke, Kıvanç Muşlu, Wing Lam, Michael D. Ernst, and David Notkin. 2014. Empirically revisiting the test independence assumption. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA 2014). Association for Computing Machinery, New York, NY, USA, 385–396. doi:https://doi.org/10.1145/2610384.2610404\n","description_content_type":"text/x-rst","docs_url":null,"download_url":null,"downloads":{"last_day":-1,"last_month":-1,"last_week":-1},"dynamic":null,"home_page":null,"keywords":"pytest, random, randomise, randomize, randomly","license":null,"license_expression":null,"license_files":null,"maintainer":null,"maintainer_email":null,"name":"pytest-randomly","package_url":"https://pypi.org/project/pytest-randomly/","platform":null,"project_url":"https://pypi.org/project/pytest-randomly/","project_urls":{"Changelog":"https://github.com/pytest-dev/pytest-randomly/blob/main/CHANGELOG.rst","Funding":"https://adamj.eu/books/","Repository":"https://github.com/pytest-dev/pytest-randomly"},"provides_extra":null,"release_url":"https://pypi.org/project/pytest-randomly/3.16.0/","requires_dist":["pytest","importlib-metadata>=3.6; python_version < \"3.10\""],"requires_python":">=3.9","summary":"Pytest plugin to randomly order tests and control random.seed.","version":"3.16.0","yanked":false,"yanked_reason":null}

Dependencies of pytest-randomly (2).
Dependency	Optional
importlib-metadata	false
pytest	false

Package search

pytest-randomly

project_urls

Dependencies

All package metadata

Releases