Skip to main content

Nose plugin to randomly order tests and control random.seed.

Project description

=============
nose-randomly
=============

.. image:: https://img.shields.io/travis/adamchainz/nose-randomly.svg
:target: https://travis-ci.org/adamchainz/nose-randomly

.. image:: https://img.shields.io/pypi/v/nose-randomly.svg
:target: https://pypi-hypernode.com/pypi/nose-randomly

.. figure:: https://raw.githubusercontent.com/adamchainz/nose-randomly/master/logo.png
:scale: 50%
:alt: Randomness power.

----

**Unmaintained:** I'm no longer maintaining this package because I haven't used
nose for some time, and nose itself has not seen a release since 2015, nor a
commit since 2016 (nearly 3 years at time of writing). If you want to continue
maintenance please contact me.

----

Nose plugin to randomly order tests and control ``random.seed``. (Also
available `for pytest <https://github.com/adamchainz/pytest-randomly>`_).

Features
--------

All of these features are on by default but can be disabled with flags.

* Randomly shuffles the submodules, ``TestCase`` classes + test functions when
loading a module of tests.
* Randomly shuffles the test functions inside a ``TestCase`` when loading it.
* Resets ``random.seed()`` at the start of every test case and test to a fixed
number - this defaults to ``time.time()`` from the start of your test run,
but you can pass in ``--randomly-seed`` to repeat a randomness-induced
failure.
* If
`factory boy <https://factoryboy.readthedocs.io/en/latest/reference.html>`_
is installed, its random state is reset at the start of every test. This
allows for repeatable use of its random 'fuzzy' features.
* If `faker <https://pypi-hypernode.com/pypi/fake-factory>`_ is installed, its
random state is reset at the start of every test. This is also for repeatable
fuzzy data in tests - factory boy uses faker for lots of data.

About
-----

Randomness in testing can be quite powerful to discover hidden flaws in the
tests themselves, as well as giving a little more coverage to your system.

By randomly ordering the tests, the risk of surprising inter-test dependencies
is reduced - a technique used in many places, for example Google's C++ test
runner `googletest
<https://code.google.com/p/googletest/wiki/V1_5_AdvancedGuide#Shuffling_the_Tests>`_.

By resetting the random seed to a repeatable number for each test, tests can
create data based on random numbers and yet remain repeatable, for example
factory boy's fuzzy values. This is good for ensuring that tests specify the
data they need and that the tested system is not affected by any data that is
filled in randomly due to not being specified.

Requirements
------------

Tested with:

* Python 2.7, 3.6
* The latest version of Nose

Usage
-----

Install from pip with:

.. code-block:: bash

pip install nose-randomly

Nose will automatically find the plugin.

To activate it on your test run, use the ``--with-randomly`` flag, for example:

.. code-block:: bash

nosetests -v --with-randomly

The output will start with an extra line that tells you the random seed that is
being used:

.. code-block:: bash

Using --randomly-seed=1234
test_D (abcd_tests.Tests) ... ok
...

If the tests then fail due to ordering or randomly created data, you can then
restart them with that seed:

.. code-block:: bash

nosetests -v --with-randomly --randomly-seed=1234

You can disable behaviours you don't like with the following flags:

* ``--randomly-dont-shuffle-modules`` - turn off the shuffling of the contents
of modules
* ``--randomly-dont-shuffle-cases`` - turn off the shuffling of test functions
inside ``TestCase`` classes
* ``--randomly-dont-reset-seed`` - turn off the reset of ``random.seed()`` at
the start of every test


Background
----------

`nose` has an `unmerged pull request
<https://code.google.com/p/python-nose/issues/detail?id=255>`_ from 2009 to add
random ordering functionality. This is available in plugin format in the
`nose-randomize <https://github.com/nloadholtes/nose-randomize/>`_ package. It
works quite well but I found that since it replaces all of the test loading
machinery inside `nose`, it can interact badly with other plugins. This plugin
was developed as a thinner layer to achieve the same thing, plus the random
seed resetting which was not available before.


License
-------

* BSD licensed, see LICENSE file
* Logo by Christian Mohr from the Noun Project
(`link <https://thenounproject.com/search/?q=dice&i=110905>`_).




History
-------

Pending release
---------------

1.2.6 (2019-02-07)
------------------

* Update PyPI development status as inactive. This package is no longer
maintained, see README.rst.
* Dropped Python 2.6 compatibility, as upstream dependency NumPy did.

1.2.5 (2016-10-28)
------------------

* Set a high plugin score to ensure that ``nose-randomly`` is loaded before
other plugins. This fixes a bug where randomization would disapper when using
the ``doctests`` plugin that is included with Nose.

1.2.4 (2016-10-27)
------------------

* Reset the random state for NumPy too.

1.2.3 (2016-08-19)
------------------

* Fixed output so the random seed is always output when the plugin is enabled,
not just when resetting ``random.seed()`` at the start of tests. Thanks
@amygdalama.

1.2.2 (2016-07-06)
------------------

* Fixed to work with ``python setup.py nosetests`` on Python 2 due to issue
with ``unicode`` not working with ``distutils.fancy_getopt``.

1.2.1 (2016-06-01)
------------------

* Support test generators.

1.2.0 (2015-12-10)
------------------

* Reset the random state for Faker (pip package ``fake-factory``) too

1.1.0 (2015-08-27)
------------------

* Reset the random seed at the start of nose test contexts (TestCases
etc.) too
* Slight performance improvement by always using ``random.setstate()`` for
reseeding

1.0.0 (2015-07-23)
------------------

* First release on PyPI.


Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nose-randomly-1.2.6.tar.gz (9.7 kB view details)

Uploaded Source

Built Distribution

nose_randomly-1.2.6-py2.py3-none-any.whl (6.9 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file nose-randomly-1.2.6.tar.gz.

File metadata

  • Download URL: nose-randomly-1.2.6.tar.gz
  • Upload date:
  • Size: 9.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/2.7.15

File hashes

Hashes for nose-randomly-1.2.6.tar.gz
Algorithm Hash digest
SHA256 7e483a3d79e13ae760d6ade57ae07ae45bb4b223b61a805e958b4c077116c67c
MD5 e358c97da4be4514b8505c27ca8e8a5d
BLAKE2b-256 64169d04cfec78f92544bc14bf9d54584a7b18ed71ce5da18d965438bbd6ef28

See more details on using hashes here.

File details

Details for the file nose_randomly-1.2.6-py2.py3-none-any.whl.

File metadata

  • Download URL: nose_randomly-1.2.6-py2.py3-none-any.whl
  • Upload date:
  • Size: 6.9 kB
  • Tags: Python 2, Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.12.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.6.3 requests-toolbelt/0.9.1 tqdm/4.30.0 CPython/2.7.15

File hashes

Hashes for nose_randomly-1.2.6-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 257a10813e85fd28bba97f830ed1ba68f564d5a0653add08f5a2e4b8711fa740
MD5 3049d081ce280e0b2a633c80aef6671f
BLAKE2b-256 d5aa7f7ef8e0690331a0c26e2cd089cb769ac6e02970ee3bf921d1b96b4999e7

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page