My first little pytest plugin

I test our backend via an API interface. It is a json rpc api that talks over a websocket. The test framework that I created is written in python. The testrunner is py.test. py.test already has a lot of plugins that are very handy.

All went well, until last week. The latest tests I wrote should not run as is. Some of them where tests that did a calibration. Others where tests that should run before the calibration was done.  I did find a pytest plugin that could order the tests and run them in that order. But I was not satisfied with the plugin itself. The markers where not chosen well for our project.

So I decided to have our own simple plugin that fits our needs. I wanted something like this:

def test_first():
    logging.info("first test")
    assert True

@pytest.mark.run_when_not_calibrated
def test_second():
    logging.info("second test")
    assert True

This is very easy to implement. I created a file sorttests.py in a plugin folder with the following content:

def pytest_collection_modifyitems(config, items):
    # pylint: disable=unused-argument
    items[:] = sorted(items,
                      key=lambda item: 0 if item.get_marker("run_when_not_calibrated") else 1)

This function is called by pytest when the collection of all tests is completed. By altering the items list, we can manipulate the tests that are running, also the order of the tests.

Now I can use the marker run_when_not_calibrated, just what I wanted.

I only needed to put the following line in the conftest.py file to enable the plugin. This is needed because I did not create a setup for the plugin yet.

pytest_plugins = ['plugins.sorttests']

So with a little bit of extra code, I easily can manipulate the order of the tests.