What is pytest?

pytest  is a test automation framework for Python.  pytest is used for unit testing, meaning it tests parts of a codebase in isolation, rather than integration testing where the parts are combined and tested to see how they work together (though it can be used for both). pytest makes it easy to write and manage standardized Python tests for your codebase, working hand in hand with your CI (Continuous Integration) system to ensure tests are reliable and repeatable across team members and environments.

Enabling XML Output

pytest can produce JUnit XML output by running with the --junit-xml= option.

1pytest --junit-xml=filepath.xml 

Test Suite naming

pytest will automatically fill in values for the <testcase> and <testsuite> name and classname attributes. 

1def test_95_percent():
2    random_number = secrets.randbelow(100)
3    assert random_number <= 95

would produce output that looks like this:

1<testsuites>
2    <testsuite name="pytest">
3        <testcase classname="random_test" 
4                  name="test_95_percent"
5                  file="python/pytest/random_test.py"
6                  />
7     ...
8</testsuites>

The suite name can be configured in the pytest.ini or similar config file.

1[pytest]
2junit_suite_name = my_suite

Configuring other XML output values is not currently supported, but experimental options are available.  To include the test filenames in the XML output, use the -o junit_family=xunit1 option.

1pytest --junitxml=output/path.xml -o junit_family=xunit1

By default, pytest will include the file attribute in the output XML.

Handling Retries

By default, pytest will not retry tests. The developer must restart the tests manually.  To have pytest automatically retry failed tests use the pytest-rerunfailures plugin.

1pip install pytest-rerunfailures
2pytest --reruns 5

If you need to retry a lot of tests that is a strong indicator that you may have flaky tests. These are tests that sometimes pass and sometimes fail, slowing down your development process. You can get early access to Trunk’s new Flaky Tests tool to manage these types of tests better.

Other Configuration

See an example of running pytest in a GitHub action here.