Changelog

We’re excited to provide a new way for you to mitigate the negative impact of flaky tests in your repo through quarantining. 

Quarantining lets you isolate failures for known flaky tests so they don't fail your CI jobs while still continuing to run them. It looks for known flaky tests and determines which tests should be isolated at runtime, avoiding the code changes usually required to disable flaky tests.

Running flaky tests without disabling them lets you validate your fixes by continuously monitoring their results. This is especially important because most attempted fixes for flaky tests don’t reduce the test’s flakiness.

Enabling Quarantining

We’re currently onboarding organizations to try quarantining manually. This is because quarantining can drastically impact the results of your CI jobs. We want to help you validate quarantining results and educate your team on its behavior during onboarding. To try Quarantining, reach out to us on Slack.

Quarantining Features

Key Impact Metrics: You can see the impact of quarantining on your repo. Understand how many CI jobs with flaky failures were rescued by quarantining and how many engineering hours were saved.

Keep track of quarantined tests: You can continue to monitor the results of quarantined flaky tests through the PR Tests Summaries generated by Trunk. You can filter failures by quarantined runs and see the failure reasons at a glance.

Per test overrides: You can override quarantining for individual tests in your repo to always quarantine or never quarantine. This helps you handle edge cases such as tests that should never break or tests that are flaky/broken due to reasons outside your team’s control, like external APIs being down.

Detailed audit trails: You can see the audit trail of quarantine setting changes in both the test’s status history and an overall audit trail in the settings.

Learn more in the Quarantining documentation.