Changelog

We’ve added additional details about commits that trigger test status changes to test timelines in the Flaky Tests dashboard.

A test will be identified as flaky if it produces inconsistent results on the same git commit. This is often seen when test failures are automatically retried in CI. This flaky behavior can be detected on stable branches (for example, main), PR branches, or merge branches.

Previously, Flaky Tests did not specify what type of commit led to a status change in the flaky test dashboard. The same “Inconsistent results on the same commit” message would be included in the timeline on the test details page.

Now, the timeline will say where inconsistent test results are seen, whether it is a stable, PR, or merge commit:

When there is a status change due to a stable branch commit, the name of the branch will be included in the message.

This helps to easily identify the source of flakiness when tests are rerun on the same commit.

You can now manually set a test’s status to Flaky, Healthy, or Broken in the Flaky Tests dashboard.

This means you can take advantage of Flaky Tests quarantining to unblock CI when you already know that a test has a high chance of failing, like in the case of a service outage affecting end-to-end tests.

This is not a permanent override. Flaky Tests will continue to classify this test using this new status. 

For example, if a test status is manually set to Healthy but becomes flaky after a series of CI runs, Flaky Tests will mark the test as Flaky.

To update a test’s status:

• Click the Edit button next to the test status in the Flaky Tests dashboard:

• Enter a comment, and click Save Status.

The status change and comment will appear in the timeline on the test detail page.

Manual test status overrides will not be enabled by default. Reach out to us on our community Slack to enable it for your organization.

🎁 New

• New linter yamlfmt

• New tool flyway

• New uv actions

🔧 Fixes

• Forward additional env vars for git-lfs

The names of a test’s suite and class are now visible on the test details page of the flaky test dashboard, along with a file search link for repos using either GitHub or Bitbucket as source control providers.

This additional information helps you navigate to the correct test when investigating flakiness or failures, and is particularly helpful in large projects and monorepos.

The suite and class will only be displayed if your test framework reporter includes that information in the JUnit XML test report uploaded to Trunk.

We will also attempt to add a file search link for projects using GitHub or Bitbucket for source control. It is not always possible to provide a link to the file, depending on how project files are compiled and whether or not those files are available in source control.

We’re happy to announce that a new test_case.quarantining_setting_changed webhook is now available for all Flaky Tests users.

This webhook is fired when a test’s quarantine setting is manually changed between ALWAYS_QUARANTINE, NEVER_QUARANTINE, or UNSPECIFIED, where UNSPECIFIED means that the repo’s quarantine settings will be applied to the test case.

The webhook payload contains useful details like the new and previous settings, a user-inputted reason for the change, who made the change, and all the info you need for the affected test case:

1{
2 "quarantine_setting_changed": {
3   "actor": {
4     "email": "riley@trunk.io",
5     "full_name": "Riley Draward"
6   },
7   "previous_quarantining_setting": "UNSPECIFIED",
8   "reason": "Test is important, should not be quarantined!",
9   "timestamp": "2025-06-11T14:17:06.251Z",
10   "updated_quarantining_setting": "NEVER_QUARANTINE"
11 },
12 "test_case": {
13   "codeowners": [],
14   "failure_rate_last_7d": 0,
15   "html_url": "https://app.trunk.io/rileys-org/flaky-tests/test/cd7ef2fe-0ec1-5e40-a323-027029579f5b?repo=rdraward%2Fflaky-remix",
16   "id": "cd7ef2fe-0ec1-5e40-a323-027029579f5b",
17   "most_common_failures": [],
18   "name": "sums some nums > adds 1 + 1 to equal 2",
19   "pull_requests_impacted_last_7d": 0,
20   "quarantined": false,
21   "repository": {
22     "html_url": "https://github.com/rdraward/flaky-remix"
23   },
24   "status": {
25     "reason": "",
26     "timestamp": "2025-03-17T21:53:43.327Z",
27     "value": "flaky"
28   },
29   "test_suite": "app/utils/sum.test.ts",
30   "ticket": {
31     "html_url": "https://linear.app/riley-at-trunk/issue/RIL-10/[flaky-test]-apputilssumtestts-sums-some-nums-adds-1-1-to-equal-2"
32   },
33   "variant": ""
34 },
35 "type": "test_case.quarantining_setting_changed"
36}
37

This allows you to build automations and custom workflows when a test’s quarantine setting is changed, such as automatically creating Linear or Jira tickets to fix flaky or broken tests marked as ALWAYS_QUARANTINED, or sending a message to the team in Slack so everyone is kept up to date on status changes.

To subscribe to the new webhook:

1. Go to Settings in the Trunk app.

2. Click on Webhooks under Organization.

3. Click Add Endpoint and connect to an existing endpoint, Slack, MS Teams, Linear, or Jira.

4. Follow our guides for handling the webhook payload.

For more information on the webhook events, check out our Svix documentation.

✨ Improvements

• Python runtime environment now passes thru required environment variables for PIP and Request component to work correctly with proxies and custom certificates

• Updated dependencies in local package.json and node runtime bumped to 22 LTS

🔧 Fixes

• Fixed up some testing environment to better support some linter specifications

✨ Improvements

• Updated embedded CURL to v8.13 (enables support for running inside Open AI Codex)

• Formatting issues can now be downgraded to non-blocking using the threshold value like any other linter issue. To make use of this feature you need to set the unformatted_level for any formatters you want to control this way as such:

1lint:
2  definitions:
3   - name: foo_formatter
4     commands:
5      - name: format
6        ...
7        formatter: true
8        unformatted_level: low

🔧 Fixes

• Specifying a runtime with equality operator will use system version if available. (e.g. ruby@=3.4.2 will now use ruby found on PATH if version matches exactly)

We’ve updated the Merge Queue Failure tab so that only PRs that failed and have not been resubmitted to the queue are displayed.

Previously, the Failure tab contained a list of all historical PR failures, making it difficult to identify current failures requiring action. This change gives you a concise list of PR failures that can be fixed and resubmitted to the queue. Upon resubmission, the PR is removed from the page.

More changes have landed on the Flaky Tests dashboard: unique failure details on the test details page are now available in a single table view.

This table is helpful for developers tasked with debugging and fixing flakes. Trunk uses AI to automatically group related failure reasons for an individual test and presents them in a single table:

The test summary, charts that display quarantine or error status, count, and dates of failures help you determine where to start your investigation. Each unique failure reason can also be expanded to show all historical stack traces for that failure type:

This means that all the information you need to start debugging flaky tests is in a single place. Developers don’t have to manually dig through historical CI logs to unearth various stack traces and logs.

Have questions or feedback on the updates? Reach out on our community Slack.

The new Flaky Tests RSpec plugin is the best way to run RSpec tests and upload the results to Trunk.

The plugin provides a couple of different advantages over generating JUnit XML files when testing with RSpec, all of which contribute to more accurate flake detection in Ruby projects:

• By default, running RSpec tests and creating a report using rspec_junit_formatter will produce programmatic test names, which makes flake detection difficult. The plugin eliminates this issue by uploading consistent test names to Trunk.

• The plugin allows for a more direct capture of test run retries.

• Flaky tests can be quarantined as the tests run, not after the fact.

Get started with rspec_trunk_flaky_tests

To get started with the rspec_trunk_flaky_tests plugin:

• Add the plugin gem to your Gemfile:

1# Gemfile
2gem "rspec_trunk_flaky_tests"

• Install the plugin by running the install command in your terminal

1bundler install

• Load the plugin in spec_helper.rb:

1# Add to spec/spec_helper.rb
2require "trunk_spec_helper"

Don’t forget to disable automatic retries on failures.

That’s it, you are ready to run your RSpec tests and upload the results to Trunk. Start by uploading locally run test results:

1TRUNK_ORG_URL_SLUG=<TRUNK_ORG_URL_SLUG> \
2TRUNK_API_TOKEN=<TRUNK_ORG_TOKEN> \
3bundle exec rspec

Your TRUNK_ORG_URL_SLUG and TRUNK_API_TOKEN are found under Settings in the Trunk web app.

When you run this command locally, your RSpec tests will run, and results will automatically be uploaded to Trunk. Any uploaded results will be displayed in the Uploads tab.

The last step is integrating with your CI provider. After integrating, Trunk will automatically detect and quarantine your flaky tests every time the CI job runs.

See the RSpec docs for more info.

Trunk CI Analytics and Code Quality web issue browser are being shut down today. This is a strategic change as we reinvent and consolidate our tools for the future.

What’s changing?

• CI Analytics has been shut down as of today.

• Code Quality’s web issue browser has been shut down as of today.

• Code Quality workflows like Check Nightly and Check on PRs have been marked as deprecated and will be shut down on July 27th, 2025.

The core value of Code Quality comes from the CLI, IDE, and CI integrations, and these critical components will continue to work as they always have.

Code Quality migration

Checking for issues nightly and on pull requests has been marked as deprecated. You can check if you’re using these features in the Code Quality section of your repo settings. You can still run Code Quality in CI if you migrate your workflow following this guide.

Code Quality will no longer collect your linting issues, which means the --upload flag will no longer work. You will also stop receiving Slack notifications about new issues discovered by linters.

1# --upload will no longer work
2trunk check --upload --series=main

What's next?

AI has changed the way we work. It allows us to go beyond traditional dashboards to surface root causes of test and CI instability and develop AI-driven tools to address these problems. We have shut down these parts of Trunk to unify them as a single AI-powered DevEx platform.

We can’t wait to show you what’s next. You can follow our progress by signing up for our in-app waitlist. If you have questions or concerns, please reach out via email (support@trunk.io) or Slack.

Webhooks are now available for batched PR merges in Merge Queue. This allows you to build custom automations and respond to events when batching is enabled.

The pull_request_batch.merged webhook payload lists all the PRs included in the batch, as well as information about the repo, the target branch you are merging into, and the test (or feature) branch:

1 "action": "merged",
2  "pr_numbers": [
3    1,
4    2
5  ],
6  "repository": {
7    "host": "github.com",
8    "name": "trunk-io",
9    "owner": "mergequeue"
10  },
11  "status": "some string",
12  "target_branch": "main",
13  "test_branch": "trunk-merge/pr-1-2/abc123",
14  "test_branch_sha": "a23f8853080d3fe873c3419a735513898c2329f500ce6aa39cf2253213fd00e4",
15  "test_pr_number": 15
16}

If you aren’t using batching on your Merge Queue, the pull_request.merged webhook is still available for individual PR merges.

Read the webhook docs to learn how to subscribe to Merge Queue webhooks.

We now have a built-in Linear integration that creates tickets with relevant test and failure information from your flaky tests.

Detecting and quarantining flaky tests aren’t the only steps required to eliminate them from a repo. A follow-up process must also be established to collect, triage, and fix flaky tests.

Ticketing system integrations help with that follow-up. Trunk generates tickets automatically from detected flaky tests, enabling you to triage and address flakes. Flakes on important tests should be dealt with immediately, and other flakes can be added to a backlog or the tests can be marked for future removal.

To set up the Linear integration:

• Go to your repo settings: Settings > repo > Ticketing Integration.

• Select Linear as your system and enter a Linear API key.

• Select a team and connect to Linear.

You can also select a default project and assignee for newly created tickets. This enables you to designate a person on a team to triage, prioritize, update, and assign tickets for flaky tests when they are created.

Once the integration is set up, you can create a new ticket:

• In the options menu on the test cases overview dashboard.

• With the Create ticket button on the test details dashboard.

You will see a preview of the ticket title and description, and can click Create Linear Ticket to create the new ticket.

When a ticket is created, any changes to that ticket, such as status or person assigned, are also visible in the timeline on the test case details page in Flaky Tests.

Read the docs for more information on setting up the Linear integration.

Today, we’re introducing a new Link Ticket to Test Case API that allows you to link your existing Linear or Jira tickets to tests in the Flaky Tests dashboard.

Trunk has built-in ticketing integrations with Jira and Linear, allowing you to create detailed bug tickets for flaky tests with a single click. If you’re already tracking flaky test issues with existing tickets, you can now link those Jira or Linear tickets directly to your flaky tests in Trunk, keeping all relevant context in one place.

Calling the API requires the ID of your test case in Flaky Tests, which can be found on incoming webhook payloads or the URL of the test case details page, and the ID of the ticket in your ticketing system:

1curl -L \​
2  --request POST \​
3  --url 'https://api.trunk.io/v1/flaky-tests/link-ticket-to-test-case' \​
4  --header 'x-api-token: YOUR_API_KEY' \​
5  --header 'Content-Type: application/json' \​
6  --data '{
7    "test_case_id": "01234567-0123-0123-0123-0123456789ab",
8    "external_ticket_id": "KAN-123"

Once a ticket is linked to a test, ticketing information will be available in Flaky Tests webhooks. This makes it possible to build custom integrations and automations with the context of any linked ticket information. And when ticket information is not included in the payload, you know one has not been created for the flake.

You can only use the Link Ticket to Test Case API if you have already set up the Linear or Jira integration.

Read the reference docs for more information on the Link Ticket to Test Case API.

Starting today, the --variant option should be used to upload test results when the same tests are run on different environments, also known as matrix builds.

If you use a matrix strategy to run the same CI tests on different architectures or environments, test flakiness could be unique to an environment. Examples include testing a mobile app on iOS and Android, running tests for your CLI tool on macOS and Linux, or performing the same e2e tests on different browsers. Tests run on separate environments must be treated as unique to guarantee accurate flake detection.

Using variants, Flaky Tests will now treat the same test run on different environments as unique. This means flakes can be detected and quarantined on a per-environment basis.

To specify a variant during upload, use the --variant option:

1./trunk flakytests upload --junit-paths "test_output.xml" \
2--org-url-slug <TRUNK_ORG_SLUG> \
3--token $TRUNK_API_TOKEN \
4--variant linux_x86_64

Variant names are visible in brackets on your Flaky Tests dashboard:

See the reference docs for more information on uploading test reports with variants.

We’ve added a new updateQueue API to Trunk Merge Queue that enables you to update a Merge Queue’s state.

This means that you no longer need to manually go into the settings page in the web app to update the merge queue state. Instead, queue state can be managed with custom tooling or automations using updateQueue:

1curl -L \
2  --request POST \
3  --url 'https://api.trunk.io/v1/updateQueue' \
4  --header 'x-api-token: YOUR_API_KEY' \
5  --header 'Content-Type: application/json' \
6  --data '{
7    "repo": {
8      "host": "github.com",
9      "name": "mergequeue",
10      "owner": "trunk-io"
11    },
12    "targetBranch": "main",
13    "state": "DRAINING"
14  }'

As a refresher, the possible queue states are:

• RUNNING: PRs are being enqueued and are merged after tests pass. This is the default state for a merge queue.

• PAUSED: PRs in the queue are tested, but the queue will not merge successful PRs or begin testing new PRs. A queue can be paused as an immediate response to a CI incident, reducing noise from failures while preserving the order of PRs that have already been enqueued.

• DRAINING: PRs currently in the queue will be tested and merged, but new PRs will not begin testing. This state is helpful for use cases like a code freeze, so PRs in the queue can be shipped, but newly enqueued PRs are not tested or merged. 

See the Merge Queue API reference for more information.

🎁 New

• New linter: golangci-lint2. Users who wish to upgrade should replace golangci-lint with golangci-lint2

✨ Improvements

• Add in-repo docs for converting some native linter ignores to trunk-ignore

• Update kube-linter to 0.7.2. Thanks @janisz for the contribution!

• Update ruby build and latest stable ruby release to 3.4.2

🔧 Fixes

• Fix Dockerfile file pattern regex to match spec

The test details page in the Flaky Tests dashboard is getting a major UX overhaul.

User interviews and feedback from beta users helped us remove metrics that were not useful and update visualizations to better show a test’s health over time. The tabbed views are also gone, which means you can get all the information about a test’s health in a single, unified dashboard, including: 

• A test’s current status.

• When a test was marked as flaky and quarantined.

• Test flake and failure rates on mainline and PR branches.

• Overall test health history.

• Grouped unique test failure reasons.

These new charts break down how often a test is successful, failing, or quarantined for CI jobs for your mainline branch and PRs and show test health changes over time.

A more eventful test history timeline

The test history timeline has been moved to the main page, and updates, including when a test was marked as flaky, quarantined, or fixed, are always visible. Teams using the Jira integration to track test flakes will also see any updates to a flaky test’s ticket reflected in the timeline.

You can still override the repo's default quarantine setting and always quarantine or never quarantine a test, but now you can leave a comment for other devs that shows up in the timeline.

For example, you might want to let your team know that the “Test is important - flakes must be fixed!” when you override the repo default to never quarantine a test:

The table shows all the unique reasons the test fails in CI.  Trunk uses AI to auto-group unique failures so multiple runs will be grouped under a single failure reason. Individual runs, including stack traces, branch, commit, and job information, are available when a row in the table is expanded. This means all the information you need to debug and fix flaky or broken tests is available in a single place.

Have feedback for us? Reach out on our community Slack.

A new Quarantined Tests API is now available to all Flaky Tests users. This API fetches a list of currently quarantined tests for a given repo, allowing organizations to implement custom workflows to review and fix quarantined tests.

1// https://api.trunk.io/v1/flaky-tests/list-quarantined-tests
2// a sample response with a single quarantined flaky test
3{
4  "quarantined_tests": [
5    {
6      "name": "sums some nums > subtracts 1 from 2 to equal 1",
7      "parent": "app/utils/subtract.test.ts",
8      "file": null,
9      "classname": "app/utils/subtract.test.ts",
10      "status": "FLAKY",
11      "codeowners": [],
12      "quarantine_setting": "AUTO_QUARANTINE",
13      "status_last_updated_at": "2025-02-13T15:52:03.000Z",
14      "test_case_id": "48cd26cb-6333-528a-919f-ed597ee11715"
15     }
16   ],
17   "page": {
18     "total_rows": 1,
19     "total_pages": 1,
20     "next_page_token": "",
21     "prev_page_token": "nUj35JF5aNJFjUkIakxff89f1b...",
22     "last_page_token": "bVUNwekgkHadgZfSgwuac75640...",
23     "page_index": 0
24   }
25}

Dev teams are using the Quarantined Tests API to do things such as:

• Automatically skip quarantined tests when testing locally

• Check for any tests that should be manually un-quarantined

• Determine what teams, projects, or modules have the most quarantined tests

• Assign engineers to fix certain quarantined tests, based on recent code changes

• Send Slack or email notifications to let teams know when the number of quarantined tests reaches a threshold

See the API reference for more information on required headers and payload.

Trunk now supports single sign-on (SSO) for enterprise plan users. SSO allows your team to access Trunk with their work email account through your organization's identity and access management solutions, making it easier to manage your Trunk organization.

SSO is only available for enterprise plan users. If you're interested in configuring SSO for your team, contact us by email or Slack. We will reach out to help you configure SSO for your organization.