Over a year ago we introduced our advanced merge queue product, Trunk Merge, to help developer teams spend less time dealing with test failures & merge conflicts and more time on making new features. Since the initial release we’ve made significant improvements, including a Parallel Mode that can turbocharge your developer pipeline. Today I’d like to talk about some of the recent improvements we’ve made, all directly inspired by customer needs and requests.
Batching
Normally Trunk Merge tests PRs order one by one. In parallel mode it can test changes to different parts of the codebase in parallel, but this still results in running the tests on every single PR individually. If your testing cost is N per PR, then 20 PRs will cost 20 N.
To make tests even faster and less expensive, Merge now supports batching. In a high velocity codebase it is more efficient to group PRs into batches so that they can be tested and merged as a larger unit. With a batch size of 4 those 20 PRs will cost 5N instead of 20 N, drastically reducing cost of automated testing. If a PR takes 30 minutes of CI time, then batching will reduce the total CI time from 600 minutes (30 min x 20 PRs), down to only 150 minutes (30 min x 5 batches), a 4x time savings!
How Batching works
Here’s how batching works. Trunk Merge will wait until enough PRs are in the queue to fill a batch or until a timeout is reached, then combine the PRs into a single test branch. This branch is then tested and merged as a single unit instead of testing them individually.
With batching enabled Trunk Merge still has to handle failed tests and logical merge conflicts which can slow down the batches. That is why batching has settings for the target batch size and maximum wait time. The ideal settings depend on your particular repo since there is a trade off between batch size and waiting until a complete batch is ready.
Properly adjusted, in a high velocity codebase batching can drastically reduce the total test time while still protecting you from flaky tests and logical merge conflicts. Batching can make your merge pipeline be many times more efficient and significantly save on CI costs.
See the Batching docs for more details.
PR Prioritization
Sometimes you need a pull request (PR) to be merged right away. It's urgent. Waiting through the entire queue of pending PRs will be too slow. With the new PR Prioritization feature you can mark some PRs as higher priority so that they can get tested and merged ASAP or, if it’s really urgent, jump to the front of the queue.
1/trunk merge --priority=high
You can set the priority either in a GitHub comment with /trunk merge --priority=<level> or from the command line with trunk merge <pr-number> --priority=<level> .The priority level is either a number from 0 to 255 or set with the keyword urgent , high, medium, or low. Higher priority PRs will move ahead of lower priority ones, but not actually interrupt them if they are in the middle of testing.
For example, suppose Merge is processing a bunch of PRs with CSS style improvements and a new PR comes in to fix a serious React UI bug. By marking the style PRs with low priority and the UI bug with high, then the UI bug will move ahead of the style PRs in the queue.
Trunk Merge tries to never interrupt testing in progress because that would be very inefficient. If a style PR in the previous example is in the middle of testing then the higher priority PR will be scheduled to run right after and will not interrupt it. The one exception to this rule is for urgent PRs. An urgent PR will interrupt a currently testing PR and begin immediately.
See the PR Prioritization docs for more details and examples.
Optimistic Merging and Pending Failure Depth
Trunk Merge is a smart merge queue. It can handle what you’d expect from a typical merge queue, ensuring all of your PRs are merged correctly, in the right order, and that all tests pass and can dynamically create parallel merge queues to merge code even faster.
One challenge a lot of our customers have is flaky tests. These are tests which fail some of the time, but not all of the time, and can waste a huge amount of testing resources dealing with false positives. To help with this Trunk's legendary engineering team has introduced Optimistic Merging and Pending Failure Depth.
Optimistic Merging lets a PR be merged even when it failed, but only in the case where another PR that depends on the first does pass. In this situation it is likely that the first failure was transient (a flaky test) or fixed by a later PR in the queue, after all the second PR had the changes from the first PR and worked.
Normally when a PR fails testing it is simply kicked out of the queue and sends a message to Slack about the failure. Sometimes you might want the PRs to hang out in the queue while later PRs are tested - maybe you want to go inspect why the PR failed before it is removed from the queue, or you want an opportunity to manually restart the PR, or you want to see if it causes other PRs to fail. This behavior is controlled by the new Pending Failure Depth setting. If you set it to 4 then, when a PR fails, it will wait for all PRs four steps below it to finish testing before doing anything.
Pending Failure Depth and Optimistic Merging combined are powerful new features that can speed up your pipeline and make it more resistant to flaky tests. If the PFD is set to 4 then up to four failed PRs can be left in the queue. If the 5 one passes then the first four will also be merged. If the 5th fails then possibly there really is something deeper going on and they will all be kicked out.
You can easily configure Pending Failure Depth and Optimistic Merging from the Merge webapp settings.
See the Optimistic Merging docs for details.
Scale Your Merge Process with Trunk Merge
As your engineering team grows, managing a high volume of pull requests can get complicated. Trunk Merge is here to help you streamline your merge process and scale efficiently, whether you're dealing with tens, hundreds, or even thousands of PRs daily. Want to see how Trunk Merge can optimize your workflow? Join our Slack community to chat with the Merge team, or schedule a demo to see Trunk Merge live. We'll work with you to simulate your CI costs and time to merge, so you can see the real impact on your pipeline.
