Merge Queue

What Happens If a Merge Queue Builds on the Wrong Commit

That which writes to main should be boring

What Happens If a Merge Queue Builds on the Wrong Commit

On April 23rd, GitHub's merge queue started silently reverting code on customers' main branches. Not a handful of lines - in some cases, thousands. And nothing looked wrong. A PR with a tidy +29/-34 diff got reviewed, approved, and queued. What actually landed on main was a single commit with +245/-1,137. Thousands of lines of unrelated, already-shipped code were quietly removed. Every merge that followed went in on top of that broken history.

No error. No conflict. No warning. Just clean, green merges that happened to delete other people's work.

If you run a busy repo, the math on this is brutal. Teams we spoke to spent hours combing through the commit graph, reconstructing commits by hand, figuring out what code had been silently dropped. In one organization, every team on GitHub's merge queue got hit — dozens of bad commits each, hundreds of existing ones clobbered before anyone noticed. The only team that came through cleanly was the one running Trunk Merge Queue.

GitHub identified the regression and rolled it back within a few hours. Merge queues are genuinely hard. But we want to talk about why this particular failure mode was possible at all, because it points at a design choice that a merge queue doesn't have to make.

What actually happened

A merge queue's job is to serialize PR merges so that every PR gets CI-tested against the exact state of main it will land on top of. To do that, GitHub constructs a temporary branch for each PR in the queue. Normally, that temp branch is the tip of `main` plus the PR's diff. That's what CI runs against, and if it passes, that's what lands.

For a few hours on Thursday, the queue was building temp branches off the wrong starting point. Instead of branching from the tip of main, it was branching from wherever the feature branch had originally diverged from main. Then it pushed the contents of that temp branch to main wholesale.

Two-panel git graph. Top: the correct merge, where the temp branch is built from the tip of main and produces a small commit matching the PR. Bottom: the April 23 bug, where the temp branch is built from the divergence point and produces a large commit that silently reverts the intermediate work on main.

Concretely: if your feature branch was 50 commits behind when it hit the queue, the "merge" to main silently removed those 50 commits of other people's work as a side effect of landing yours. CI passed, because the temp branch on its own was internally consistent. main blew up, because the temp branch had nothing to do with the current main.

Three properties of this bug made it unusually nasty:

  1. The PR UI lied. You reviewed +29/-34 and the commit that landed was +245/-1,137. The thing engineers approved was not the thing that merged. That breaks the most fundamental contract of a code review system.

  2. It was silent. No merge conflicts surfaced, no check failed, no banner went up on the PR. Teams only caught it when they noticed code missing from main that should have been there.

  3. It scaled with activity. The faster a repo was merging, the further its feature branches had drifted from main, and the more damage each bad merge did. The teams who most need a merge queue got hit the hardest.

The architectural lesson

The bug is structural. In GitHub's merge queue, temp branch construction lives in a separate code path from how GitHub normally merges PRs. Two code paths, two places where behavior can diverge. When the queue-specific path broke, the result was a commit on main that no other part of GitHub agreed with: not the PR view, not the diff, not the reviewer's approval.

We think about this a lot, because it generalizes. The more a dev tool reinvents its own version of an existing git operation, the more surface area it has for quiet divergence from what you actually expected. A queue can batch PRs and run CI speculatively without changing what eventually lands on main; those optimizations sit on top of a normal merge. The dangerous kind of optimization is logic that builds the merge commit itself differently from how a human would build it. That's where you get a bug no review can catch, because the thing being reviewed and the thing being merged have become two different artifacts.

That's why dev tools with write access to main can't be allowed to go rogue.

Same org, different queue

A senior engineer at a top-20 US fintech company we spoke to has many teams that operate in different ways - one of those ways is through their merge queues (GitHub's and Trunk's).

The teams on the GitHub-queued repos spent the afternoon in incident mode: auditing the commit graph, reconstructing deleted code, coordinating fixes, and so on. The team on the Trunk-backed repo had a normal Thursday; they shipped PRs and then slept through the night. Some of them didn't hear about the incident until the next morning.

We didn't do anything special during those hours. That's the point.

How Trunk Merge Queue is designed

When we built Trunk Merge Queue, we drew the architectural line carefully. The queue does plenty of work to keep things fast - optimistic merging, parallel queues, predictive testing, batching, and more - but all of it happens around the merge, not inside it. The actual merge is a normal merge. The branches we build and the merges we perform are the same ones you'd run if you merged the PR yourself. Pull a PR out of the queue, merge it by hand, and you get the same commit Trunk would have produced.


We build temp branches too (every merge queue does) to run CI on prospective merge states. The difference is that ours are pure test scaffolding: they never get pushed to main. The commit that actually lands on main is a separate operation, built the same way you'd build it yourself. We think that's the right scope for a merge queue: optimize around the merge, not inside it.

The design is deliberately boring. Anything with write access to main has to be held to a different standard than the rest of your toolchain. A linter that misbehaves costs you a few minutes, but a merge queue that misbehaves costs you commits.

There's a structural reason we can't do otherwise: Trunk isn't the platform here. We go through the same merge flow a developer uses when they click "Merge pull request" on a PR. That constraint is the guarantee - a bug in our temp-branch logic tomorrow still couldn't silently rewrite main, because we don't have the privilege to bypass a normal merge. The worst we can do is what a developer bound by branch protection settings with your repo's write access could do.

The bar for anything with write access to main

A lot of engineering work is getting delegated right now: to queues, to bots, to AI agents. That delegation is real and mostly good. It also creates a new kind of failure: tools that produce output no human would have produced, because they've drifted from the boring, well-understood operation they were supposed to be automating.

A merge queue is the purest example. As soon as it does something a human wouldn't do when merging a PR, it can silently produce commits nobody wrote. Operational fixes don't help here. A better test suite runs against the same internally-consistent temp branch. Louder monitoring sees a clean-looking commit on main. The design itself has to make divergence impossible.

Our bar for anything that writes to main: the commit it lands should be identical to what a human would land merging the same PR by hand. Speed, batching, and orchestration can all sit on top of that guarantee without breaking it. We think that's the right bar for anyone building in this category, and last Thursday is a good reminder of what happens when it slips.

Try Trunk Merge Queue

If this is the standard you want holding the door to your main branch, it's the one we built Trunk Merge Queue to meet. You can get started for free or book a demo to see it on your own setup.

Try it yourself or
request a demo

Get started for free