Keeping trunk green

Trunk-based development rests on one promise: trunk always works. Everyone branches from it several times a day, releases are cut from it, and some teams deploy it continuously. When trunk breaks, every new branch inherits the breakage, every pull request’s CI goes red for reasons unrelated to its change, and people stop trusting the pipeline. A green trunk is not a matter of everyone being careful. It comes from a few mechanisms: testing what will actually land rather than what was pushed, landing through a queue, treating a red trunk as an incident with a fix-or-revert rule measured in minutes, and measuring how often trunk is green so drift is noticed. This page sets them up, within trunk-based development setup.

When to use this approach

  • Your team practises trunk-based development, or deploys from main.
  • Main is red several times a week, and nobody is sure who should fix it.
  • Pull requests pass their checks but main breaks after merging them.
  • Branch CI is often red because of breakage inherited from main.

Step 1 — Test the merge result, not the branch

The most common way a green pull request breaks trunk is a semantic conflict with something merged after its checks ran. Test the merge of each pull request into current trunk, and make sure what lands is what was tested.

# pull_request workflows test refs/pull/<n>/merge by default — keep it that way
on: pull_request
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4      # do not override ref with the PR head
      - run: make test

The mechanics and the stale-merge problem are covered in catching semantic conflicts in CI.

Mechanisms that keep trunk greenTesting merge results catches breakage before landing. A merge queue ensures the tested combination is exactly what lands. A fix-or-revert rule limits how long any breakage lasts. Measuring green time shows whether the mechanisms are working.prevention first, fast recovery secondTest merge resultscatch before landingMerge queuetested = landedFix-or-revert ruleminutes, not hoursMeasure green timenotice driftno layer is enough alone — prevention misses some, recovery bounds the rest

Step 2 — Land through a merge queue

When several pull requests merge in quick succession, each was tested against a trunk that no longer exists. A merge queue builds and tests the combinations in order and lands only what passed, closing that window entirely.

# Is main protected by a merge queue?
gh api "repos/$OWNER/$REPO/rules/branches/main" --jq '.[] | select(.type=="merge_queue") | .parameters'

Setting one up is covered in setting up a GitHub merge queue, and GitLab’s equivalent in GitLab merge trains.

Step 3 — Treat red trunk as an incident: fix or revert in minutes

Some breakage will get through — a flaky dependency, an environment change, a test that only fails on main. The rule that keeps the impact small: whoever’s change broke trunk fixes it within a short window, and if a fix is not ready, the change is reverted. Reverting is not a judgement; it is the fastest way back to green.

# Identify the change that turned main red, then revert it
last_green=$(gh run list --branch main --workflow ci.yml --status success --limit 1 --json headSha --jq '.[0].headSha')
git log --oneline --first-parent "$last_green..origin/main"
git revert -m 1 <suspect-merge> && git push origin HEAD:refs/heads/revert-suspect && gh pr create --fill
Trunk just went red — what now?If the breaking change is obvious and a fix is a few minutes away, the author fixes forward. If the cause is clear but a fix will take longer, revert the change immediately and re-land it later. If the cause is unclear, revert the most recent merges since the last green run and bisect off trunk.Is the cause known and the fix quick?yes, minutesFix forwardsmall PR, fast pathknown, fix is slowRevert nowre-land laterunknownRevert since greenbisect off trunktime on red matters more than whose change it was

Reverting merge commits correctly is covered in undoing a bad merge on a shared branch.

Step 4 — Deal with flaky tests separately

Flaky tests make trunk look red when nothing is broken, which teaches people to ignore red builds — the worst outcome. Quarantine flaky tests quickly: track them, move them out of the required set, and fix them on a schedule.

# Tests that failed and then passed on retry for the same commit, last 2 weeks
gh run list --branch main --limit 200 --json headSha,conclusion,attempt \
  --jq 'group_by(.headSha)[] | select((map(.conclusion) | index("failure")) and (map(.conclusion) | index("success"))) | .[0].headSha'

Flaky test handling in queues is covered in handling flaky tests in a merge queue.

Step 5 — Measure how green trunk is

Track the share of time trunk is green and how long red periods last. A weekly number makes regressions visible before they become normal.

# Share of main CI runs that succeeded in the last 30 days, and the longest red streak
gh run list --branch main --workflow ci.yml --limit 300 --json conclusion,createdAt \
  --jq 'map(select(.createdAt > (now - 30*86400 | todate))) | sort_by(.createdAt) |
        (map(select(.conclusion=="success")) | length) as $g |
        "green: \($g)/\(length)"'
Trunk green rate before and after the mechanismsAn illustrative team over three quarters. Before testing merge results and using a queue, trunk was green on about four in five runs. With the merge queue and a fix-or-revert rule, it rose above ninety-five percent, and red periods shrank from hours to minutes.share of green runs on main (illustrative)Q1, no queue81%Q2, merge results90%Q3, + queue + revert rule97%the remaining few percent are mostly external — dependencies and infrastructure

Step 6 — Make the rule visible and shared

Write the fix-or-revert rule into the team’s working agreement, with the time window, and show trunk’s status where everyone sees it — a channel notification on red, a badge on the repository. Shared visibility turns keeping trunk green into a team habit rather than one person’s job.

# Notify the team channel when main turns red
on: { workflow_run: { workflows: ["CI"], types: [completed], branches: [main] } }
jobs:
  notify:
    if: github.event.workflow_run.conclusion == 'failure'
    runs-on: ubuntu-latest
    steps:
      - run: curl -fsS -m 5 -X POST "$CHAT_WEBHOOK" -d "{\"text\":\"main is red: ${{ github.event.workflow_run.html_url }}\"}"
        env: { CHAT_WEBHOOK: "${{ secrets.CHAT_WEBHOOK }}" }

Validation checklist

Frequently Asked Questions

Who is responsible when trunk breaks?

The author of the breaking change, with the whole team responsible for noticing. If the author is unavailable, anyone may revert — reverting is safe and reversible.

Should we block merging while trunk is red?

Merge queues effectively do this, because candidates are tested on top of trunk and fail while it is red. Without a queue, pausing merges until trunk is green prevents piling more changes onto a broken base.

Is post-merge CI still needed with a queue?

Yes, for slow or environment-dependent suites that do not fit in the queue. Treat failures there with the same fix-or-revert rule.