Catching semantic conflicts in CI

Two pull requests pass CI. Both are approved. Both merge within an hour of each other, without a conflict. Main is now broken, because one renamed a function the other started calling, or one changed a default the other depends on. Neither pull request was wrong; their combination was, and CI never tested the combination. Fixing this is mostly configuration: test the merged result rather than each branch, make sure the tested result is the one that lands, and add a handful of fast static checks for the shapes semantic conflicts usually take. This page sets up all three, within semantic conflicts and merge verification.

When to use this approach

  • Main has broken after merging pull requests that were each green.
  • Bisect has landed on merge commits whose two sides each pass, as described in bisecting across merges with --first-parent.
  • Your CI checks out the pull request’s head commit rather than its merge with the base branch.
  • Several pull requests often merge in quick succession on the same branch.

Step 1 — Check what your CI actually tests

Many pipelines test the pull request head, not the merged result. Find out which yours does by printing the checked-out commit’s parents in a job.

# In a CI job on a pull request
git log -1 --format='commit %h, parents: %p'
# one parent  -> testing the branch head (does not see main's newer changes)
# two parents -> testing a merge of the branch into its base
Testing the head against testing the mergeTesting the pull request head checks the branch against the main it branched from, so changes merged to main since are invisible. Testing the merge result checks the branch combined with main as it is now, which is where semantic conflicts appear.Test PR headTest merge resultcode under testbranch on old basebranch + current mainsees renames on mainnoyessees new main callersnoyesdefault on GitHub PRsonly if configuredpull_request checkoutmost semantic conflicts are invisible from the head and obvious from the merge

On GitHub, actions/checkout in a pull_request workflow checks out the synthetic merge ref by default. Pipelines that pass ref: ${{ github.event.pull_request.head.sha }} — often added to make something else work — lose that. GitLab offers merged-results pipelines for the same purpose.

Step 2 — Close the window between test and merge

The merge ref is computed when the pull request is updated. If another pull request merges afterwards, the tested combination is no longer what will land. Three options close the gap, with different costs.

Choosing how to keep the tested merge currentRequiring branches to be up to date forces a rebase or merge and re-test whenever main moves, which is simple but slow on busy branches. A merge queue builds and tests the exact final combination automatically. Re-testing on a schedule catches problems after the fact rather than preventing them.How busy is the target branch?a few merges a dayRequire up to datere-test on base changemany merges a dayMerge queuetests final combinationcan't change processPost-merge CIdetect quickly, revertonly the first two prevent a broken main; the third shortens how long it stays broken 
# Require branches to be up to date before merging (classic protection, strict checks)
gh api -X PATCH "repos/$OWNER/$REPO/branches/main/protection/required_status_checks" -F strict=true

For busy branches, a merge queue is the better tool: it batches pull requests, builds the combined result, and lands only what passed. Setting one up is covered in setting up a GitHub merge queue, and GitLab’s equivalent in GitLab merge trains.

Step 3 — Add fast static checks on the merged tree

Some semantic conflicts have recognisable shapes that a type checker or a few lines of script catch in seconds, before the test suite runs. Run them as their own job on the merge result, so they report quickly.

jobs:
  merge-sanity:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4            # the merge ref
        with: { fetch-depth: 0 }
      - name: Type check the merged code
        run: mypy src/ || npx tsc --noEmit
      - name: Duplicate migration numbers
        run: |
          d=$(ls db/migrations | sed -n 's/^\([0-9]\{4,\}\)_.*/\1/p' | sort | uniq -d)
          [ -z "$d" ] || { echo "duplicate migrations: $d"; exit 1; }
      - name: Duplicate route paths
        run: |
          d=$(grep -rhoE "path\(['\"][^'\"]+" src/routes | sort | uniq -d)
          [ -z "$d" ] || { echo "duplicate routes: $d"; exit 1; }

A type checker is the single most effective tool here: renamed functions, changed signatures and removed attributes are all type errors in the merged code, even though neither branch had any.

Step 4 — Write tests that exercise interactions

Static checks catch structural breakage. Behavioural semantic conflicts — a changed default, a changed ordering assumption — need tests that exercise one module through another. Look at past incidents for the interactions that broke, and add a test at each.

# tests/integration/test_renewal_billing.py
def test_renewal_charges_through_current_billing_api(account_factory, billing):
    account = account_factory(plan="pro")
    renew_subscription(account)                     # subscriptions module
    assert billing.charges_for(account) == [account.plan.price]   # billing module

Tests like this are what fail on the merge result when one side changed the billing API and the other added a new renewal path. They are cheap insurance on the boundaries that change most often.

Step 5 — Recognise and fix a semantic conflict that got through

When main breaks after a clean merge, confirm it is a semantic conflict: each side passes on its own, the merge fails. Then fix forward with a small commit that reconciles the two changes, rather than reverting one side.

m=$(git log --merges -1 --format=%h main)
git checkout -q "$m^1" && make test >/dev/null && echo "main before merge: pass"
git checkout -q "$m^2" && make test >/dev/null && echo "branch alone: pass"
git checkout -q "$m"   && make test >/dev/null || echo "merge: FAIL — semantic conflict"
git checkout -q main
A semantic conflict, caught by the queueTwo pull requests each pass CI against main. The merge queue builds them together, the type check fails because one renamed a function the other calls, and the queue ejects the second pull request. Its author updates the call and it lands in the next batch.PR A (rename)PR B (new call)merge queuemainenqueued, greenenqueued, greenbuild A+B: type errorland A onlyejected: update callwithout the queue, both land and main breaks; with it, B's author fixes a one-line call

Validation checklist

Frequently Asked Questions

Does testing the merge ref slow CI down?

Not by itself — it replaces testing the head. The extra cost comes from re-testing when main moves, which is why busy branches use merge queues that batch several pull requests per run.

Why not just run CI again after every merge to main?

Do that too; it detects breakage quickly. But it detects rather than prevents, so main is broken until someone reverts or fixes forward. Testing the merge before landing keeps main green.

Can a type checker catch semantic conflicts in dynamic languages?

Only as far as the code is annotated. Even partial typing catches most renamed and removed functions. Where there are no types, a smoke test that imports every module and exercises main entry points catches a surprising share.