Batching pull requests to cut CI cost

Every pipeline run pays a fixed cost before it tests anything: provisioning a runner, checking out the repository, restoring or rebuilding dependencies, starting services. On a mature project that overhead is frequently larger than the tests themselves. Batching queue entries spreads it across several merges — but a red batch has to be bisected, so the saving is bounded by how often batches fail. This recipe finds the size that is actually cheapest for your project, extending merge queues and required checks.

When to use this approach

  • Your pipeline spends more than a third of its wall-clock time before the first test runs.
  • More than about ten pull requests merge per day, so entries are usually waiting together.
  • Your entry failure rate is low — below roughly one in ten, measured rather than assumed.
  • CI spend or runner contention is a real constraint, not a theoretical one.
  • If the fixed cost is small because of good caching, the gain is small too; start with caching dependencies keyed on the lockfile.

Step 1 — Separate fixed cost from test cost

Batching only amortises the fixed part, so measure it before predicting any saving.

# Time the phases separately in a scratch run
time git clone --filter=blob:none --no-checkout "$REPO" /tmp/bench && \
  time npm ci --prefer-offline && \
  time npm test
# Or read it from a recent run's step timings
gh run view --log | awk '/##\[group\]/{name=$0} /real|elapsed/{print name, $0}'

What this gives you: two numbers, F (fixed) and T (tests). A batch of n costs F + nT instead of n(F + T), so the saving per merge is F(n−1)/n — which flattens quickly. With a two-minute fixed cost and a three-minute suite, a batch of three already recovers two thirds of what batching can ever save, and a batch of ten recovers barely more.

Where the saving stops arrivingWith a two-minute fixed cost and a three-minute test suite, moving from one entry per run to three cuts the cost per merge sharply. Beyond about five the curve flattens, so the extra bisection risk of a large batch buys almost nothing.minutes charged per merged pull requestbatch of 15 minbatch of 24 minbatch of 33.7 minbatch of 53.4 minbatch of 103.2 minthe curve is F/n — past five you are paying bisection risk for rounding error

Step 2 — Measure the entry failure rate

The cost of a red batch is the re-runs needed to find the culprit. That cost scales with batch size, so the failure rate decides the ceiling.

# Failures among recent queue runs
gh run list --event merge_group --limit 200 --json conclusion \
  --jq '{total: length, failed: ([.[] | select(.conclusion=="failure")] | length)}'

A rate under 5% supports batches of five to eight comfortably. Between 5% and 15%, three is a sensible ceiling. Above 15%, batching is a way of spending more machine time than you save, and the real fix is upstream — usually flakiness, which is dealt with in handling flaky tests in a merge queue.

Step 3 — Configure the batch and the bisection behaviour 

# Queue settings worth being explicit about, whatever implements them:
#   max entries per batch   — start at 3, raise on evidence
#   minimum wait            — a short delay lets entries accumulate
#   failure handling        — bisect and eject, never fail the whole batch
#   timeout                 — shorter than the pipeline's worst case, not its median
# Verification: confirm multiple entries share one candidate run
gh run list --event merge_group --limit 5 --json headBranch,databaseId
git ls-remote origin 'refs/heads/gh-readonly-queue/*' | wc -l

A minimum wait is the part teams skip. Without it, the first approved entry starts a run alone and the two approvals thirty seconds later form a second batch, so the configured size is never reached.

How a short accumulation window forms a useful batchA queue with no waiting period starts a run for the first entry immediately, so later approvals arrive too late to join it. A ninety-second window lets three entries accumulate into one candidate tree without anyone perceiving a delay.First approvalentry enqueuedt+0sSecond approvaljoins the batcht+40sThird approvaljoins the batcht+80sWindow closesone candidate treet+90sThree mergesone pipeline runt+6mninety seconds of patience turns three runs into one

Step 4 — Make bisection cheap when a batch fails

Bisection re-tests subsets of the batch. You can shrink the number of runs by attributing the failure before splitting: a failing test that only touches one entry’s files is almost certainly that entry’s fault.

# Which entry touched the code the failing test covers?
failing_test=$(xmllint --xpath 'string(//testcase[failure]/@file)' reports/junit.xml)
for sha in $(git log --format=%H origin/main..HEAD); do
  git show --name-only --format= "$sha" | grep -q "$(dirname "$failing_test")" \
    && echo "candidate: $sha"
done
# Verification: a deliberately broken entry is ejected on the first split
git commit --allow-empty -m "test: force a queue failure" && echo "watch the ejection"
Bisecting a red batch of fourA candidate tree containing four entries fails. Rather than re-testing each entry alone, the queue splits the batch in half, finds the failing half in one run, and splits again — two extra runs instead of four.two extra runs, not fourbatch 1-4E1E2E3E4split AE1E2split BE3E4culpritE4halving beats one-at-a-time as soon as the batch is larger than two

Step 5 — Recheck the numbers after the pipeline changes

Batch size is derived from two measurements, and both drift. A new caching layer cuts the fixed cost and reduces the benefit of batching; a new integration suite raises the failure rate and lowers the safe ceiling.

# A monthly one-liner worth keeping in a scheduled job
gh run list --event merge_group --limit 300 --json conclusion,createdAt,updatedAt \
  --jq '{failures: ([.[] | select(.conclusion=="failure")] | length), sampled: length}'

Validation checklist

Frequently Asked Questions

Does batching make merges take longer?

Slightly, and mostly during the accumulation window. The queue wait is dominated by pipeline duration, not by batch size, so a ninety-second window is invisible next to a six-minute run. If the wait is painful, the pipeline is the thing to fix.

What if one entry in a batch is a revert?

Treat it like any other entry, but consider a bypass path for incident reverts specifically. During an outage the value of merging immediately outweighs the value of the batch, and a documented break-glass route is better than someone discovering one under pressure.

Can we batch across different pipelines?

Only if every required check runs on the candidate tree. A batch validated by a subset of the checks is not validated, and the gap will be found by the change that needed the missing one. Keep one candidate run that covers the whole required list.