Automating git bisect with a test script Jump to heading#

Manual bisection works and is tedious enough that people abandon it halfway. Fourteen rounds of building, running a reproduction, interpreting the result and typing good or bad is fourteen chances to mislabel a commit, and a single mislabel sends the search into the wrong half and produces a confident wrong answer. git bisect run removes the human from the loop entirely — the only thing that needs care is the script, and specifically its exit codes. This recipe writes one, within bisect and history forensics.

When to use this approach Jump to heading#

• A regression exists and you can reproduce it with a command.
• The suspect range is more than about ten commits.
• Building and testing one commit takes less than a few minutes.
• The failure is deterministic — if it is not, start with bisecting a flaky failure.
• If the range is five commits, read them; bisection has setup cost that will not pay back.

Step 1 — Reduce the reproduction to an exit code Jump to heading#

Everything else depends on this being fast and unambiguous.

# From a full suite run to the single failing case
npm test -- --testNamePattern 'clamps the refund window' 2>&1 | tail -5
echo "exit: $?"

# Better still: a standalone script with no test-runner overhead
cat > /tmp/repro.mjs <<'JS'
import { clampRefundWindow } from './src/payments/refund.js';
const got = clampRefundWindow(120);
process.exit(got === 90 ? 0 : 1);
JS
node /tmp/repro.mjs; echo "exit: $?"

# Verification: the reproduction must be fast — this is multiplied by every step
time node /tmp/repro.mjs

Step 2 — Write the script with three outcomes Jump to heading#

Good and bad are obvious. The third — untestable — is what separates a reliable bisect from a misleading one.

#!/usr/bin/env sh
# bisect-test.sh
#   exit 0   → good (the bug is absent)
#   exit 1   → bad  (the bug is present)
#   exit 125 → skip (this commit cannot be judged)
set -eu

# A commit that does not build tells us nothing about the bug.
npm ci --silent --ignore-scripts >/dev/null 2>&1 || exit 125
npm run --silent build >/dev/null 2>&1 || exit 125

node /tmp/repro.mjs

# Verification: exercise all three paths deliberately
sh bisect-test.sh; echo "current commit: $?"

Returning 1 from a build failure is the classic mistake: Git concludes the bug is present, halves the range in the wrong direction, and the answer it eventually produces is a commit that never had the bug.

Step 3 — Verify the endpoints before starting Jump to heading#

The search is only as good as its bounds.

git switch --detach v2.6.0 && sh bisect-test.sh && echo "good endpoint confirmed"
git switch --detach main   && sh bisect-test.sh || echo "bad endpoint confirmed"

# How many commits are in play?
git rev-list --count v2.6.0..main

# Verification: expect roughly log2(range) steps
python3 -c "import math,sys; print(math.ceil(math.log2(int(sys.argv[1]))))" \
  "$(git rev-list --count v2.6.0..main)"

Step 4 — Run it, in a worktree of its own Jump to heading#

git worktree add --detach ../app-bisect main
cd ../app-bisect

git bisect start main v2.6.0
git bisect run sh ../app/bisect-test.sh

# The result, and the commit itself
git bisect log | tail -5
git show --stat "$(git rev-parse refs/bisect/bad 2>/dev/null || echo HEAD)"

# Verification: confirm by hand at the identified commit and its parent
git switch --detach "$(git rev-parse HEAD)"   && sh ../app/bisect-test.sh || echo "bad, as expected"
git switch --detach "$(git rev-parse HEAD~1)" && sh ../app/bisect-test.sh && echo "parent is good"

That final confirmation is worth the thirty seconds. A bisect result that has been verified at the commit and its parent is a fact; one that has not is a strong hypothesis.

Step 5 — Record and share the search Jump to heading#

git bisect log > /tmp/bisect-refund-window.log
head -5 /tmp/bisect-refund-window.log

# Anyone can replay it exactly, including on another machine
git bisect replay /tmp/bisect-refund-window.log

git bisect reset
cd ../app && git worktree remove ../app-bisect

SAFETY WARNING — a bisect script runs at every commit in the range, and npm ci executes whatever lifecycle scripts that commit’s lockfile specifies. Bisecting through a range that includes an untrusted contribution means executing code from every one of those commits. Use --ignore-scripts where the build permits, and run the search in a container when the range includes anything you have not reviewed.

Recording the log matters beyond reproducibility: a colleague replaying it sees exactly which commits were tested and how each was judged, which is the difference between “bisect says it was this commit” and an investigation someone else can check.

Validation checklist Jump to heading#

• The reproduction is a single command with a meaningful exit code
• The reproduction runs in seconds, not minutes
• The script returns 125 for anything it cannot judge
• Both endpoints were verified by running the script
• The search ran in a worktree, leaving the main one untouched
• The result was confirmed manually at the commit and its parent
• The bisect log was saved and can be replayed
• Lifecycle scripts are disabled or the run is contained

Frequently Asked Questions Jump to heading#

What if the reproduction needs a service or a database? Jump to heading#

Start it in the script and stop it afterwards, returning 125 if it fails to start. Keep the setup inside the script rather than around the bisect, or a service that dies midway through the search silently converts the remaining steps into false results.

Can I bisect a performance regression rather than a failure? Jump to heading#

Yes — have the script measure and compare against a threshold, exiting 1 when the measurement exceeds it. The caveat is noise: a threshold close to the normal variation produces a flaky test, and the search will converge on nothing useful. Pick a threshold well clear of the noise floor, and average several runs.

How do I handle commits where the test file itself does not exist? Jump to heading#

Keep the reproduction outside the repository, as in Step 1, so it does not change as the search moves. A test that lives in the tree changes with each commit, which means the search is testing a moving target and its results are not comparable.

Related Jump to heading#

• Bisect & History Forensics — the parent topic and the surrounding tools.
• Bisecting a Flaky Failure — when the reproduction is not deterministic.
• Tracing a Regression Through a Squashed History — when the culprit commit contains a month of work.