Running long builds in a separate worktree

A long build in the directory you are working in creates a rule nobody can follow: do not edit anything for ten minutes. Editing a file mid-build produces a mixture of old and new inputs and a result that is neither, and the failure usually surfaces later as something inexplicable. Pinning the build to its own worktree at a fixed commit removes the rule entirely — the build reads a tree that cannot change while it runs. This recipe sets that up, within Git worktrees and parallel development.

When to use this approach

  • Your full build takes long enough that waiting is not an option.
  • You have been caught editing during a build and debugging the result.
  • You want to build a release candidate while continuing on the next change.
  • Your build system writes into the repository and you want that traffic elsewhere.
  • If your build takes fifteen seconds, this is overhead; run it where you are.

Step 1 — Create a build worktree pinned to a commit

Detached and pinned is the point: nothing about this directory should move unless you move it.

git worktree add --detach ../app-build HEAD
# Verification: pinned to a specific commit, claiming no branch
git -C ../app-build rev-parse --short HEAD
git -C ../app-build symbolic-ref -q HEAD || echo "detached, as intended"
# Advance it deliberately when you want the build to see newer work
git -C ../app-build fetch origin && git -C ../app-build checkout --detach "$(git rev-parse HEAD)"
Why editing during a build produces incoherent outputA build reads files over a period of minutes. An edit landing between two reads means the compiler sees the old version of one file and the new version of another, producing an artefact that corresponds to no commit and a failure that reproduces nowhere.editorworking treebuildartefactread module Asave module A (edited)read module B (new)artefact from two treespinned worktree: one treethe resulting failure does not reproduce, because that combination never existed as a commit

Step 2 — Give it its own output directory

Two worktrees writing to the same build cache is the second way this goes wrong, and it is quieter than the first.

# Explicit output path per worktree
export BUILD_DIR="$PWD/.build"
npm run build -- --outDir "$BUILD_DIR"

# Tools that key their cache on a project name rather than a path need telling
export TURBO_CACHE_DIR="$PWD/.turbo"
export CARGO_TARGET_DIR="$PWD/target"
# Verification: the two worktrees write to different places
ls -d ../app/.build ../app-build/.build 2>/dev/null

Cache directories that default to a location under the user’s home — shared across every checkout — are the usual culprit. Checking this once per tool is worth more than debugging a corrupted incremental build later.

Step 3 — Keep dependency state consistent

The build worktree needs its own dependency tree, and it must match the commit it is pinned to rather than the one you are editing.

(cd ../app-build && npm ci)      # exactly what the pinned lockfile says
# Verification: the two worktrees may legitimately differ — confirm which is which
sha256sum package-lock.json ../app-build/package-lock.json

SAFETY WARNING — do not symlink the dependency directory into a build worktree. The build worktree is pinned to an older commit precisely so that it does not change, and a symlink makes its dependencies change whenever you install in the directory you are editing. The build then links a tree assembled from two different lockfiles, and the artefact it produces corresponds to nothing you can reproduce.

Step 4 — Run the build from the pinned tree 

# One command that records exactly what was built
commit=$(git -C ../app-build rev-parse HEAD)
(cd ../app-build && npm run build) && echo "built $commit"
# Stamp the artefact so it can be traced back later
echo "$commit" > ../app-build/.build/COMMIT
# Verification: the stamp matches the worktree's HEAD
diff <(git -C ../app-build rev-parse HEAD) ../app-build/.build/COMMIT && echo "traceable"

Stamping is a small habit with a large return: when an artefact behaves oddly a week later, the alternative to a recorded commit is guessing. The same reasoning scales to releases in linking a container image to its commit.

Two worktrees, two jobs, no interferenceThe editing worktree holds uncommitted work and a fast incremental build. The build worktree is pinned to a commit, has its own dependency tree and output directory, and produces an artefact stamped with the commit it came from.Editing worktreeyour branchuncommitted changesfast incremental buildBuild worktreedetached at a commitown dependenciesown output directorystamped artefactnothing in the right-hand column changes while the build runs Where a build cache collision comes fromA build tool that keys its cache on the absolute output path keeps two worktrees separate automatically. One that keys on a project name, or writes to a fixed directory under the user's home, will have both worktrees overwriting each other's incremental state.How does your build tool decide where its cache lives?by output pathNo collisionseparate by constructionby project nameCollidesset an explicit cache dirfixed path under HOMECollidesoverride per worktreecheck this once per tool — the symptom is a rebuild that should have been incremental

Step 5 — Reuse it rather than recreating it

Creating a worktree is cheap; installing dependencies into it is not. Keep one and re-point it.

# Move the build worktree to a new commit without recreating anything
git -C ../app-build checkout --detach "$(git rev-parse HEAD)"

# Reinstall only when the lockfile actually differs
if ! diff -q package-lock.json ../app-build/package-lock.json >/dev/null; then
  (cd ../app-build && npm ci)
fi
# Verification: dependency install is skipped when nothing changed
echo "lockfiles match: $(diff -q package-lock.json ../app-build/package-lock.json >/dev/null && echo yes || echo no)"

Validation checklist

Frequently Asked Questions

Why not just commit before building?

Committing is a fine discipline and does not solve the problem: you can still edit during the build, and the build still reads the working tree rather than the commit. The pinned worktree makes the guarantee structural instead of relying on restraint.

Does this help with continuous integration?

It is the same idea CI applies by construction — build a specific commit in a clean directory — brought to a developer machine. If your local builds and your pipeline builds disagree, a pinned worktree removes one of the likeliest explanations before you go looking for others.

What about builds that write into the source tree?

Those are exactly the builds this helps most, because the writes land in the pinned worktree rather than in the files you are editing. Add the output paths to .gitignore so the pinned worktree stays clean and git status there remains meaningful.