Migrating large binaries to Git LFS Jump to heading#

Git stores a complete copy of every version of every file. For text that is fine — deltas compress beautifully. For a 40 MB design file edited weekly, it means the repository grows by roughly 40 MB a week forever, and every clone pays for all of it. Git LFS replaces the file’s content in the object store with a small text pointer and keeps the real bytes in a separate store fetched on checkout. This recipe carries out that move, one of the interventions catalogued in Large Repository Performance, and is careful about the part that trips people: tracking a pattern does nothing to the copies already in history.

When to use this approach Jump to heading#

• The largest-blob listing shows a handful of file types dominating the repository.
• Those files genuinely need versioning next to the source — assets, fixtures, models — rather than belonging in artefact storage.
• The files change over time. A single large file committed once and never touched costs little; the same file edited weekly is what compounds.
• Your host supports LFS and you understand its storage and bandwidth quotas, which are usually billed separately.
• If the files are build outputs, stop here: publish them as artefacts instead, and reference them by commit.

Step 1 — Find what is actually large Jump to heading#

# The ten heaviest blobs anywhere in history, with paths
git rev-list --objects --all \
  | git cat-file --batch-check='%(objecttype) %(objectsize) %(rest)' \
  | awk '$1=="blob" {print $2, $3}' \
  | sort -rn | head -10

# Total bytes per extension — this is what tells you which patterns to track
git rev-list --objects --all \
  | git cat-file --batch-check='%(objecttype) %(objectsize) %(rest)' \
  | awk '$1=="blob" && $3 ~ /\./ {n=split($3,a,"."); s[a[n]]+=$2} END {for (e in s) printf "%12d  .%s\n", s[e], e}' \
  | sort -rn | head -10

What changed: nothing yet — but the second command is the one that determines the migration. It aggregates by extension across all of history, so a thousand medium-sized PNGs correctly outrank one enormous file that was committed once.

Step 2 — Choose between rewriting history and starting today Jump to heading#

This is the decision that determines everything else, and the two options are not variations on a theme.

Choose “from today” unless the repository is genuinely painful to work with right now. It is reversible, invisible to colleagues, and stops the growth immediately — and you can always do the rewrite later, at a moment you have planned for.

Step 3 — Track the patterns and convert the files Jump to heading#

# Install and initialise once per machine
git lfs install

# Track the extensions the Step 1 analysis identified
git lfs track "*.psd"
git lfs track "*.mp4"
git lfs track "assets/renders/**"

# .gitattributes is what makes this shared — it MUST be committed
git add .gitattributes
git commit -m "chore: track large asset types with Git LFS"

What changed: from this commit forward, matching files are stored as pointers and their bytes go to the LFS store. Files already committed are untouched.

git lfs track                 # list the active patterns
cat .gitattributes            # each line ends with filter=lfs diff=lfs merge=lfs -text

To convert files that are already in the working tree, re-add them so the filter runs:

# Re-stage existing matching files so they are rewritten as pointers
git add --renormalize .
git status --short            # matching files appear as modified
git commit -m "chore: move existing assets to LFS pointers"

If you chose the rewrite path instead:

# Rewrite every commit on every ref, replacing matching blobs with pointers
git lfs migrate import --everything --include="*.psd,*.mp4"

# Inspect the result BEFORE pushing anything
git count-objects -vH
git log --oneline -5

SAFETY WARNING — git lfs migrate import rewrites history: every commit SHA after the earliest rewritten commit changes, open branches and pull requests are orphaned, and force-pushing is required. Run it on a fresh mirror clone first, verify the result, and schedule the cutover with the team. If your remote blocks force pushes — as it should — you will need the documented override, and everyone must re-clone afterwards.

Step 4 — Verify pointers are stored, not contents Jump to heading#

# What Git stores for the file — expect a 3-line pointer, not binary
git show HEAD:assets/hero.psd | head -3
# version https://git-lfs.github.com/spec/v1
# oid sha256:9f2c...
# size 41943040

# What is on disk — expect the real file
file assets/hero.psd
ls -lh assets/hero.psd

# Which LFS objects this checkout knows about
git lfs ls-files | head

The distinction in that first command is the whole mechanism: Git’s object store holds three lines of text, the working tree holds 40 MB, and a filter converts between them on checkout and commit. If git show prints binary, the file was committed before the pattern was tracked and needs the --renormalize step.

Step 5 — Prevent the problem from returning Jump to heading#

A migration without prevention buys about a year. Two guards close the loop.

# A local guard: refuse to commit an untracked large file
cat > .git/hooks/pre-commit <<'HOOK'
#!/bin/sh
limit=$((5 * 1024 * 1024))
git diff --cached --name-only --diff-filter=A | while read -r f; do
  [ -f "$f" ] || continue
  size=$(wc -c < "$f")
  [ "$size" -le "$limit" ] && continue
  git check-attr filter "$f" | grep -q 'filter: lfs' && continue
  echo "$f is $((size / 1024 / 1024)) MB and is not tracked by LFS" >&2
  exit 1
done
HOOK
chmod +x .git/hooks/pre-commit

What changed: a new large file that is not covered by an LFS pattern is refused locally, with a message naming the file and its size.

head -c 8000000 /dev/urandom > test.bin && git add test.bin && git commit -m "probe"
# Expect: test.bin is 7 MB and is not tracked by LFS
git reset && rm test.bin

The local hook is a convenience and is skippable; the authoritative version belongs on the server, as described in Enforcing File Size Limits on the Remote. Run both — the local one so contributors find out immediately, the server one so the rule actually holds.

Validation checklist Jump to heading#

• .gitattributes is committed and lists every pattern you intend to track
• git show HEAD:<asset> prints a pointer, not binary
• The working tree still contains the real files, and builds pass
• Git LFS is installed in every CI image that checks out these paths
• A size guard rejects new untracked large files, locally and on the remote
• If history was rewritten, everyone has re-cloned and open branches were re-created

Frequently Asked Questions Jump to heading#

Does moving files to LFS shrink the existing repository? Jump to heading#

Only if you rewrite history. Tracking a pattern from today onward means new versions are stored as pointers, but every version already committed stays exactly where it is, so the clone size is unchanged. Rewriting with git lfs migrate import rewrites past commits to reference pointers instead — which does shrink the repository, and changes every SHA after the rewrite point.

What happens for someone without Git LFS installed? Jump to heading#

They get the pointer file — a few lines of text naming the object and its size — instead of the asset. Nothing is corrupted and nothing is lost, but a build that expects a real file will fail in a confusing way. Add Git LFS to the contributing guide and to CI images before migrating, and consider a check that fails clearly if a pointer file reaches a build step.

Is LFS the right home for build outputs? Jump to heading#

Usually not. LFS is for large files that genuinely need versioning alongside the source — design assets, test fixtures, models. Build outputs are reproducible from the source and belong in artefact storage keyed by commit, where they can be expired on a retention policy. Putting them in LFS moves the storage cost without questioning whether the cost should exist.

Related Jump to heading#

• Large Repository Performance — the parent guide, and how to tell whether size is really your problem.
• Enforcing File Size Limits on the Remote — the server-side guard that makes this migration a one-time event.
• Speeding Up Clones with Partial Clone — what to do about the historical weight you decided not to rewrite.