Vendoring dependencies with git subtree

Vendoring — copying a dependency’s source into your repository — trades convenience for control. Every line you build is in your history, reviewed like your own code, and available even if the upstream repository disappears tomorrow. Done by hand, though, vendoring rots: someone copies a tarball in, someone else patches it, and two years later nobody knows which upstream version it was or what was changed. git subtree keeps the benefits and removes the rot. Upstream history is merged in, so each update is a normal merge with a diff, and local patches are commits you can see. This page sets that up and keeps it maintainable, within submodule and dependency integrity.

When to use this approach

  • You want a dependency’s code reviewed and versioned as part of your repository.
  • You need to carry local patches on top of upstream and still take upstream updates.
  • Submodules have caused clone, CI or reachability problems for your team.
  • The dependency is modest in size; vendoring a very large project inflates every clone. For converting an existing submodule, see converting a submodule to a subtree.

Step 1 — Add the dependency at a specific tag

Add the upstream as a named remote for readability, then add it as a subtree at a tag. Use --squash to bring in one commit representing the upstream state rather than its entire history.

git remote add -f --no-tags parser-upstream https://github.com/example/fast-parser.git
git fetch parser-upstream tag v3.2.1 --no-tags
git subtree add --prefix=vendor/fast-parser parser-upstream v3.2.1 --squash \
  -m "Vendor fast-parser v3.2.1"
# Verification: the code is present and the squash commit records the upstream revision
ls vendor/fast-parser
git log -2 --format='%h %s%n%b' -- vendor/fast-parser | grep -E 'git-subtree-(dir|split)'

The squash commit’s message contains git-subtree-dir and git-subtree-split trailers. git subtree reads them on the next pull to know where upstream was last time, so do not edit them away.

Submodule, subtree squash and subtree full historyA submodule stores only a pointer and fetches code separately. A squashed subtree stores the code plus one commit per upstream update, which keeps history light. A full-history subtree imports every upstream commit, which helps with blame but grows the repository.subtree --squashsubmodulecode in your repoyesno, a pointerclone needs upstreamnoyesupdate shows a diffyes, a mergehash changehistory addedone commit per updatenonesquash keeps the repository small while still giving every update a reviewable diff Submodule, subtree squash and subtree full historyA submodule stores only a pointer and fetches code separately. A squashed subtree stores the code plus one commit per upstream update, which keeps history light. A full-history subtree imports every upstream commit, which helps with blame but grows the repository.subtree --squashsubmodulecode in your repoyesno, a pointerclone needs upstreamnoyesupdate shows a diffyes, a mergehash changehistory addedone commit per updatenonesquash keeps the repository small while still giving every update a reviewable diff

Step 2 — Record where the code came from

Add a small manifest beside the vendored code naming the upstream URL, the tag and the commit. It is redundant with the trailers, but it is readable without Git tooling and easy to check in CI.

cat > vendor/fast-parser/VENDORED.txt <<EOF
upstream: https://github.com/example/fast-parser.git
version:  v3.2.1
commit:   $(git rev-parse parser-upstream/v3.2.1^{commit} 2>/dev/null || git ls-remote parser-upstream refs/tags/v3.2.1 | cut -f1)
patches:  none
EOF
git add vendor/fast-parser/VENDORED.txt && git commit -m "Record fast-parser vendoring metadata"

Step 3 — Pull upstream updates as reviewable merges

Updating is a subtree pull to a new tag. The result is a merge whose diff shows exactly what changed upstream, which is what you review.

git switch -c vendor/fast-parser-v3.3.0
git fetch parser-upstream tag v3.3.0 --no-tags
git subtree pull --prefix=vendor/fast-parser parser-upstream v3.3.0 --squash \
  -m "Update vendored fast-parser to v3.3.0"
sed -i 's/^version:.*/version:  v3.3.0/' vendor/fast-parser/VENDORED.txt
git commit -am "Bump fast-parser metadata to v3.3.0"
# Review just the upstream changes
git diff HEAD~2 HEAD -- vendor/fast-parser ':!vendor/fast-parser/VENDORED.txt' | less
Vendored history with squashed updatesEach upstream version enters the repository as one squash commit, merged into the main line. Local patches sit on the main line between updates, so a later subtree pull merges the new upstream squash with those patches like any other merge.upstream squashes merged into your historymainaddpatchmergemergeupstream squashv3.2.1v3.3.0v3.4.0local patchesfix-1fix-2local patches survive updates because they are ordinary commits on your side of the merge Vendored history with squashed updatesEach upstream version enters the repository as one squash commit, merged into the main line. Local patches sit on the main line between updates, so a later subtree pull merges the new upstream squash with those patches like any other merge.upstream squashes merged into your historymainaddpatchmergemergeupstream squashv3.2.1v3.3.0v3.4.0local patchesfix-1fix-2local patches survive updates because they are ordinary commits on your side of the merge

Step 4 — Keep local patches visible

Patches to vendored code should be separate commits with a recognisable prefix, and listed in the manifest. When upstream later includes the same fix, the merge will either apply cleanly or conflict, and either way you know which patch to drop.

git commit -m "vendor(fast-parser): fix overflow on empty input

Upstream-Issue: example/fast-parser#412
Drop-When: upstream releases a fix" -- vendor/fast-parser/src/lexer.c

# List carried patches at any time
git log --format='%h %s' --grep='^vendor(fast-parser)' -- vendor/fast-parser

If a patch is worth carrying, it is usually worth sending upstream too. git subtree split --prefix=vendor/fast-parser produces a branch with only the subtree’s history, from which patches can be cherry-picked into a fork for an upstream pull request.

Step 5 — Check vendored code in CI

Two checks keep vendoring honest: the manifest must match the last subtree trailer, and changes under the vendored directory must come either from a subtree merge or from a commit with the patch prefix.

# Manifest version matches the last squash commit
last=$(git log -1 --grep='git-subtree-dir: vendor/fast-parser' --format=%B | sed -n 's/^git-subtree-split: //p')
grep -q "$last" vendor/fast-parser/VENDORED.txt || echo "manifest out of date"

# Changes under vendor/ in this PR are subtree merges or prefixed patches
git log --format='%h %s' "$BASE..$HEAD" -- vendor/fast-parser |
  grep -vE 'Update vendored|vendor\(fast-parser\)|Bump fast-parser metadata' && exit 1 || true
What CI checks on vendored codeThe manifest must name the same upstream commit as the last subtree trailer. Changes under the vendored path must be either a subtree update or a clearly labelled local patch. Anything else is an unexplained edit to third-party code.Manifestmatches trailerAllowed changessubtree updateprefixed patchRejectedunlabelled editan unexplained edit to vendored code is exactly what tampering looks like What CI checks on vendored codeThe manifest must name the same upstream commit as the last subtree trailer. Changes under the vendored path must be either a subtree update or a clearly labelled local patch. Anything else is an unexplained edit to third-party code.Manifestmatches trailerAllowed changessubtree updateprefixed patchRejectedunlabelled editan unexplained edit to vendored code is exactly what tampering looks like

Validation checklist

Frequently Asked Questions

Should I use --squash or import full upstream history?

Squash for most third-party code: the repository stays small and each update is one reviewable unit. Import full history only when you need git blame to reach upstream authors or plan to contribute heavily, accepting the extra size.

Does git subtree need to be installed separately?

It ships in Git’s contrib directory and is packaged with Git on most systems. If git subtree is not found, install your distribution’s Git contrib package or the subtree script.

What happens if someone edits vendored files without the prefix?

The CI check fails. If the edit is legitimate, amend the commit message to the patch convention and add it to the manifest; if not, it is a red flag worth investigating.