Repository Migration & Consolidation Jump to heading
Repositories outlive the decisions that created them. A service split off five years ago now changes only alongside its caller; a team that maintained four repositories has become one team; a migration from an older version control system left commits attributed to usernames nobody recognises. Each of these is a repository-shaped problem with a Git-shaped answer, and each one is irreversible enough to deserve a rehearsal. This part of Git Workflow Architecture & Branching Strategies covers the four shapes migrations take and the verification that tells you a migration succeeded.
Prerequisites Jump to heading
The Four Shapes Jump to heading
Almost every migration is one of four operations, and naming yours first prevents applying the wrong technique.
The distinction that matters most is whether commit identifiers survive. A move preserves them, so every existing clone, tag, signature and external reference stays valid. The other three rewrite history, which invalidates signatures, breaks links in issue trackers, and requires everyone to reclone.
Step 1 — Take a Backup That Is Actually Complete Jump to heading
An ordinary clone does not include every ref. A mirror does.
git clone --mirror https://github.com/acme/service-a.git backup/service-a.git
du -sh backup/service-a.git# Verification: refs and objects should match the source exactly
git --git-dir=backup/service-a.git for-each-ref | wc -l
git ls-remote https://github.com/acme/service-a.git | wc -lSAFETY WARNING — history rewriting tools operate on the repository in place and are not reversible.
git-filter-repodeliberately refuses to run on a clone that has a remote configured, precisely to stop an accidental force-push of a rewritten history over the original. Work on a fresh mirror, keep the untouched backup elsewhere, and do not remove it until the migration has been live for a full release cycle.
Step 2 — Rehearse the Whole Thing Jump to heading
A migration has one rehearsal cost and an unbounded failure cost. Run it end to end against copies.
# A scratch area that is safe to destroy
rm -rf /tmp/migration && mkdir -p /tmp/migration && cd /tmp/migration
git clone --no-local backup/service-a.git work-a# Record what you will compare against afterwards
git -C work-a rev-list --count --all > /tmp/before-commits.txt
git -C work-a log --format='%H %T' --all | sort > /tmp/before-trees.txtThe tree hashes are the useful artefact. Commit ids change during a rewrite; tree hashes do not unless the content changed, so comparing them proves the migration moved content faithfully.
Step 3 — Preserve the Property That Matters for Your Shape Jump to heading
Each shape has one property whose loss is the usual complaint afterwards.
# Merge: keep both histories and rewrite paths so nothing collides
git -C combined remote add service-a ../work-a && git -C combined fetch service-a
git -C combined merge --allow-unrelated-histories service-a/main# Split: filter to a subdirectory, keeping only the commits that touched it
git filter-repo --path services/billing/ --path-rename services/billing/:# Convert: map identities before the first commit is written, not afterwards
git filter-repo --mailmap ../authors.map# Move: nothing is rewritten at all
git push --mirror git@newhost:acme/service-a.gitStep 4 — Verify Before Publishing Jump to heading
# Commit counts: expect a drop for a split, equality for a move
git -C result rev-list --count --all
cat /tmp/before-commits.txt# Tree comparison: the content that survived must be byte-identical
git -C result log --format='%T' --all | sort -u > /tmp/after-trees.txt
comm -13 <(cut -d' ' -f2 /tmp/before-trees.txt | sort -u) /tmp/after-trees.txt | head# The final state must match the source exactly where nothing was filtered out
git -C result archive HEAD | sha256sum
git -C work-a archive HEAD | sha256sumAn archive hash comparison is the bluntest and most convincing check available: identical output means the working tree at the tip is identical, whatever happened to the commits behind it.
Step 5 — Plan the Cutover, Not Just the Rewrite Jump to heading
The technical work is usually the smaller half. Everything pointing at the old repository has to move too.
What Gets Lost, and How to Soften It Jump to heading
Three things do not survive a history rewrite, and being explicit about them beforehand avoids the awkward discovery afterwards.
Signatures are the first. Every signed commit’s signature covers its exact object, so a rewrite invalidates all of them — the commits remain, the verification does not. There is no way to re-sign someone else’s commit honestly, so the usual approach is to sign the migration commit itself, tag the result, and treat the pre-migration history as attested by the tag rather than commit by commit. The mechanics are in signing and verifying release tags.
External references are the second. Issue trackers, code review threads, incident write-ups and documentation all cite commit ids that will no longer resolve. A mapping file from old id to new is produced by git-filter-repo automatically, and publishing it somewhere durable turns a broken link into a lookup. It is a small piece of work that pays back for years.
Local clones are the third, and the most disruptive day to day. After a rewrite, a colleague’s git pull produces a divergence that Git cannot reconcile, and the resulting merge — if anyone completes it — reintroduces the entire old history. Say clearly that the instruction is to reclone, not to pull, and consider protecting against it: an updated default branch whose first commit is unrelated will refuse to merge without --allow-unrelated-histories, which is a useful accident.
Finally, remember that the repository is not the only thing referencing itself. Submodule pointers in other repositories, pipeline configuration, deployment manifests, container build files and dependency declarations may all name the old location or an old commit. Searching the wider codebase for the repository’s name before the cutover is a ten-minute task that routinely finds three or four references nobody remembered — and finding them afterwards means finding them through a failure.
Configuration Reference Jump to heading
| Tool or option | Effect | When to use |
|---|---|---|
git clone --mirror | Complete copy of every ref | Always, before anything else |
git filter-repo --path | Keeps only matching paths | Splitting out a subdirectory |
git filter-repo --path-rename | Moves paths during the rewrite | Merging, to avoid collisions |
git filter-repo --mailmap | Rewrites author and committer identities | Conversions and cleanups |
merge --allow-unrelated-histories | Joins two root commits | Merging repositories |
git push --mirror | Pushes every ref as-is | Moving hosts without rewriting |
.mailmap | Maps identities at read time, no rewrite | When a rewrite is not justified |
Troubleshooting Jump to heading
| Symptom | Likely cause | Fix |
|---|---|---|
refusing to destructively overwrite repo history | filter-repo sees a configured remote | Work on a fresh clone with no remote |
| Merged repository has colliding paths | No path rename before the merge | Rewrite paths in each source first |
| Commit count far lower than expected after a split | Commits that never touched the path were dropped | Expected behaviour; verify by tree instead |
| Colleagues’ pulls create enormous merges | They pulled instead of recloning | Announce reclone; consider an unrelated root |
| Tags missing on the new host | push --mirror not used, or tags excluded | Push refs explicitly, including refs/tags/* |
| Signatures all show as unverified | History was rewritten | Expected; attest the result with a signed tag |
Frequently Asked Questions Jump to heading
Can we correct author emails without rewriting history? Jump to heading
Yes, with a .mailmap file, which maps identities at display time. git log, git shortlog and most forges honour it, the commits are untouched, and no clone breaks. It is almost always the right first answer, and a rewrite is justified only when the stored identities themselves must change — a legal requirement, or a migration where the old identities are meaningless.
Is it safe to merge two repositories that share files? Jump to heading
Safe, but you must rewrite paths first, or the merge will present every shared path as a conflict between two unrelated histories. Move each source into its own subdirectory during the rewrite, merge, and then move files to their final locations in an ordinary commit afterwards.
How long should the old repository stay available? Jump to heading
Read-only, indefinitely if you can afford it; archived rather than deleted. The cost is negligible and the value shows up unpredictably — an audit, an old branch someone needs, a reference in a document nobody thought to update. Deleting it converts a lookup into a loss.
Does splitting a repository preserve blame? Jump to heading
Within the extracted paths, yes: commits that touched those files are kept with their content, so git blame continues to work. What is lost is the context of changes to other parts of the original repository, which is usually the point of splitting — and is worth saying out loud, because “we can still see the history” means something narrower afterwards.
How do we decide whether a migration is worth doing at all? Jump to heading
Weigh it against what it costs the people who are not doing it. A migration is a day or two of concentrated work for whoever runs it and an interruption for everyone else: reclones, broken links, pipelines that need updating, and a period where nobody is quite sure which repository is authoritative. That cost is worth paying when the current arrangement imposes a recurring tax — coordinated changes across repositories that always land out of order, a history nobody can search, a hosting arrangement that blocks work.
It is not worth paying for tidiness. A repository whose layout offends someone’s sense of order, but which nobody trips over, is a repository to leave alone. The clearest signal that a migration is justified is that people have already invented workarounds: a script that clones three repositories in the right order, a documented procedure for landing a change across two of them, a convention about which one to update first. Those workarounds are the recurring cost made visible, and removing them is what the migration actually buys.
When the decision is genuinely marginal, the cheaper half is often enough. Merging two repositories is a large operation; adding a .mailmap, marking generated files, or fixing the ownership rules costs an afternoon and removes a good share of the friction that prompted the question.
Related Jump to heading
- Merging Two Repositories While Keeping History — the consolidation case, step by step.
- Migrating From Subversion to Git — conversion, including branch and tag reconstruction.
- Moving a Repository Between Hosting Providers — the shape that preserves every commit id.
- Rewriting Author Emails During a Migration — identity mapping and when to avoid it.
- Archiving a Repository Without Losing History — retiring the source safely.