Diffing binary formats with textconv

Binary files differ is technically true and completely useless. A spreadsheet whose numbers changed, a PDF whose text was edited, a compiled asset whose metadata moved — all produce the same line, and reviewing any of them means downloading both versions and opening them side by side. textconv lets Git run a converter before diffing, so the diff shows the extracted text instead of the bytes. It does not make the file mergeable, and knowing that boundary is most of using it well. This recipe covers both, within merge strategies and gitattributes.

When to use this approach

  • Reviews stall because a changed binary cannot be inspected.
  • The format has a text representation a command can extract.
  • You want git log -p and git blame to work on these files.
  • The files change often enough that opening two copies is a real cost.
  • If the format has no meaningful text representation — a photograph, say — textconv cannot help; mark it binary and review it another way.

Step 1 — Find a converter that is deterministic

The converter runs on every diff, so it must be fast and must produce identical output for identical input.

# Common converters, by format
pdftotext -layout file.pdf -        # poppler
in2csv file.xlsx                    # csvkit
exiftool -s file.jpg                # metadata only, for images
unzip -p file.docx word/document.xml | xmllint --format -
# Verification: the same input must produce identical output twice
pdftotext -layout spec.pdf - | sha256sum
pdftotext -layout spec.pdf - | sha256sum

A converter that embeds a timestamp or a temporary path fails this check and will make every file look changed. Most tools have a flag to suppress that; find it before going further.

Where textconv sits in a diffGit reads both blobs from the object database, runs each through the configured converter, and diffs the resulting text. The repository still stores the original binary — nothing is converted on the way in.Two blobsstored unchangedtextconvruns per bloboutput cachedTextextracted representationReadable diffin review and logthe object database is untouched — this changes only how Git shows the content

Step 2 — Define the filter and attach it to paths 

git config diff.pdf.textconv 'pdftotext -layout -q'
git config diff.pdf.cachetextconv true
git config diff.pdf.binary true

git config diff.xlsx.textconv 'in2csv'
git config diff.xlsx.cachetextconv true
cat >> .gitattributes <<'ATTR'
*.pdf   diff=pdf
*.xlsx  diff=xlsx
*.docx  diff=docx
ATTR
git add .gitattributes && git commit -m 'chore: readable diffs for document formats'
# Verification: a change to a PDF now shows as text
git diff HEAD~1 -- docs/spec.pdf | head -20

cachetextconv stores the converted output in the object database’s notes, so a second diff of the same blob is instant. Without it, git log -p over a directory of documents runs the converter once per blob per invocation, which is slow enough to be abandoned.

Step 3 — Confirm the cache is working 

# First run populates the cache
time git log -p --follow -- docs/spec.pdf >/dev/null
# Second should be much faster
time git log -p --follow -- docs/spec.pdf >/dev/null
# The cache lives in a notes ref
git notes --ref=textconv list | wc -l
# Verification: clearing it and re-running restores the original timing
git notes --ref=textconv prune 2>/dev/null; time git log -p -- docs/spec.pdf >/dev/null
Time to run git log -p over forty documentsWithout caching, the converter runs once per blob on every invocation. With caching enabled the converted text is stored alongside the object, so repeat invocations read it directly and the command becomes usable interactively.seconds for git log -p over 40 PDFsno textconv (binary files differ)0.2 stextconv, cache cold47 stextconv, cache warm1.1 sthe middle bar is why cachetextconv is not optional in practice

Step 4 — Know what textconv does not do

It affects diffs, logs and blame. It does not affect merging, and it does not make the file mergeable.

# Diff: converted. Merge: still binary.
git check-attr diff merge -- docs/spec.pdf
# So a conflict in the file is still a conflict with no textual resolution
git merge other-branch 2>&1 | grep -i 'spec.pdf'
# Pair it with a driver that keeps both versions for a person to reconcile
git config merge.keepboth.name 'keep both versions for manual reconciliation'
git config merge.keepboth.driver 'cp "%A" "%A.ours" && cp "%B" "%A.theirs" && exit 1'
printf '*.pdf merge=keepboth\n' >> .gitattributes

SAFETY WARNING — never configure a textual merge for a binary format on the strength of a readable diff. The extracted text is a projection, not the file: merging two projections and writing the result back produces a file the application cannot open, and the corruption is discovered by whoever needs the document next. Diffs may be textual; merges must not be.

# Verification: a conflict leaves both versions on disk and stops the merge
git merge other-branch; ls docs/spec.pdf.*

Step 5 — Distribute it, and keep it optional

Converters are external commands that not everyone will have installed, so the setup must degrade gracefully.

# In the team configuration, guarded by availability
command -v pdftotext >/dev/null && git config --local diff.pdf.textconv 'pdftotext -layout -q'
# A clear message when the converter is missing, rather than a silent fallback
git config --local diff.pdf.textconv \
  'sh -c "command -v pdftotext >/dev/null || { echo \"[install poppler-utils for readable PDF diffs]\"; exit 0; }; pdftotext -layout -q \"\$1\" -" --'
# Verification: a machine without the converter still diffs, with an explanation
PATH=/usr/bin git diff HEAD~1 -- docs/spec.pdf | head -3
Three formats, three levels of what is possibleA document with extractable text diffs well and must still be merged by a person. A spreadsheet converts to rows, which diff usefully but lose formatting. A photograph has no text representation at all and is reviewed by looking at it.PDF, DOCXtext extracts cleanlydiff is usefulmerge: keep bothXLSX, CSV-likerows extractformatting lostmerge: keep bothPhotographsno textmark binaryreview visuallythe third column is not a failure of textconv — it is a format with nothing to extract

Validation checklist

Frequently Asked Questions

Does textconv change what is stored in the repository?

No. It runs at display time on blobs read out of the object database, so the committed bytes are untouched. That is what makes it safe to add and remove freely — nothing about history depends on it.

Can it make binary files reviewable in the web interface?

Not directly: forges do not run your local converters. Some support their own rendering for common formats, and for the rest the practical answer is to commit a generated text representation alongside the binary, marked generated — which trades repository size for reviewability.

Is there a performance cost on large repositories?

The first pass over each blob costs a converter run, and with caching the rest is negligible. The cache is stored in notes, which do replicate if you push that ref — usually you should not, since every machine can regenerate it locally and the notes ref grows quickly.