Reproducing a release build from a tag

An attestation says a build produced an artefact. Reproducing that build independently and getting the identical artefact proves it, without trusting the builder, the signature or the transparency log. Full bit-for-bit reproducibility is genuinely hard, and the useful news is that most of the distance is covered by a handful of pins — and that even partial reproducibility catches the failure modes that matter. This recipe gets as far as your toolchain allows, within build provenance and attestations.

When to use this approach

  • You want a check on the build system rather than a check by it.
  • A customer or auditor asks whether releases are reproducible.
  • An artefact behaves unexpectedly and you suspect the build rather than the source.
  • You are hardening a release pipeline and want a measurable property.
  • If your builds embed a timestamp you cannot control, expect partial results — which are still worth having.

Step 1 — Pin every input the build consumes

Reproducibility is mostly a pinning exercise.

# Base image by digest, never by tag
FROM node:20.11.1-bookworm-slim@sha256:4f2a1b3c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a

# Dependencies from the lockfile, with no lifecycle scripts
COPY package.json package-lock.json ./
RUN npm ci --ignore-scripts
# Actions by commit id, not by tag
      - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4.1.1
# Verification: nothing in the build references a mutable tag
grep -nE 'FROM [^@]+$|uses: [^@]+@v' Dockerfile .github/workflows/release.yml || echo "all inputs pinned"
Where nondeterminism actually comes fromEmbedded timestamps and build paths account for most differences between two builds of the same source. File ordering in archives and locale-dependent sorting account for most of the rest, and all four have standard fixes.share of reproducibility failures, by causeembedded timestamps44%absolute build paths23%archive file ordering18%locale-dependent sorting9%genuinely different inputs6%the bottom bar is the one you actually want to detect — the rest is noise to eliminate

Step 2 — Remove the common sources of nondeterminism 

# A fixed timestamp derived from the commit, not from the clock
export SOURCE_DATE_EPOCH="$(git log -1 --format=%ct "$TAG")"
# Deterministic archives: fixed ordering, fixed timestamps, no owner
tar --sort=name --mtime="@${SOURCE_DATE_EPOCH}" \
    --owner=0 --group=0 --numeric-owner \
    -cf dist.tar dist/
# A fixed locale, so sorting is stable
export LC_ALL=C TZ=UTC
# Verification: two local builds produce the same archive hash
sh build.sh && sha256sum dist.tar
sh build.sh && sha256sum dist.tar

SOURCE_DATE_EPOCH is honoured by a surprising amount of tooling, and setting it from the commit date rather than the build time is what makes a rebuild months later produce the same bytes.

Step 3 — Rebuild from the tag, in a clean environment 

git clone --branch v2.8.0 --depth 1 https://github.com/acme/app /tmp/rebuild
cd /tmp/rebuild && git verify-tag v2.8.0
export SOURCE_DATE_EPOCH="$(git log -1 --format=%ct v2.8.0)" LC_ALL=C TZ=UTC
docker build --no-cache -t rebuild:local .
# Compare against the published artefact
crane digest ghcr.io/acme/app:v2.8.0
docker inspect --format='{{.Id}}' rebuild:local
# Verification: the image config, which is what differs most often
crane config ghcr.io/acme/app:v2.8.0 | jq -S . > /tmp/published.json
docker inspect rebuild:local | jq -S '.[0].Config' > /tmp/local.json
diff /tmp/published.json /tmp/local.json | head -20
The rebuild comparison, and what each level provesIdentical filesystem contents prove the source produced the same files. Identical layers prove the build steps matched. An identical manifest digest proves everything including metadata. Each level is a weaker claim than the one after it and still a useful one.File contentssame outputsLayer digestssame build stepsConfigsame metadataManifest digestbit-for-bitmost projects reach the second box easily and the fourth with real effort

Step 4 — Compare at the level you can actually reach

Full digest equality is the goal; partial comparisons are useful long before you get there.

# Level 1: the filesystem contents, ignoring metadata
crane export ghcr.io/acme/app@"$DIGEST" - | tar -tvf - | awk '{print $NF}' | sort > /tmp/a.txt
docker save rebuild:local | tar -xO --wildcards '*/layer.tar' | tar -tvf - | awk '{print $NF}' | sort > /tmp/b.txt
diff /tmp/a.txt /tmp/b.txt | head
# Level 2: the hash of each file, which catches content differences
crane export ghcr.io/acme/app@"$DIGEST" - | tar -xO | sha256sum
# Verification: record which level you reached, honestly
echo "v2.8.0: file contents identical; layer digests differ (timestamps)" >> reproducibility.log

SAFETY WARNING — a rebuild that does not match is not automatically a security finding, and treating it as one wastes the mechanism. Timestamps, build paths and tool versions produce differences constantly. What matters is whether the file contents differ: identical contents with different metadata is a reproducibility gap, while different contents from the same source is the thing worth investigating immediately.

Step 5 — Automate it, and record the trend

A reproducibility check that runs once is a curiosity; run per release it is a property.

# .github/workflows/verify-reproducible.yml
name: verify-reproducible
on:
  schedule: [{ cron: '0 3 * * 1' }]
jobs:
  rebuild:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { ref: ${{ github.event.inputs.tag || 'v2.8.0' }}, fetch-depth: 0 }
      - run: |
          export SOURCE_DATE_EPOCH="$(git log -1 --format=%ct)" LC_ALL=C TZ=UTC
          docker build --no-cache -t rebuild:ci .
          crane export "ghcr.io/acme/app:${GITHUB_REF_NAME}" - | tar -xO | sha256sum > /tmp/published.sha
          docker save rebuild:ci | tar -xO --wildcards '*/layer.tar' | tar -xO | sha256sum > /tmp/rebuilt.sha
          diff /tmp/published.sha /tmp/rebuilt.sha
# Verification: the job's result is recorded per release
tail -5 reproducibility.log
Trusting an attestation against reproducing the buildAn attestation is a signed claim by the builder, which requires trusting the builder and its identity provider. A reproduction is a check anyone can perform, requiring trust in nothing but the source. They are complementary, and the second is a far stronger statement.Attestation onlyPlus reproductionrequires trusting the builderyesnoverifiable by a third partywith the identityby anyonecost per releasezeroone rebuilddetects a compromised buildernoyesthe last row is why reproducibility is worth pursuing even when it is only partial

Validation checklist

Frequently Asked Questions

Is full reproducibility realistic?

For some stacks yes, for others not yet, and partial reproducibility is worth having regardless. Reaching “identical file contents, differing metadata” catches a compromised builder, a substituted dependency and an unexpected input — which is most of what the exercise is for. Chasing the last few bytes has sharply diminishing returns.

What if the rebuild differs and we cannot explain it?

Narrow it: compare file lists, then file hashes, then the differing file’s content. Almost every unexplained difference resolves to a timestamp, a path or a tool version once you look at the actual bytes. If the difference is in compiled output from identical source, that is the case worth escalating.

Does this replace signing?

No — they answer different questions. Signing says who produced an artefact; reproduction says the artefact corresponds to the source. An attacker who compromises the builder can sign perfectly and cannot make a rebuild match, which is exactly why having both is stronger than having either.