Build Provenance & Attestations Jump to heading

A signed commit proves who wrote a change. It says nothing about the container image running in production, and the gap between the two is where most supply-chain risk actually lives: an artefact built from an unreviewed branch, pushed by a credential that should not have had access, or assembled on a machine nobody can identify. Provenance closes that gap by recording — verifiably — which commit, which builder and which inputs produced a given artefact. This part of Commit Signing & Git Supply-Chain Security covers generating it, verifying it, and the places the chain still breaks.

Prerequisites Jump to heading

What Provenance Actually Records Jump to heading

An attestation is a signed statement about an artefact. The useful ones say: this digest was produced by this builder, from this source repository at this commit, using this entry point, at this time. Everything else is detail; those five facts are what make a verification gate possible.

The chain from a commit to a running artefactA signed commit anchors the source. The build produces an artefact and an attestation naming the commit, the builder and the digest. The deployment gate verifies the attestation before admitting the artefact, so nothing runs that cannot be traced back to reviewed source.Signed commitwho wrote itBuildidentified builderrecorded inputsAttestationdigest + commitsignedVerified deploygate refuses the resteach link is checkable; a missing link makes every link after it unverifiable

Step 1 — Anchor the Source End Jump to heading

Provenance that names a commit is only as good as the commit’s own integrity.

# A signed, annotated tag is the anchor for a release
git tag -s v2.8.0 -m 'Release 2.8.0'
git push origin v2.8.0
# Verification: the tag verifies against a key you trust
git tag -v v2.8.0
git rev-parse v2.8.0^{commit}
# And the commits it covers are themselves signed
git log --format='%H %G?' v2.7.0..v2.8.0 | awk '$2 != "G" {print "unsigned: " $1}' | head

The tag signing mechanics are in signing and verifying release tags; the commit-level enforcement is in commit verification gates.

Step 2 — Generate the Attestation During the Build Jump to heading

The attestation has to be produced by the build itself, because that is the only moment when the relationship between source and artefact is directly observable.

# .github/workflows/release.yml
permissions:
  contents: read
  id-token: write          # for keyless signing
  attestations: write
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - id: build
        run: |
          digest=$(docker build -q . | tee /dev/stderr)
          echo "digest=$digest" >> "$GITHUB_OUTPUT"
      - uses: actions/attest-build-provenance@v1
        with:
          subject-name: registry.example.com/app
          subject-digest: ${{ steps.build.outputs.digest }}
          push-to-registry: true
# Verification: the attestation exists and names the expected commit
gh attestation verify oci://registry.example.com/app@"$digest" --owner acme
cosign download attestation registry.example.com/app@"$digest" | jq -r '.payload' \
  | base64 -d | jq -r '.predicate.buildDefinition.resolvedDependencies[0].digest.gitCommit'
The five facts a useful attestation carriesThe digest identifies the artefact exactly. The source repository and commit identify what it was built from. The builder identity says which system produced it, and the entry point says how. Together they make a verification gate possible.Digestthis exact artefactSource + commitwhat it came fromBuilderwhich systemEntry pointwhich workflowa gate that checks the digest alone verifies integrity, not provenance

Step 3 — Verify Before Deploying, Not After Jump to heading

An attestation nothing checks is metadata. The gate is the point.

# Refuse anything without a verifiable attestation naming your repository
gh attestation verify oci://registry.example.com/app@"$digest" \
  --owner acme --repo acme/app \
  || { echo "refusing to deploy: provenance not verified" >&2; exit 1; }
# With cosign, against a specific identity and issuer
cosign verify-attestation \
  --certificate-identity-regexp '^https://github.com/acme/app/\.github/workflows/release\.yml@' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  registry.example.com/app@"$digest"
# Verification: an artefact built elsewhere must fail the gate
cosign verify-attestation --certificate-identity-regexp '^https://github.com/acme/' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  registry.example.com/someone-elses:latest 2>&1 | tail -2

SAFETY WARNING — verifying that an attestation exists is not verification. The check must pin the identity that signed it and the repository it names; without those constraints, an attestation produced by any workflow in any repository will satisfy the gate, which is precisely the situation an attacker with access to one unimportant repository would engineer. Pin the issuer, the identity pattern and the source repository explicitly.

Step 4 — Make the Gate Fail Closed Jump to heading

A verification step that can be skipped when it errors is a verification step that will be.

set -euo pipefail          # a failed verify must stop the deployment
gh attestation verify oci://"$IMAGE" --owner acme --repo acme/app
kubectl set image deploy/app app="$IMAGE" -n production
# Or in an admission controller, where the cluster refuses rather than the pipeline
# (policy shape; the specific controller varies)
#   verify every image against the expected identity and repository
#   deny on missing or unverifiable attestation
# Verification: a deliberately unsigned image is refused
kubectl run probe --image=nginx:latest --dry-run=server -n production 2>&1 | tail -2

Moving the gate into the cluster rather than the pipeline is the stronger form: a pipeline gate protects the deployment path you control, while an admission gate protects every path including the one somebody uses at three in the morning.

Provenance is most useful during an incident, and only if it is queryable quickly.

# From a running image back to the commit, in one command
digest=$(kubectl get deploy app -n production -o jsonpath='{.spec.template.spec.containers[0].image}')
gh attestation verify oci://"$digest" --owner acme --format json \
  | jq -r '.[0].verificationResult.statement.predicate.buildDefinition
           .resolvedDependencies[0].digest.gitCommit'
# And from the commit back to what it contains
git log -1 --format='%H %an %ad %s' --date=short "$commit"
git tag --points-at "$commit"
# Verification: the round trip resolves in under a minute
Answering 'what is running in production?' during an incidentThe deployment names a digest. The attestation for that digest names a commit and a builder. The commit resolves to a tag, an author and a diff. Without provenance, each of those steps is a conversation instead of a command.responderclusterregistryrepositorywhat digest is running?sha256:4f2a…attestation for this digestcommit a1b2, built by release.ymlwhat is in a1b2?tag v2.8.0, 14 commitsthree commands, no guesswork — which is the entire operational case for provenance

Where the Chain Actually Breaks Jump to heading

Provenance is often introduced as a compliance exercise and its real value is narrower and more useful: it makes a specific set of questions answerable. Knowing where it does not help is what stops it being oversold.

The first break is the source itself. An attestation naming commit a1b2 proves the build used that commit, not that the commit was reviewed, signed or on a protected branch. A pipeline that builds from any branch on any push will happily attest to an artefact built from unreviewed code, and the attestation will verify perfectly. Constraining which refs can produce a release build matters as much as the attestation does.

The second is the builder. Keyless signing binds the attestation to a workflow identity, which is strong — but a workflow that can be triggered by anyone, or that checks out untrusted code before building, produces a genuine attestation for an artefact an attacker influenced. The permissions discipline in limiting workflow permissions per job is part of the provenance story, not separate from it.

The third is the dependencies. Provenance records what the build consumed only to the extent that the build declares it. A build that fetches a script from a URL at runtime has an attestation that is accurate and incomplete, and the incompleteness is invisible. Pinning what the build consumes — actions by digest, dependencies by lockfile, base images by digest — is what makes the recorded inputs meaningful.

The fourth, and the one most often left unclosed, is the gap between verification and use. An artefact verified at deploy time and then mutated, re-tagged or replaced by a mutable tag reference has been verified and is no longer the thing that runs. Deploying by digest rather than by tag closes it, and is the single cheapest improvement available in most pipelines.

Configuration Reference Jump to heading

MechanismRecords or checksWhen to use
Signed annotated tagWho approved this release pointEvery release
Build provenance attestationDigest, source commit, builder, entry pointEvery artefact that is deployed
SBOM attestationWhat the artefact containsWhere dependency inventory is required
Keyless signingBinds signature to a workflow identityPreferred over long-lived keys
--certificate-identity-regexpWhich workflow may have signedAlways, in verification
Admission-time verificationRefuses unverified images cluster-sideStronger than a pipeline gate
Deploy by digestThe verified artefact is the one that runsAlways

Troubleshooting Jump to heading

SymptomLikely causeFix
Verification passes for any imageIdentity and repository not pinnedAdd identity regexp and issuer
Attestation missing after a buildPermissions lack attestations: writeGrant it on that job only
Gate skipped when verification errorsScript does not fail closedset -euo pipefail, check exit codes
Verified image differs from deployed oneDeployed by mutable tagDeploy by digest
Attestation names an unexpected commitBuild ran on a merge commit, not the tipRecord which ref the build used
Cannot trace a running image to a commitNo attestation, or not pushed to the registryPush attestations alongside artefacts

Frequently Asked Questions Jump to heading

Is this worth it for an internal service? Jump to heading

The compliance case may not apply; the operational one usually does. Being able to answer “which commit is running in production” with a command rather than an investigation pays for itself the first time it is asked during an incident, and that is the same mechanism regardless of who the software is for.

Does keyless signing mean there is no key to protect? Jump to heading

It means there is no long-lived key to protect, which removes an entire class of risk. The signing key is ephemeral and bound to a workflow identity by an identity provider, so what you are protecting instead is the workflow’s ability to run and what it is allowed to build from — which is a more tractable thing to reason about than key custody.

What is the relationship between provenance and an SBOM? Jump to heading

Provenance says where an artefact came from; an SBOM says what is inside it. They answer different questions and are usually published together as separate attestations against the same digest. A vulnerability report needs the SBOM; an incident asking “was this built from reviewed code” needs the provenance.

Can provenance be faked? Jump to heading

Not the signature, if verification pins the identity and issuer. What can be influenced is what the build does — which is why the section above treats the builder and its inputs as part of the chain rather than as background. An attestation is a truthful record of a build; whether that build deserved to be trusted is a separate question you answer with permissions and ref constraints.

Where should provenance verification live — the pipeline or the cluster? Jump to heading

Both, if you can, and the cluster if you have to choose. A pipeline gate is simpler and covers the deployments that go through the pipeline, which is most of them on an ordinary day. The deployment that matters is the one made during an incident, by hand, at an hour when nobody is checking — and that one bypasses the pipeline entirely. An admission controller sees every path into the cluster, which is what turns the control from a convention into an enforced property.

The practical route is to start in the pipeline, because it is an afternoon’s work and immediately useful, and to treat the admission gate as the second phase. Running both is not redundant: the pipeline gate gives a clear, early failure that names the problem, while the admission gate gives the guarantee. A failure at the admission gate alone is harder to diagnose, which is why keeping the earlier check is worth the small duplication.