Caching Docker layers in CI Jump to heading
On a developer machine, Docker reuses layers from the previous build automatically, so a code change rebuilds in seconds. On an ephemeral CI runner there is no previous build: every job starts with an empty cache, downloads base images, reinstalls every dependency and recompiles everything. A five-second local rebuild becomes a six-minute CI build. BuildKit can export its layer cache to a registry or the CI system’s cache store and import it in the next job, which restores most of the local speed. How much it helps depends on two things you control: the order of instructions in the Dockerfile, and where the cache is stored and keyed. This page covers both, within CI caching and runner performance.
When to use this approach Jump to heading
- Container image builds are a large part of your pipeline time.
- CI runners are ephemeral, so Docker’s local cache is always empty.
- Small code changes trigger full dependency reinstalls in the image build.
- You already cache language dependencies outside Docker, as in caching dependencies keyed on the lockfile.
Step 1 — Order the Dockerfile so code changes invalidate little Jump to heading
Docker invalidates a layer and everything after it when its inputs change. Copy the files that change rarely — dependency manifests — before the files that change constantly, so a code change reuses the dependency layer.
# Before: any source change reinstalls every dependency
COPY . .
RUN npm ci
# After: dependencies install only when the manifests change
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build # Verification: a source-only change reuses the dependency layer locally
docker build -t app . && touch src/index.ts && docker build -t app . 2>&1 | grep -E 'CACHED|npm ci' Step 2 — Export and import the cache in CI Jump to heading
With BuildKit (through docker buildx), choose a cache backend. The CI provider’s cache store is simplest; a registry works across CI systems and runner types.
# GitHub Actions: cache in the Actions cache backend
- uses: docker/setup-buildx-action@v3
- uses: docker/build-push-action@v6
with:
context: .
push: false
tags: app:ci
cache-from: type=gha,scope=app-${{ github.ref_name }}
cache-to: type=gha,scope=app-${{ github.ref_name }},mode=max # Any CI: cache in a registry
docker buildx build \
--cache-from "type=registry,ref=registry.example.com/app:buildcache-main" \
--cache-to "type=registry,ref=registry.example.com/app:buildcache-$BRANCH,mode=max" \
-t app:ci . mode=max exports every intermediate layer, including those of multi-stage builder stages; the default min exports only the final image’s layers, which misses most of the expensive work in a multi-stage build.
Step 3 — Scope caches by branch, with a fallback to main Jump to heading
A single shared cache is overwritten by every branch, so builds alternate between warm and cold. Scope the cache per branch and fall back to main’s cache when a branch has none yet. Writing only to the branch’s own scope also prevents a branch from poisoning the cache main uses.
cache-from: |
type=gha,scope=app-${{ github.ref_name }}
type=gha,scope=app-main
cache-to: type=gha,scope=app-${{ github.ref_name }},mode=max The security side of shared caches is covered in fixing cache poisoning between branches.
Step 4 — Use cache mounts for package managers Jump to heading
Layer caching reuses a whole layer or nothing. When the lockfile changes, the dependency layer rebuilds from scratch and downloads every package again. BuildKit cache mounts keep the package manager’s download cache between builds, so a lockfile change downloads only what is new.
# syntax=docker/dockerfile:1
COPY package.json package-lock.json ./
RUN npm ci COPY pyproject.toml uv.lock ./
RUN uv sync --frozen Cache mounts are not exported with cache-to by default. On ephemeral runners they help most when combined with a persistent builder or self-hosted runners that keep BuildKit state, as described in autoscaling self-hosted runners.
Step 5 — Measure the cache’s effect Jump to heading
Check the build log for cache hits and compare durations before and after. BuildKit prints CACHED for reused steps.
docker buildx build --progress=plain . 2>&1 | grep -cE '^#[0-9]+ CACHED'
gh run list --workflow build.yml --limit 20 --json displayTitle,updatedAt,createdAt \
--jq '.[] | "\(.displayTitle[:40]) \(((.updatedAt|fromdateiso8601) - (.createdAt|fromdateiso8601))/60 | floor) min"' Step 6 — Keep caches from growing without bound Jump to heading
Caches accumulate layers that are never used again. Most CI cache backends evict old entries automatically; registry caches need cleanup. Overwrite one cache tag per branch, and delete branch caches when branches are deleted.
# Delete the registry cache tag of a deleted branch (registry API varies)
crane delete "registry.example.com/app:buildcache-$DELETED_BRANCH" Validation checklist Jump to heading
Frequently Asked Questions Jump to heading
Is the Actions cache big enough for Docker layers? Jump to heading
It has a per-repository size limit with automatic eviction. For large images, a registry cache avoids the limit and is shared across runners and CI systems.
Does caching make builds less reproducible? Jump to heading
Layer caching reuses the output of identical inputs, so results match a clean build as long as steps are deterministic. Steps that fetch “latest” anything — apt-get upgrade, unpinned downloads — break that, cached or not. Pin them.
Should release builds use the cache? Jump to heading
Many teams build releases without importing caches, accepting the extra minutes for a clean, independently reproducible build. That also removes any risk of a poisoned cache entry reaching a release.
Related Jump to heading
- CI Caching & Runner Performance — the parent topic.
- Skipping Husky in CI and Production Installs — a common Docker build failure.
- Signing Commits Inside Ephemeral Containers — keeping secrets out of layers.
- Measuring Pipeline Duration Trends — tracking whether the gains last.