Choosing required status checks that actually gate

Required-check lists grow by accretion. Someone adds a job, someone marks it required so it cannot be ignored, and two years later there are fourteen required checks, three of which fail regularly for reasons unrelated to the change. At that point the gate has inverted: a red result no longer means “do not merge”, it means “re-run it”. This page is about getting back to a list short enough to be believed, within merge queues and required checks.

When to use this approach

  • Re-running a failed check before reading it has become a reflex on your team.
  • The required list has more than about five entries.
  • Some required checks fail on changes that cannot possibly have affected them.
  • You are introducing a merge queue, which makes every required check a throughput cost.
  • If your pipeline is one job and it is trustworthy, there is nothing to classify here — spend the effort on preventing broken builds with pre-push hooks instead.

Step 1 — Measure how each check behaves before judging it

Opinions about which checks are noisy are unreliable. Pull the numbers first: how often each job fails, and how often a failure was followed by a code change rather than a re-run.

# Failure count per workflow over the last 200 runs
gh run list --limit 200 --json name,conclusion \
  --jq 'group_by(.name)[] | {check: .[0].name,
        runs: length,
        failed: (map(select(.conclusion=="failure")) | length)}'
# Re-runs are the tell: a job re-run more than once per failure is not a gate
gh run list --limit 200 --json name,conclusion,attempt \
  --jq '[.[] | select(.attempt > 1)] | group_by(.name)[] | {check: .[0].name, reruns: length}'

What this shows: a job with a 20% failure rate and a re-run rate to match is reporting something other than “this change is broken”, and requiring it costs throughput without buying safety.

Failure and re-run rates across a real required listFour checks measured over two hundred runs. The unit tests fail rarely and are almost never re-run, so a red result is trusted. The end-to-end suite fails often and is re-run almost as often, which is the signature of a check that has stopped gating anything.failures per hundred runsunit tests3%type check4%licence scan11%end-to-end suite27%a check above roughly ten per cent needs a reason before it is allowed to block

Step 2 — Sort every check into one of three buckets

With the numbers in hand, put each job in exactly one bucket. The buckets are deliberately crude, because a finer taxonomy invites everything to be classified as important.

The three buckets, and what each one commits you toBlocking checks are reproducible, fast and always acted on. Reporting checks post their result without holding the merge. Deleted checks are the ones that nobody has read in months, and keeping them running costs machine time for no signal.Blocks the mergedeterministicacted on in minutesunder five entriesReports onlycoverage, size, advisoriesposted as a commentno merge impactDelete itunread for monthsfailures ignoredmachine time for nothingevery job goes in exactly one bucket — 'important but not blocking' is the reporting bucket

The test for the first bucket is behavioural, not architectural: when this check went red last month, did someone change the code, or did someone press the button again? Only the first answer belongs in a gate.

Step 3 — Make the reporting checks visible without blocking

Demoting a check must not mean hiding it. The result still belongs on the pull request, and a summary comment is the right medium — it is readable, it can be updated in place, and it does not hold the merge.

# .github/workflows/report.yml — runs, reports, never blocks.
name: report
on: pull_request
permissions:
  contents: read
  pull-requests: write
jobs:
  bundle-size:
    runs-on: ubuntu-latest
    continue-on-error: true          # a regression here informs, it does not gate
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm run size > size.txt
      - run: gh pr comment "$PR" --edit-last --body-file size.txt || gh pr comment "$PR" --body-file size.txt
        env:
          GH_TOKEN: ${{ github.token }}
          PR: ${{ github.event.number }}
# Verification: the check appears on the PR but is absent from the required list
gh api repos/:owner/:repo/rulesets --jq '.[].rules[]?
  | select(.type=="required_status_checks")
  | .parameters.required_status_checks[].context'

The --edit-last pattern matters more than it looks: without it, every push adds another comment, and by the fifth one people stop reading the thread — the failure mode covered in posting CI results as PR comments without spam.

Step 4 — Handle checks that are skipped by path filters

A required check that does not run because a path filter excluded it will block the merge forever — the gate is waiting for a result that was never going to be produced. The fix is a fallback job that reports success under the same name.

jobs:
  backend-tests:
    if: ${{ needs.changes.outputs.backend == 'true' }}
    runs-on: ubuntu-latest
    steps:
      - run: make test-backend
  backend-tests-skipped:               # same reported name, trivially green
    if: ${{ needs.changes.outputs.backend != 'true' }}
    runs-on: ubuntu-latest
    steps:
      - run: echo "no backend changes in this pull request"
# Verification: open a docs-only PR and confirm the name still reports
gh pr checks --json name,state --jq '.[] | select(.name | test("backend"))'
Why a skipped required check blocks foreverA documentation-only pull request matches no backend paths, so the backend job never starts. The gate is waiting for a check run that will never be reported, and the pull request cannot merge until a fallback job reports success under the same name.pull requestpath filterbackend jobmerge gatedocs-only changeno match — skipwaiting for backend-testsfallback job runsreported greenthe gate does not distinguish 'skipped' from 'not finished yet'

What changed: the required name always reports, whether or not the expensive job ran. The path-filter design behind this is covered in optimizing CI triggers for path-specific changes.

Step 5 — Write the list down and review it on a schedule

Required checks accumulate precisely because nothing prompts anyone to remove one. Put the list in the repository next to the workflows, with a sentence of justification per entry, and revisit it when the pipeline changes.

# A drift check worth running in CI: the documented list versus the live one
gh api repos/:owner/:repo/rulesets --jq '.[].rules[]?
  | select(.type=="required_status_checks")
  | .parameters.required_status_checks[].context' | sort > /tmp/live.txt
sort docs/required-checks.txt | diff - /tmp/live.txt && echo "required checks match the documented list"

SAFETY WARNING — do not demote a security or licence gate to reporting because it is noisy. Noise there means the scanner needs tuning or an allowlist, not that the risk went away. Fix the signal, keep the gate, and document the allowlist as described in allowlisting test fixtures without blinding the scanner.

Validation checklist

Frequently Asked Questions

Is it not safer to require everything?

It feels safer and measurably is not. A long required list trains people to treat red as a transient state, so the one genuine failure a quarter is re-run alongside the noise and merged when it happens to pass. Safety comes from a short list where red is believed, plus reporting for everything else.

What about checks that are slow but genuinely important?

Move them off the merge path rather than out of the pipeline. Nightly runs, post-merge verification and release gates all catch the same problem without charging every pull request for it. If it must block, budget the time honestly and cut the pipeline’s fixed costs elsewhere.

How do we stop the list growing again?

Make adding a required check require the same justification as removing one — a line in the documented list saying what a failure means and who acts on it. The drift check in Step 5 then makes an undocumented addition fail CI, which is a far more reliable reviewer than good intentions.