Choosing a secret scanner for git history Jump to heading#

Every scanner claims high accuracy, and the number that matters is its false-positive rate on your repository rather than on a benchmark. A tool that flags every high-entropy string finds all your credentials and a thousand hashes, test fixtures and base64 blobs; one that only matches known provider formats produces a readable report and misses your internal token scheme entirely. The only way to choose is to run the candidates over the same history and count. This recipe does that, within secret scanning and remediation.

When to use this approach Jump to heading#

• You are introducing scanning and have not picked a tool.
• An existing scanner produces too much noise to be read.
• A leak got through and you want to know whether detection would have caught it.
• You need scanning in several places and want one tool for all of them.
• If your provider’s built-in scanning already covers your credential types, start there and add a tool only for what it misses.

Step 1 — Assemble a realistic test corpus Jump to heading#

Benchmarks do not predict behaviour on your code. Your own history does.

# A known-leak corpus: credentials you have already rotated, from real commits
git log --all --oneline | wc -l

# Plus deliberately planted samples of the formats you care about
mkdir -p /tmp/corpus && cd /tmp/corpus && git init -q
printf 'AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY\n' > a.env
printf 'internal_token=int_live_9f8a7b6c5d4e3f2a1b0c\n' > b.env
printf 'sha256=4f2a1b3c5d6e7f8a9b0c1d2e3f4a5b6c7d8e9f0a1b2c3d4e5f6a7b8c9d0e1f2a\n' > c.txt
git add -A && git commit -qm 'corpus'

# Verification: the corpus contains both true positives and likely false ones
ls /tmp/corpus

The third file matters as much as the first two: a scanner that flags a checksum as a secret will flag thousands of them across a real repository.

Step 2 — Run each candidate over the same history Jump to heading#

gitleaks detect --source . --report-format json --report-path /tmp/gitleaks.json --redact
jq 'length' /tmp/gitleaks.json

trufflehog git file://. --json --only-verified > /tmp/trufflehog.jsonl 2>/dev/null
wc -l < /tmp/trufflehog.jsonl

# The provider's own scanning, where available
gh api repos/:owner/:repo/secret-scanning/alerts --jq 'length' 2>/dev/null || echo "not enabled"

# Verification: all three ran against the same commits
git rev-list --count --all

Step 3 — Count true and false positives honestly Jump to heading#

This is the step that decides the answer, and it takes an hour.

# Extract findings to a reviewable list
jq -r '.[] | "\(.File):\(.StartLine)\t\(.RuleID)"' /tmp/gitleaks.json | sort -u > /tmp/g.txt
wc -l /tmp/g.txt

# Classify a sample by hand: real, fixture, checksum, example, unclear
head -40 /tmp/g.txt | nl

# Verification: compute the rate from the classified sample
# false positives / total, over the sample

Tools that verify a candidate against the issuing service — attempting an authenticated call and reporting only what succeeds — collapse the noise dramatically. The cost is that they make network requests during scanning, which has its own implications in CI.

Step 4 — Decide what runs where Jump to heading#

One tool rarely fits all three layers, and that is fine.

# Pre-commit: fast, local, no network
gitleaks protect --staged --redact

# Pull request: the range, covering forks
      - run: gitleaks detect --log-opts="$(git merge-base origin/main HEAD)..HEAD" --redact

# Scheduled: full history, verification enabled, network permitted
trufflehog git file://. --only-verified --json > /tmp/verified.jsonl

# Verification: each layer runs and reports separately
gitleaks version; trufflehog --version 2>&1 | head -1

SAFETY WARNING — a verifying scanner sends candidate credentials to the services that issued them in order to test validity. That is what makes it accurate and it means your CI runner makes authenticated requests with values from your repository, which may be logged by the provider and may trigger alerts on their side. Understand that before enabling verification, and never enable it on a runner you do not control.

Step 5 — Tune, then make it blocking Jump to heading#

A scanner that blocks before it is tuned gets disabled within a week.

# Start in report-only mode, for a fortnight
gitleaks detect --report-path /tmp/report.json --exit-code 0

# .gitleaks.toml — narrow allowlists, each with a reason
[[rules.allowlist]]
description = "Documented example key from the AWS documentation"
paths = ['''^docs/examples/aws\.md$''']
regexes = ['''AKIAIOSFODNN7EXAMPLE''']

# Then enable blocking, once the report is short enough to read
gitleaks detect --exit-code 1

# Verification: a real credential still fails, a fixture does not
gitleaks detect --source /tmp/corpus --exit-code 1 | tail -3

Validation checklist Jump to heading#

• The corpus came from your own history, not a benchmark
• Candidates ran over the same commits
• False positives were counted by hand on a sample
• The tool choice differs per layer where that makes sense
• Verification’s network behaviour is understood before enabling it
• The scanner ran in report-only mode while being tuned
• Blocking was enabled only once the report was short
• Allowlist entries are narrow and carry reasons

Frequently Asked Questions Jump to heading#

Is the provider’s built-in scanning enough? Jump to heading#

It covers the credential formats its partners have registered, which is a large and growing set, and it has one advantage no tool can match: partner notification, where the provider tells the issuer to revoke a leaked key automatically. It will not know about your internal token formats, which is where an additional tool earns its place.

How often should the full-history scan run? Jump to heading#

Weekly or monthly is enough, because its purpose is to catch what new detection rules now recognise rather than to find today’s mistakes — the pre-commit and pull request layers handle those. Running it unattended on a schedule means the cost is zero and the coverage improves as rules improve.

Does scanning slow down commits noticeably? Jump to heading#

A staged-content scan is milliseconds, because it looks at the diff rather than the repository. A full-history scan is minutes on a large repository, which is why it belongs in a scheduled job and not in a hook — the layering in Step 4 exists precisely to keep the interactive path fast.

Related Jump to heading#

• Secret Scanning & Remediation — the parent topic and the response order.
• Scanning Pull Requests for Secrets in CI — the layer that covers forks.
• Blocking Secrets With a Pre-Push Scan — the local layer, in detail.