Measuring review latency from git history Jump to heading#

Every team has an opinion about how long reviews take, and the opinions disagree by a factor of five. That is not because people are unobservant but because the experience is dominated by the worst case, and the worst case is rare. A handful of commands turns the argument into a distribution, and a distribution tells you which stage to fix — which is usually not the one anybody was arguing about. This recipe produces the numbers and the counterweight that stops them being gamed, within code review workflow engineering.

When to use this approach Jump to heading#

• Review speed is being discussed and nobody has figures.
• You are about to change the process and want a baseline.
• You have made changes and need to know whether they worked.
• Someone has proposed a target and you would like to know if it is achievable.
• If your repository has fewer than fifty merged changes, the sample is too small for medians to mean much.

Step 1 — Pull the raw events Jump to heading#

The forge holds the timestamps; Git alone cannot tell you when a review happened.

gh pr list --state merged --limit 200 \
  --json number,createdAt,mergedAt,reviews,additions,deletions,labels \
  > /tmp/prs.json

jq 'length' /tmp/prs.json

# Verification: the sample covers a representative period
jq -r '[.[].createdAt] | sort | "\(.[0][0:10]) to \(.[-1][0:10])"' /tmp/prs.json

Step 2 — Split elapsed time into waiting and working Jump to heading#

The single number “time to merge” hides the distinction that matters.

jq -r '
  map(select(.reviews | length > 0)) |
  map({
    pr: .number,
    wait_h: (((.reviews[0].submittedAt | fromdate) - (.createdAt | fromdate)) / 3600),
    work_h: (((.mergedAt | fromdate) - (.reviews[0].submittedAt | fromdate)) / 3600)
  }) |
  {
    n: length,
    median_wait: (map(.wait_h) | sort | .[length/2|floor] | floor),
    median_work: (map(.work_h) | sort | .[length/2|floor] | floor),
    p90_wait: (map(.wait_h) | sort | .[(length*0.9)|floor] | floor)
  }' /tmp/prs.json

# Verification: the two medians should sum to roughly the median total
jq -r 'map(select(.mergedAt) | ((.mergedAt|fromdate) - (.createdAt|fromdate))/3600)
       | sort | .[length/2|floor] | floor' /tmp/prs.json

Step 3 — Correlate with size, because that is the usual cause Jump to heading#

jq -r '
  map(select(.reviews | length > 0)) |
  map({size: (.additions + .deletions),
       wait: (((.reviews[0].submittedAt | fromdate) - (.createdAt | fromdate)) / 3600)}) |
  group_by(if .size < 100 then "S" elif .size < 400 then "M" else "L" end) |
  map({bucket: (if .[0].size < 100 then "S" elif .[0].size < 400 then "M" else "L" end),
       n: length,
       median_wait_h: (map(.wait) | sort | .[length/2|floor] | floor)})' /tmp/prs.json

# Verification: the relationship should be monotonic; if not, something else dominates

If waiting time rises steeply with size, the fix is structural and belongs in splitting a branch into reviewable pull requests. If it does not, the cause is routing or capacity, and assigning reviewers automatically is the relevant lever.

Step 4 — Track the quality counterweight Jump to heading#

Latency alone is trivially improvable by approving faster, so it must never be published alone.

# Approvals that carried no comment at all
jq -r '
  map(select(.reviews | length > 0)) |
  {total: length,
   silent: (map(select([.reviews[] | select(.body != "")] | length == 0)) | length)} |
  "\(.silent) of \(.total) approved without a single comment"' /tmp/prs.json

# Comment density by size bucket, which is the honest quality proxy
jq -r 'map({size: (.additions + .deletions), comments: (.reviews | length)})
       | group_by(if .size < 100 then "S" elif .size < 400 then "M" else "L" end)
       | map({bucket: (if .[0].size < 100 then "S" elif .[0].size < 400 then "M" else "L" end),
              avg_reviews: ((map(.comments) | add) / length)})' /tmp/prs.json

SAFETY WARNING — publishing review latency as a target without a quality measure beside it reliably produces faster approvals rather than faster reviews, and the change is invisible until a defect reaches production. Publish the pair, and treat a fall in comment density alongside a fall in latency as a regression rather than a success.

Step 5 — Publish one figure, on a schedule Jump to heading#

A number nobody sees changes nothing. One number, weekly, where the team already looks.

#!/usr/bin/env sh
# scripts/review-latency — one line, suitable for a weekly post.
set -eu
gh pr list --state merged --limit 100 --json createdAt,reviews,additions,deletions |
jq -r '
  map(select(.reviews | length > 0)) |
  (map(((.reviews[0].submittedAt | fromdate) - (.createdAt | fromdate)) / 3600) | sort) as $w |
  "median wait \($w[($w|length/2|floor)] | floor)h · " +
  "p90 \($w[($w|length*0.9|floor)] | floor)h · " +
  "silent approvals \(map(select([.reviews[] | select(.body != "")] | length == 0)) | length)%"'

# Verification: the script runs in under a second and prints one line
sh scripts/review-latency

Validation checklist Jump to heading#

• The sample covers at least fifty merged changes and a representative period
• Waiting time and reviewing time are reported separately
• Both the median and a tail percentile are published
• Latency is correlated with change size before choosing a fix
• A quality counterweight is published alongside the latency figure
• The measurement script is committed and runs in seconds
• Only one structural change is made between measurements

Frequently Asked Questions Jump to heading#

Why not use the average? Jump to heading#

Because review latency has a long tail and the average sits somewhere nobody experiences. The median describes the typical case and the ninetieth percentile describes the case people complain about; together they say more than any single statistic.

Should time outside working hours be excluded? Jump to heading#

For comparing teams in different time zones, yes, and the arithmetic gets fiddly. For tracking your own team over time it rarely changes the conclusion, because the working-hours distortion is roughly constant. Start simple and refine only if the number is being disputed.

Does this work without a forge API? Jump to heading#

Partially. Merge timestamps are derivable from Git, so time-to-merge is available from history alone. First-review time is not recorded anywhere in Git, so that half needs the forge — and it is the half that matters most.

Related Jump to heading#

• Code Review Workflow Engineering — the parent topic and the levers these numbers point at.
• Splitting a Branch Into Reviewable Pull Requests — the fix when size and latency correlate.
• Assigning Reviewers Automatically — the fix when they do not.