Skip to content

fix(pr-merge-tracker): review complete diffs and compute honest review depth - #4

Open
sunwrobert wants to merge 1 commit into
mainfrom
chore/pr-merge-tracker-full-diffs
Open

sunwrobert wants to merge 1 commit into
mainfrom
chore/pr-merge-tracker-full-diffs

Conversation

@sunwrobert

Copy link
Copy Markdown
Owner

Summary

The skill used to review gh pr diff output. Large PRs got cut to a sample, and the saved review depth still said full. In one run, 182 of 266 reviews came from incomplete patches, but the state labeled them full-net-diff.

  • Adds skills/pr-merge-tracker/scripts/collect_diffs.py. It diffs the merge base against the head in a blob-filtered bare git cache, so there are no API file or line caps. It writes the full patch, a file inventory, and 80,000-character chunks, with hunks split and never cut off.
  • Sorts files into required (code, config, tests, docs, export changes), mechanical (import-path-only) and generated chunks.
  • With a saved reviewed head, it reviews only files whose own +/- lines changed, so changes merged in from main drop out.
  • depth computes full or sampled N% from the chunk IDs a reviewer actually read. SKILL.md now forbids writing the depth by hand and requires sampled N% in report bullets.

Testing

Ran against OpenRouterTeam/openrouter-web:

  • #55990 (4 files): 1 chunk, full.
  • #56021 (893 files): 3 required chunks plus 661 import-only files; reading 1 of 3 chunks reports sampled 36%.
  • #56069 (17,376 files): reading 2 chunks reports sampled 1%.
  • Delta mode on #56021 cut the review from 893 files to 54.

bun run check reports a format issue in executor/executor.jsonc, which was already on main; the changed files pass oxfmt --check.

…w depth

Adds scripts/collect_diffs.py, which diffs merge base and head in a blob-filtered bare cache so reviews are not capped at 300 files or 20,000 lines, chunks the full patch, separates required from mechanical and generated files, and computes reviewDepth from the chunks a reviewer actually read.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant