feat(ci): add AI Contribution Report workflow#61
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA |
Adds a GitHub Actions workflow that automatically measures AI tool involvement across the repository's full commit history on every merge to main. ## What it does - Triggers on every push to `main` - Scans all commit messages for `Co-Authored-By:` trailers stamped by AI coding tools (Cursor, Claude Code, GitHub Copilot) - Counts commits attributed to each tool vs. purely human-authored - Renders a markdown summary table directly in the GitHub Actions job summary (visible in the Actions tab after each merge) ## Signal measured The `Co-Authored-By:` trailer is written automatically by: - **Cursor** → `Co-authored-by: Cursor <cursoragent@cursor.com>` - **Claude Code** → `Co-Authored-By: Claude <noreply@anthropic.com>` - **GitHub Copilot** → `Co-authored-by: GitHub Copilot <...>` Commits without any such trailer are counted as human-only. Note that commits may still have had AI assistance without a trailer (e.g. code pasted from a chat interface), so this is a lower-bound estimate. Co-Authored-By: Claude <noreply@anthropic.com>
94379f8 to
21fb473
Compare
fercor-cisco
left a comment
There was a problem hiding this comment.
🤖 This review was generated by the Astra agent (claude-opus-4-8). It may contain mistakes.
Verdict: approve — Informational CI reporting workflow; logic is sound for the common case, with a few correctness/hardening improvements worth addressing but none merge-blocking.
Follow-ups
Suggested follow-up work that could be tracked as Shortcut stories:
.github/workflows/ai-contribution.yaml:8-11: Consider adding an explicit least-privilegepermissions:block (e.g.permissions: contents: read) to the job. The step only writes to$GITHUB_STEP_SUMMARYand never uses the token, so restricting the defaultGITHUB_TOKENscope is good hardening across CI workflows in this repo..github/workflows/ai-contribution.yaml:3-6: The job re-scans the entire commit history on every push to main and only renders an ephemeral job summary (no artifact/badge/trend). Consider aconcurrencygroup to cancel superseded runs, and optionally persisting the numbers somewhere durable if trend tracking is desired.
| cursor=$(git log --format="%B" | grep -i "co-author.*cursor" | wc -l | tr -d ' ') | ||
| claude=$(git log --format="%B" | grep -i "co-authored.*claude" | wc -l | tr -d ' ') | ||
| copilot=$(git log --format="%B" | grep -i "co-author.*copilot" | wc -l | tr -d ' ') | ||
| ai=$((cursor + claude + copilot)) | ||
| human=$((total - ai)) |
There was a problem hiding this comment.
🟡 minor (bug): grep ... | wc -l counts matching lines, whereas total counts commits. Two problems follow:
- A single commit that carries more than one AI trailer (e.g. both
Co-Authored-By: ClaudeandCo-authored-by: Cursor, which happens on hand-offs between tools) is counted once intotalbut once in each ofcursor/claude/copilot. That inflatesaiabove the true count of AI-attributed commits and makeshuman=$((total - ai))potentially negative, which then renders as a negative count and a negative percentage. grep -i "co-author.*cursor"matches on the whole commit body, so a prose mention (e.g. "discussed co-author roles with the Cursor team") is counted as a trailer.
To count commits rather than trailer lines, and to anchor on the trailer, consider matching per-commit. For example, count commits whose body contains the trailer:
cursor=$(git log --format="%H %(trailers:key=Co-authored-by,valueonly)" | grep -ic "cursor")or iterate commits and grep each body once, so a commit is attributed at most once per tool.
🤖 Generated by Astra
| echo "| Cursor | $cursor | $(( cursor * 100 / total ))% |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| Claude | $claude | $(( claude * 100 / total ))% |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| Copilot | $copilot | $(( copilot * 100 / total ))% |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| **AI total** | **$ai** | **$(( ai * 100 / total ))%** |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| Human | $human | $(( human * 100 / total ))% |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| **Total** | **$total** | 100% |" >> "$GITHUB_STEP_SUMMARY" |
There was a problem hiding this comment.
🟡 minor (bug): Every Share cell divides by $total. On a repository with zero commits reachable from HEAD, total is 0 and $(( cursor * 100 / total )) aborts the step with a divide-by-zero. Unlikely on main today, but cheap to guard. Consider computing shares only when total > 0, e.g.:
if [ "$total" -eq 0 ]; then total=1; fi(or skip the percentage column when there are no commits).
🤖 Generated by Astra
| echo "| Cursor | $cursor | $(( cursor * 100 / total ))% |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| Claude | $claude | $(( claude * 100 / total ))% |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| Copilot | $copilot | $(( copilot * 100 / total ))% |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| **AI total** | **$ai** | **$(( ai * 100 / total ))%** |" >> "$GITHUB_STEP_SUMMARY" | ||
| echo "| Human | $human | $(( human * 100 / total ))% |" >> "$GITHUB_STEP_SUMMARY" |
There was a problem hiding this comment.
🔵 nit (other): Integer division truncates each share, so the per-row percentages will not sum to 100% (e.g. three rows at 33% each show 99%). If exactness matters for the summary, compute the human share as 100 - ai_share rather than independently, or round. Non-blocking.
🤖 Generated by Astra
Adds a GitHub Actions workflow that automatically measures AI tool involvement across the repository's full commit history on every merge to main.
What it does
mainCo-Authored-By:trailers stamped by AI coding tools (Cursor, Claude Code, GitHub Copilot)Signal measured
The
Co-Authored-By:trailer is written automatically by:Co-authored-by: Cursor <cursoragent@cursor.com>Co-Authored-By: Claude <noreply@anthropic.com>Co-authored-by: GitHub Copilot <...>Commits without any such trailer are counted as human-only. Note that commits may still have had AI assistance without a trailer (e.g. code pasted from a chat interface), so this is a lower-bound estimate.