feat: vouched-ci — approve fork CI runs for verified commits from vouched users

[claude]

Requested by Samuel Attard · Slack thread

Before: Every GitHub Actions run on a pull request from a fork by an external contributor sits at "Awaiting approval" until a maintainer with write access clicks "Approve and run", even when the person who pushed is a trusted maintainer who works out of a fork. Sheriff had no notion of "trusted for CI".

After: The permissions config gains an optional org-level vouched_ci list of users pinned by numeric GitHub id (- { login, id }). When a fork pull request run created by one of them is gated on approval, Sheriff approves that run, provided every commit in the pull request was authored, committed and verified-signed by that same user. Anything else is left alone with the reason logged, and every approval is posted to Slack.

How: GitHub emits workflow_run requested for a fork pull request run the moment it is created and gated on "Approve and run" (the payload reports it as status: completed, conclusion: action_required; the org webhook already delivers workflow_run). A new workflow_run.requested handler in src/index.ts runs its checks cheapest-first, since this event fires for every run in the org: isApprovableRun (src/vouched-ci.ts) rejects anything that is not a pull_request run, not action_required, not from a fork, or has no head repository/owner/branch, all from the payload alone; then the org must have a vouched_ci list and the run's triggering_actor (the authenticated user whose push created the run) must be vouched by both id and login, as a pre-filter. Because workflow_run.pull_requests is always empty for fork runs, the handler then lists the open pull requests for the run's owner:branch head (GET /pulls?head=…) and matchPullRequestForRun keeps the one whose head.sha and head.repo.id equal the run's head_sha and head_repository.id, refusing on zero or several matches. It lists that PR's commits (GET /pulls/{n}/commits, paginated), re-fetches the PR to detect a racing push and cross-checks its head repository against the run, and runs the pure decision function evaluateVouchedCI with the triggering actor as the sender and the run's head_sha as the tree to verify. Only once that passes does it POST /actions/runs/{run_id}/approve for that single run (in IS_DRY_RUN it logs "would approve run" instead); a 4xx from GitHub (run already approved by a maintainer or no longer pending) is logged as a no-op, anything else propagates. The API calls use a dedicated installation token narrowed to actions: write + pull_requests: read (getVouchedCIOctokit, actions: read in dry run) so a missing App permission only breaks this feature. The config key is validated with Joi in validateConfigFast (positive integer id, non-empty login, no duplicates) so the .permissions dry-run check catches mistakes. The decision, run-filter and PR-matching logic is covered by 36 node:test cases (yarn test, wired into the Test workflow). There is no polling: each workflow file gets its own run and its own event, so every run is verified and approved individually.

Safety / threat model

Analysed against GitHub's documented behaviour (webhook triggering_actor semantics, the REST verification object and its reason table, signature persistence across the fork network, the fork-PR approval policy, pull_request_target). The premise as literally stated, "approve based on commit author", is not safe: git author/committer metadata is free-form text, GitHub's email-to-account attribution of author is spoofable by writing someone's public email into a commit, and a verified signature by user X only proves X made that commit object, not that X pushed it, nor anything about the rest of the tree the run would execute (GitHub reuses verification records across the whole repository network, so X's signed commits can be replayed from any fork). What is safe is the stricter subset implemented here: the pusher is vouched, and every commit in the PR range is that pusher's own verified-signed commit.

Trust anchors used: workflow_run.triggering_actor.id (server-side attribution of the authenticated push that created the run, the same actor GitHub's own approval policy checks), commit.committer.id bound to a key on that account by verification.verified === true && reason === 'valid', commit.author.id === committer.id (the vigilant-mode condition, enforced ourselves since REST does not expose it), and the run's head_sha. Web-UI commits are signed by web-flow, not the user, so they are refused; so are unsigned commits and commits the vouched user committed but did not author. Merge commits are allowed only because every commit in the PR range is checked, so merging the PR base adds just the (signed) merge commit while merging anything else drags in foreign commits and is refused.

Refusal conditions (any one refuses the run, nothing is approved, reason is logged):

1. run.event != 'pull_request' (so pull_request_target etc. are never touched), or the run is not action_required.
2. The run's head repository is the base repo (not a fork), is missing, or has no owner/branch to resolve a pull request from.
3. The org has no vouched_ci list.
4. triggering_actor.id is not a vouched id, or the pinned login does not match triggering_actor.login (case-insensitive) — an id typo can never vouch an unrelated account; a rename fails closed until the config is updated. The ghost user fails here too.
5. Zero or more than one open pull request has head.sha == run.head_sha and head.repo.id == run.head_repository.id (the run cannot be attributed to exactly one PR).
6. PR head SHA changed between the event and verification, the PR's head repository differs from the run's, or the commit listing does not contain the head SHA.
7. PR has more than 250 commits (the endpoint's cap), the listing length differs from the PR's commits count, or there are zero commits.
8. Any commit: author/committer unresolved, author.id != triggering_actor.id, committer.id != triggering_actor.id, verification.verified != true, or verification.reason != 'valid'.

Approvals are per run and pinned to the verified head SHA (the run's own head_sha); a later push creates new runs, new events and new decisions. Residual risk: a compromised vouched account or signing key can run arbitrary code in fork-PR CI (read-only GITHUB_TOKEN, no repo secrets, but runner compute and anything a workflow exposes to pull_request runs), which is the same exposure as a write-access collaborator approving a run. Keep the list short; every approval is posted to Slack.

Deploy notes

• GitHub App permissions: add repository Actions: Read and write and Pull requests: Read-only to the Sheriff App (and accept the permission update on the installation) before adding a vouched_ci list to the config. Without them the vouched_ci token cannot be minted; the rest of Sheriff is unaffected.
• Webhook: no change. The org-wide webhook is already configured for "send me everything", which includes workflow_run. No GitHub App event subscription is needed since Sheriff consumes the org webhook.
• Config order: deploy Sheriff, then open a .permissions PR adding vouched_ci (the dry-run check validates the new key), then merge. Numeric ids come from https://api.github.com/users/<login>.
• Vouched users must sign every commit they push with a GPG/SSH/S/MIME key registered on their GitHub account, push their own commits themselves, and rebase rather than merge non-base branches. See the README "Vouched CI" section.
• Follow-ups worth considering: (a) the cron run could verify each vouched_ci id still resolves to the pinned login via GET /user/{account_id}; (b) a push that triggers N workflows produces N events and N independent commit listings for the same head, which could share a short-lived per-head cache if API volume ever matters; (c) a fork branch opened as pull requests against two base branches at once is refused (two matches) rather than approved, which is deliberate but could be relaxed to "all matches are from the same vouched head".

🤖 Generated with Claude Code

https://claude.ai/code/session_01Wd5WZCENbhvKnJi2EMqrbi

---

Generated by Claude Code