fix(si): harden evidence, proposal ids, eval mode, registry, and eval gate - #232
Merged
Merged
Conversation
… gate - record refuses duplicate evidence basenames before writing, so a successful record always leaves check passing. - propose allocates unique proposal ids (-2, -3 suffix, exclusive create) and refuses an explicit --id that already exists. - register refuses eval-mode repositories, and the eval-mode check covers subdirectories of a workspace or worktree (up to the same jj-first boundary resolve_toplevel uses, so nested .git repos do not escape it); si_global skips eval-mode projects in scan/cluster/propose and flags them in projects. - propose folds --title, and the rule text of agents-md proposals (also on apply), so they cannot inject markdown into AGENTS.md; skill proposal text stays verbatim as the whole SKILL.md. - si_global fails on an invalid or wrongly shaped registry; a missing registry is still empty. - Replace the deleted evals/run_evals.py gate with Nova's CI checks in propose output, si-global SKILL.md, and ADR 0035. - si.py init/register refuse an invalid or wrongly shaped registry instead of treating it as empty, leaving the file untouched. - propose validates ids and skill names before any filesystem access: new ids must match the raw trace id pattern, --apply accepts an existing proposal named by any single safe path component, and --skill-name or a stored skillName must be a safe component (no separators, leading dot, or control characters). A corrupt proposal file on --apply is a controlled si error. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VnZxcmzmwzvXQXfwFk6vXW
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the bugs found by an end-to-end run of the installed si-project / si-global CLIs, plus follow-ups from two independent review passes.
Fixes
recordrefuses duplicate evidence basenames instead of silently overwriting (which leftcheckfailing).-2,-3suffixes via exclusive create; an existing explicit--idis refused.registerrefuses; the marker is honoured from any subdirectory of a jj workspace or git worktree (including nested git repos inside a jj workspace); si-global skips eval-mode projects and flags them inprojects.agents-mdrule text and titles are folded to one line (no markdown injection into AGENTS.md);skillproposal text stays verbatim.si.py(no silent rewrite) andsi_global.py; a missing registry is still empty.evals/run_evals.py; it now names the CI checks (unittest discover,update-guide.py --check,package.py), with a test that the paths exist and appear in ci.yml.--idand--skill-name/ storedskillNameare validated before filesystem access (no traversal, no control characters); legacy ids still apply; corrupt proposal JSON givessi error:exit 1.Verification
python3 -m unittest discover -s tests: 157 tests OK;update-guide.py --checkOK;package.pyOK🤖 Generated with Claude Code
https://claude.ai/code/session_01VnZxcmzmwzvXQXfwFk6vXW