@@ -10,7 +10,7 @@ Chromium** — not what the agent said it did.
1010``` bash
1111pnpm eval:skill --agent oracle --track all # reference solutions (must be 100%)
1212pnpm eval:skill --agent claude --track polycss,polycss-noskill # the real control
13- pnpm eval:skill --agent claude --track all # one agent, both tracks, all tasks
13+ pnpm eval:skill --agent claude --track all # one agent, all three tracks
1414pnpm eval:skill --agent all --track all --keep --json out.json
1515pnpm eval:skill --task 03-cube-with-shadow --agent claude,codex
1616pnpm eval:skill --agent claude --reuse --run r1 # re-grade only; never calls an agent
@@ -40,6 +40,13 @@ For each (agent, task) pair:
4040| ` polycss-noskill ` | The ** control** . Same library, task and contract; skill withheld. ` polycss ` minus this is the skill's effect. |
4141| ` three ` | An external ** baseline** , not a control. |
4242
43+ Track order is fixed: skill-less tracks always run first, so no control ever
44+ executes while an installed skill exists on disk. That buys isolation at the
45+ cost of confounding * timing* — every control runs before its intervention, so
46+ cache warming, rate limits and provider variation are inseparable from any
47+ duration difference. Treat wall-clock deltas as a fixed-order observation, not
48+ an effect of the skill. Only the per-check scores are comparable.
49+
4350The Three.js track changes the library, its API and its authoring contract at
4451the same time as it removes the skill, so a polycss-vs-three delta conflates the
4552skill with how familiar and how costly each library is for that model. It still
@@ -143,9 +150,12 @@ failed to say something, said it somewhere the agent did not look, or the agent
143150ignored it. The first two are fixable in ` packages/skills/skill/ ` .
144151
145152Use ` --keep ` to leave the workspaces under ` $TMPDIR/polycss-skill-eval/<run> `
146- and read what the agent actually wrote. Runs are namespaced by ` --run <id> `
147- (default ` current ` ) so a later run cannot wipe evidence you kept, and
148- concurrent runs do not collide.
153+ and read what the agent actually wrote. Runs are namespaced by ` --run <id> ` so a later
154+ run cannot wipe evidence you kept, and concurrent runs do not collide.
155+
156+ Run ids are random by default (` run-a1b2c3d4 ` ) and printed at the top of the
157+ run, so nothing can overwrite anything; ` --reuse ` therefore requires an explicit
158+ ` --run <id> ` .
149159
150160` --reuse ` re-grades a kept run and ** never invokes an agent** : a case with no
151161` scene.mjs ` is skipped and reported, not re-run. That is how to iterate on a
0 commit comments