v0.3.1 — adversarial-review iter-1 closure (sdist fix)

[0.3.1] — 2026-05-22

Fixed

• sdist no longer includes maintainer-local working files. Hatchling's
  default sdist target picked up any non-gitignored files in the project
  root; pre-0.3.1 the .idea/, RESEARCH.md, and adversarial-loop/
  directories were all bundled into the published tarball. Adds an explicit
  [tool.hatch.build.targets.sdist].exclude list. The wheel was always
  clean (only benchstone/ source under [tool.hatch.build.targets.wheel]),
  so this affects sdist consumers only. Tagging the v0.3.0 commit early
  meant the fix landed in v0.3.1; v0.3.0 was never published to PyPI.

[0.3.0] — 2026-05-22

This release closes the six CONFIRMED findings from the first adversarial-review
iteration against benchstone's integrity-claim cluster, plus three deferred
quick-wins from the same iteration and the structural keystone fix that
subsumes three more. Several are breaking changes; the minor version bump
reflects that.

Added

• STALE verdict kind and a tier-orthogonal freshness guard on
  bench evaluate / api.evaluate. The guard checks, in order:
  the working tree's git status --porcelain (dirty?), the manifest's
  corpus_hash against the on-disk corpus, and a new working_tree_hash
  against the most recent candidate run at the current SHA. Mismatch
  on any check returns Verdict(kind="STALE", category="freshness")
  with exit code 5. Pass --allow-dirty (CLI) / allow_dirty=True
  (API) to bypass. Skipped for correctness-tier benchmarks.
• bench evaluate --allow-dirty flag, mirroring bench run --allow-dirty.
• project.tracked_paths manifest field (default ["bench/"]).
  Declares the project-relative paths whose byte content is part of the
  benchmark's input surface. Hashed by the runner on each dispatch,
  re-hashed by the freshness guard at evaluate time. Projects that
  import from src/ (or anywhere else) should extend the list.
• benchstone.tree_hash module with working_tree_hash(project_path, tracked_paths) -> "sha256:...". Deterministic recursive hash;
  excludes well-known byproducts (__pycache__/, *.pyc, *.pyo,
  .pytest_cache/, .mypy_cache/, .ruff_cache/, .DS_Store) and
  symlinks.
• benchstone.api.check_freshness as a public helper for external
  callers (custom dashboards, alternate CLIs).
• runs.working_tree_hash column. Nullable; pre-existing rows are
  treated as "no hash to compare against" by the freshness guard.
• baselines.run_ids column (JSON array of run IDs). The baseline
  pointer now records the exact set of runs that constitute its
  snapshot, not a query specification that future inserts can expand.
  See "Changed" → baseline snapshot semantics.
• Store.fetch_runs_by_ids(run_ids) method — primary read path for
  baseline snapshots.

Changed

• Baseline pointer records a run-ID snapshot (the v0.3.0 keystone).
  Baseline.run_ids is the authoritative selector for the baseline
  distribution. compute_verdict reads runs by ID, not by
  (git_sha, meta_seed) query. Closes CONFIRMED-5 structurally
  (subsequent inserts at the same SHA cannot expand the baseline),
  closes F-PROMO-FLOOR (promoted-group size disconnect from manifest's
  baseline_seeds), closes F-MANIFEST-EVOLVE (editing the manifest's
  baseline_seeds no longer invalidates historical baselines), and
  closes F-META-RERUN (re-running with the same meta_seed inserts new
  rows but the snapshot stays fixed). Legacy baselines (pre-0.3.0,
  run_ids=NULL) fall back to the query-based lookup unchanged.
• Gate baseline floor is now hardcoded 2 (the math minimum for a
  sample variance), down from max(len(benchmark.baseline_seeds), 2).
  Sample-size intent is recorded in the snapshot at establish/promote
  time; the gate doesn't re-derive it from the live manifest. Strictly
  more permissive than the old behavior for any sample with ≥ 2 ok
  rows. Candidate floor unchanged (max(repetitions, 2)).
• Append AND git_dirty=0 to both branches of
  Store.fetch_baseline_runs. Dirty rows remain in the timeline
  (fetch_runs is unchanged) — they no longer count as baseline. This
  closes the INV-05 baseline-poisoning vector where an agent with
  --allow-dirty access could inject additional meta_seed=NULL rows
  at the baseline SHA and shift the gate's view of baseline. With the
  keystone snapshot, this filter is also redundant for non-legacy
  baselines (the snapshot doesn't include the injected rows in the
  first place), but it remains as defense-in-depth and as the active
  protection for the legacy-fallback path.
• api.compute_verdict gains optional candidate_meta_seed: int | None = None. Default behavior unchanged (still "latest meta_seed
  group at this SHA"). Non-None pins the candidate fetch to that group.
• api.promote and bench promote evaluate the gate against the
  chosen meta_seed group before moving the baseline pointer, then
  record the chosen group's run IDs as the new baseline snapshot.
  Previously, promote --meta-seed N validated only that group N
  existed and moved the pointer using whichever group was latest at
  the SHA — letting an explicit --meta-seed ride on a different
  group's PROMOTE verdict.
• api.check_freshness signature changed to take project: Project
  (new first arg, for tracked_paths) and an optional store: Store | None = None kwarg. The function was added in the same release cycle
  so no realistic external caller exists yet.
• references.freeze / references.replace ordering inverted —
  history event is appended before the reference JSON is atomically
  written. If _write fails after the history append, get() walks
  history newest-first on every read and raises ReferenceError when
  the most-recent install event's content_hash disagrees with the
  live JSON's. Catches the crash-mid-write half-state.

Fixed

• CONFIRMED-1: bench evaluate / api.evaluate / compute_verdict
  now refuse with STALE when the working tree is dirty, the corpus
  has drifted from corpus_hash, or the tracked_paths byte content
  has changed since the most recent candidate run. Previously emitted
  the gate's verdict against stored runs without checking the on-disk
  state.
• CONFIRMED-2: gitignored modifications under tracked_paths no
  longer slip past the freshness check. The legacy git status --porcelain view (which omits gitignored entries) is now
  complemented by the byte-level working_tree_hash comparison, so
  payloads written to .venv/, __pycache__/, or any
  project-gitignored path under tracked_paths produce STALE.
• CONFIRMED-3: promote --meta-seed N now re-evaluates the gate
  against group N before moving the baseline pointer. Previously a
  REJECT group could become baseline via the verdict of a separate
  PROMOTE group inserted later at the same SHA.
• CONFIRMED-4: bench run --seed-set baseline and bench baseline establish are refused for correctness-tier benchmarks (the
  invocations produced rows with meta_seed=NULL that
  fetch_candidate_runs couldn't see, leaving the artifact invisible
  to evaluate and the default freeze-reference resolution).
• CONFIRMED-5: baseline poisoning via --seed-set baseline --allow-dirty at the baseline SHA. Closed structurally by the
  run-ID snapshot (the gate doesn't read additional rows beyond the
  snapshot) and defense-in-depth by the dirty-row filter.
• CONFIRMED-6: untracked-only dirty trees no longer have
  effectively-no audit content. The working_tree_hash is itself the
  audit primitive for "did the relevant surface change since the
  run?"; the existing git diff HEAD capture remains unchanged for
  the human-inspection use case.
• F-NONFINITE: ProjectResult.from_json rejects NaN /
  ±Infinity metrics at the protocol boundary. The runner's existing
  ProtocolError handler converts the violation into a loud
  status='error' row instead of silently shrinking the sample (NaN
  → SQL NULL while status='ok') or producing nan/inf statistics
  (Infinity → mean_se → directed_z).
• F-REF-ATOMIC: references.freeze and references.replace use
  history-first ordering (see "Changed"), and references.get raises
  ReferenceError when the live reference's content_hash doesn't
  match the most recent history event's recorded hash.
• F-REF-REVALIDATE: references._require_artifact rehashes the
  archived bytes and compares to run.artifact_hash before freeze /
  replace. Catches out-of-band corruption (bit rot, manual tamper,
  cleanup script) of the content-addressed store.
• F-PROMO-FLOOR / F-MANIFEST-EVOLVE / F-META-RERUN: subsumed by
  the run-ID snapshot keystone (see "Changed"). The baseline's
  sample-size is determined at establish/promote time, not re-derived
  from a baseline_seeds query at evaluate time.

Breaking changes

• New exit code 5 for STALE. Consumer scripts that previously
  only branched on 0/1/2/3/4 should add a 5 case (refuse, ask the
  user, re-run, whatever the appropriate response is for the
  consumer). Default bench evaluate behavior on a clean tree is
  unchanged.
• bench run --seed-set baseline on correctness-tier benchmarks
  now raises RunnerError. No callers in the test suite used this
  combination; external callers (if any) need to switch to
  --seed-set fresh (the default).
• bench baseline establish on correctness-tier benchmarks now
  raises RunnerError for the same reason. Use bench run + bench freeze-reference instead.
• promote --meta-seed N now refuses when group N's standalone
  verdict isn't PROMOTE. Previously succeeded silently using a
  different group's verdict. Pass --force to bypass the gate (same
  as before, just now actually applied to the chosen group).
• api.check_freshness signature changed (see "Changed"). Added
  in the same re...