Skip to content

0.3.11: a coverage problem says so at every door - #74

Merged
KayhanB21 merged 2 commits into
masterfrom
0311-coverage-problems-say-so
Oct 10, 2026
Merged

KayhanB21 merged 2 commits into
masterfrom
0311-coverage-problems-say-so

Conversation

@KayhanB21

@KayhanB21 KayhanB21 commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

A malformed coverage_map shard is exit 2, a dot-prefixed map key matches, and a TypeScript report that matches nothing says so. This is PR 2 of 0.3.11.

The findings

These were reproduced on the installed 0.3.10. A shard is one coverage file in a [tool.riskratchet.coverage_map] table.

Finding Observed
A malformed shard is a warning With one shard set to [], check and the pytest plugin score that package at 0% and exit 1 with regressions. check --coverage on the same bytes exits 2, and doctor says FAIL.
A map prefix that starts with a dot never matches ".tools/a" = ... scores every function under .tools/a at 0% line coverage.
An Istanbul report that matches 0 TypeScript files is silent on stdout stderr has one line. The text, markdown, and PR comment formats say nothing. 0.3.10 added the note for Python files only.

The fix

  • coverage.load_coverage_map raises ValueError for a shard that exists and can't be parsed. _build_report_or_exit and the plugin's _report_or_fail already turn that into a setup error. A missing shard still goes through on_error and the allow_missing_coverage rule.
  • _normalize_prefix removes only a leading ./. .lstrip("./") strips characters, so .tools/a became tools/a.
  • RiskReport gains ts_coverage_checked_files and ts_coverage_unmatched_files. They aren't serialized. coverage_note() joins the Python note and TypeScript coverage matched 0 of N scanned files, so every renderer and the stderr warning print it. The two counts stay separate, so one backend matching everything doesn't hide the other matching nothing.

Behavior change

A gate that passes today with a corrupt shard starts failing with exit 2. That needs missing_coverage = "optimistic" or "skip"; under the default policy the gate already exits 1. allow_missing_coverage doesn't cover it, because the shard is there. The release CHANGELOG states this under Changed.

The following don't change: scores for a project whose shards parse and whose prefixes have no leading dot, the gate verdict in those projects, SCORING_MODEL_VERSION, and all nine JSON schemas.

Tests

The suite has 1413 passed and 2 skipped, with 93.46% coverage. 13 test cases fail against the pre-fix src/: 5 commands for the malformed shard, the plugin, the loader, the .tools/a prefix, and 5 TypeScript note tests in the new tests/test_typescript_coverage_overlap.py. Three tests pinned the old warning (test_load_coverage_map_skips_unreadable_shard, test_load_coverage_map_without_callback_skips_silently, and test_scan_with_malformed_coverage_map_shard_continues), and this PR rewrites them.

Baseline bump rationale

regenerate-baseline.yml regenerated the baseline in CI (run 38063619367, commit f9bf5df). It goes from 576 to 577 entries, and the scoring block doesn't change. 15 of the 576 existing entries change score. The largest rise is +3.75, under fail_regression_above = 5.

  • New (1 function): models.py::RiskReport._coverage_notes at 2.0.
  • Edited here, up (5 functions): RiskReport.coverage_note +3.75 and RiskReport.coverage_warning +2.5, which now handle two notes; _normalize_prefix +3.5, which gains a loop and a . case; analyze_typescript +1.34, which counts measurable files; and coverage_overlap_note +1.0, which gains the label argument.
  • Edited here, down (1 function): _load_shard -0.25.
  • Churn only (9 entries): load_coverage_map, _coverage_shard_warn, and _ensure_coverage_map_exists at +1.0 each, where this PR edits a docstring or shifts the span, and 6 entries in typescript_engine.py and cli.py between -2.0 and +1.0.

🤖 Generated with Claude Code

KayhanB21 and others added 2 commits October 10, 2026 08:26
- A `coverage_map` shard that exists and cannot be parsed is exit 2 at every
  command and fails the plugin session. It was a warning, and the gate then
  exited 1 for an I/O failure. A missing shard behaves as before.
- A map prefix loses only a leading `./`. `.lstrip("./")` turned `.tools/a`
  into `tools/a`, which matched nothing.
- A TypeScript report that matches none of the scanned files says so on
  stderr and in every stdout format, with its own counts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@KayhanB21
KayhanB21 merged commit c3b4172 into master Oct 10, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant