Skip to content

chore: automate ReDoS/config-consistency checks; fix local/CI hook drift - #13

Merged
nkratk merged 4 commits into
mainfrom
chore/redos-safety-net-and-hook-fix
Jul 23, 2026
Merged

nkratk merged 4 commits into
mainfrom
chore/redos-safety-net-and-hook-fix

Conversation

@nkratk

@nkratk nkratk commented Jul 23, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds permanent, automated safeguards for the two bug classes found manually
during the v0.21.4 hardening pass, so future regressions are caught by the
standard test suite / pre-push hook instead of a one-off human sweep.

  • tests/test_redos_safety.py (new): extracts regex patterns via two
    AST shapes — re.<func>(pattern, ...) calls and any call with a
    pattern= keyword argument holding a string literal — and stress-tests
    each with a scaling-ratio check. Found and fixed three real
    catastrophic-backtracking regexes: heuristic_analyzer.py's
    _QA_PATTERN, encoding_detector.py's template_injection, and (after
    broadening the extractor per review below) output_filter.py's email
    PII pattern.
  • tests/test_decode_variants.py: content-length consistency
    regression test.
  • .githooks/pre-push: fetches origin's tags before running
    scripts/verify.sh — parity fix with the npm sibling, and this exact
    drift was caught live during development (a manual verify.sh run
    passed G11 moments before a commit, then the pushed commit failed G11
    once the fetch revealed the real diff against origin's tag; fixed by
    adding the missing README update).

Independent adversarial review found real issues, all fixed:

  1. _QA_PATTERN's 1000-char-bound ReDoS fix created a real many-shot
    detection bypass
    (parity with the same bug found in npm's sibling PR):
    any turn whose Q→A gap exceeds 1000 chars silently stopped being
    counted (verified 5/5 → 0/5 on a long-turn payload). Replaced with a
    linear marker-position scan (_count_qa_pairs) — no cap, no
    backtracking risk, full detection restored. Added
    tests/test_heuristic_analyzer.py (this guard had no dedicated unit
    tests at all).
  2. The extractor only matched re.compile(...) calls — broadened to also
    cover re.search/match/sub/etc. (found no new bugs there), then
    broadened again to the pattern= kwarg shape after review found the
    entire DEFAULT_PII_PATTERNS/DEFAULT_SECRET_PATTERNS list in
    output_filter.py (35 patterns) was invisible to both prior shapes —
    this time it found a real, live bug: the email PII pattern had the
    exact same catastrophic-backtracking shape already fixed elsewhere
    (ExternalDataGuard, npm's output-filter.ts) but never got the parity
    fix here. Fixed with the same bounded pattern used elsewhere.
  3. A portability bug: the structural-seed timing loop called
    signal.alarm-based _time_search unconditionally, unlike the
    single-char loop's correct hasattr(signal, "SIGALRM") branch — would
    crash on a platform without SIGALRM (e.g. Windows). Fixed to no-op
    gracefully.
  4. Added tests/test_encoding_detector.py coverage for template_injection
    (detection + ReDoS timing) — previously untested after its ReDoS fix.

No behavior change to any guard's detection logic beyond the fixes above.
Mirrors npm PR #25 (nkratk/llm-trust-guard#25).

Test plan

  • python3 -m pytest --ignore=tests/adversarial -q — 988 passed
  • bash scripts/verify.sh — all gates pass
  • Reintroduced each fixed regex in throwaway local edits, confirmed the
    test fails and correctly identifies the offending pattern (clean
    ~16x ratios), then reverted
  • Verified _count_qa_pairs matches the original unbounded regex's
    behavior exactly on both short and long (>1000 char) turns
  • CI green on all jobs (Python 3.9-3.13, lint, security, verification gate)

nkratk added 4 commits July 22, 2026 19:59
Parity with npm sibling's same fix. Independent AST-based pattern
extraction and a scaling-ratio (not absolute-time) detection strategy,
since CPython's regex engine is slow enough that some already-fixed,
genuinely-linear patterns overlap in absolute time with real quadratic
bugs at any single seed size.

Writing the permanent test found two real bugs the earlier manual
sweep round had missed: heuristic_analyzer.py's _QA_PATTERN (parity
with npm's identical bug) and encoding_detector.py's template_injection
(this file wasn't in the earlier manual sweep's scope here at all).
Both fixed in this commit.
Satisfies G11 (README documents API changes) for the guard fixes
in the prior commit — src/ changed since v0.21.4 without a
corresponding README update.
Parity check with the npm sibling's analogous extractor gap (which missed
~28% of its regex literals and hid a real bug). Broadened here to also
cover re.search/match/fullmatch/sub/subn/split/findall/finditer calls with
inline pattern strings (81 previously-invisible call sites) — found no new
bugs, but that's now verified rather than assumed.
- output_filter.py: found a real, live catastrophic-backtracking bug in
  the email PII pattern (ratio ~16x, one 5s+ timeout) once the extractor
  was broadened to cover pattern= kwarg dataclass fields (DEFAULT_PII_
  PATTERNS/DEFAULT_SECRET_PATTERNS were entirely invisible before this).
  This exact bug was already fixed elsewhere (ExternalDataGuard, npm's
  output-filter.ts) but never ported here. Applied the same bounded fix.
- heuristic_analyzer.py: the 1000-char bound on _QA_PATTERN created a real
  many-shot detection bypass (verified 5/5 -> 0/5 on long-turn payloads),
  parity with the same bug found in npm's heuristic-analyzer.ts. Replaced
  with a linear marker-position scan (_count_qa_pairs) — no cap, no
  backtracking risk. Added tests/test_heuristic_analyzer.py (this guard
  had no dedicated unit tests at all).
- test_redos_safety.py: fixed a portability bug where the structural-seed
  loop called signal.alarm unconditionally (would crash without SIGALRM,
  e.g. on Windows) unlike the single-char loop's correct branch.
- Added tests/test_encoding_detector.py coverage for template_injection
  (detection + ReDoS timing) — previously untested after its ReDoS fix.

All findings from two independent adversarial-review agents (one per
guard-safety-net repo).
@nkratk
nkratk merged commit 933478d into main Jul 23, 2026
8 checks passed
@nkratk
nkratk deleted the chore/redos-safety-net-and-hook-fix branch July 23, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant