chore: automate ReDoS/config-consistency checks; fix local/CI hook drift - #13
Merged
Merged
Conversation
Parity with npm sibling's same fix. Independent AST-based pattern extraction and a scaling-ratio (not absolute-time) detection strategy, since CPython's regex engine is slow enough that some already-fixed, genuinely-linear patterns overlap in absolute time with real quadratic bugs at any single seed size. Writing the permanent test found two real bugs the earlier manual sweep round had missed: heuristic_analyzer.py's _QA_PATTERN (parity with npm's identical bug) and encoding_detector.py's template_injection (this file wasn't in the earlier manual sweep's scope here at all). Both fixed in this commit.
Satisfies G11 (README documents API changes) for the guard fixes in the prior commit — src/ changed since v0.21.4 without a corresponding README update.
Parity check with the npm sibling's analogous extractor gap (which missed ~28% of its regex literals and hid a real bug). Broadened here to also cover re.search/match/fullmatch/sub/subn/split/findall/finditer calls with inline pattern strings (81 previously-invisible call sites) — found no new bugs, but that's now verified rather than assumed.
- output_filter.py: found a real, live catastrophic-backtracking bug in the email PII pattern (ratio ~16x, one 5s+ timeout) once the extractor was broadened to cover pattern= kwarg dataclass fields (DEFAULT_PII_ PATTERNS/DEFAULT_SECRET_PATTERNS were entirely invisible before this). This exact bug was already fixed elsewhere (ExternalDataGuard, npm's output-filter.ts) but never ported here. Applied the same bounded fix. - heuristic_analyzer.py: the 1000-char bound on _QA_PATTERN created a real many-shot detection bypass (verified 5/5 -> 0/5 on long-turn payloads), parity with the same bug found in npm's heuristic-analyzer.ts. Replaced with a linear marker-position scan (_count_qa_pairs) — no cap, no backtracking risk. Added tests/test_heuristic_analyzer.py (this guard had no dedicated unit tests at all). - test_redos_safety.py: fixed a portability bug where the structural-seed loop called signal.alarm unconditionally (would crash without SIGALRM, e.g. on Windows) unlike the single-char loop's correct branch. - Added tests/test_encoding_detector.py coverage for template_injection (detection + ReDoS timing) — previously untested after its ReDoS fix. All findings from two independent adversarial-review agents (one per guard-safety-net repo).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds permanent, automated safeguards for the two bug classes found manually
during the v0.21.4 hardening pass, so future regressions are caught by the
standard test suite / pre-push hook instead of a one-off human sweep.
tests/test_redos_safety.py(new): extracts regex patterns via twoAST shapes —
re.<func>(pattern, ...)calls and any call with apattern=keyword argument holding a string literal — and stress-testseach with a scaling-ratio check. Found and fixed three real
catastrophic-backtracking regexes:
heuristic_analyzer.py's_QA_PATTERN,encoding_detector.py'stemplate_injection, and (afterbroadening the extractor per review below)
output_filter.py'semailPII pattern.
tests/test_decode_variants.py: content-length consistencyregression test.
.githooks/pre-push: fetches origin's tags before runningscripts/verify.sh— parity fix with the npm sibling, and this exactdrift was caught live during development (a manual
verify.shrunpassed G11 moments before a commit, then the pushed commit failed G11
once the fetch revealed the real diff against origin's tag; fixed by
adding the missing README update).
Independent adversarial review found real issues, all fixed:
_QA_PATTERN's 1000-char-bound ReDoS fix created a real many-shotdetection bypass (parity with the same bug found in npm's sibling PR):
any turn whose Q→A gap exceeds 1000 chars silently stopped being
counted (verified 5/5 → 0/5 on a long-turn payload). Replaced with a
linear marker-position scan (
_count_qa_pairs) — no cap, nobacktracking risk, full detection restored. Added
tests/test_heuristic_analyzer.py(this guard had no dedicated unittests at all).
re.compile(...)calls — broadened to alsocover
re.search/match/sub/etc. (found no new bugs there), thenbroadened again to the
pattern=kwarg shape after review found theentire
DEFAULT_PII_PATTERNS/DEFAULT_SECRET_PATTERNSlist inoutput_filter.py(35 patterns) was invisible to both prior shapes —this time it found a real, live bug: the
emailPII pattern had theexact same catastrophic-backtracking shape already fixed elsewhere
(
ExternalDataGuard, npm'soutput-filter.ts) but never got the parityfix here. Fixed with the same bounded pattern used elsewhere.
signal.alarm-based_time_searchunconditionally, unlike thesingle-char loop's correct
hasattr(signal, "SIGALRM")branch — wouldcrash on a platform without
SIGALRM(e.g. Windows). Fixed to no-opgracefully.
tests/test_encoding_detector.pycoverage fortemplate_injection(detection + ReDoS timing) — previously untested after its ReDoS fix.
No behavior change to any guard's detection logic beyond the fixes above.
Mirrors npm PR #25 (nkratk/llm-trust-guard#25).
Test plan
python3 -m pytest --ignore=tests/adversarial -q— 988 passedbash scripts/verify.sh— all gates passtest fails and correctly identifies the offending pattern (clean
~16x ratios), then reverted
_count_qa_pairsmatches the original unbounded regex'sbehavior exactly on both short and long (>1000 char) turns