Skip to content

fix(repetition): skip detection for table-heavy output to prevent false positive loops#1574

Open
gorevyoneticisi wants to merge 1 commit into
XiaomiMiMo:mainfrom
gorevyoneticisi:fix/repetition-detector-table-false-positive
Open

fix(repetition): skip detection for table-heavy output to prevent false positive loops#1574
gorevyoneticisi wants to merge 1 commit into
XiaomiMiMo:mainfrom
gorevyoneticisi:fix/repetition-detector-table-false-positive

Conversation

@gorevyoneticisi

Copy link
Copy Markdown

Summary

The sliding n-gram repetition detector triggers false positives on markdown table output. Structured tables naturally have high n-gram overlap because of pipe separators (|) and uniform row structure. This causes the agent to enter an infinite retry loop when generating tables with 5+ rows and 4+ columns (issue #1441).

Root Cause

detectConsecutiveRepeat() scans for tokens that repeat at a fixed period. In a markdown table, each row starts with | followed by similar cell patterns, creating exactly the kind of periodic signal the detector looks for. The detector cannot distinguish "format-driven repetition" (table rows) from "content-driven repetition" (actual loops).

Fix

Added isTableHeavy() that detects when >40% of non-empty lines are table rows (lines starting with | or separator lines). When detected, skip consecutive repeat detection entirely. This is a clean, targeted fix:

  • No-op for non-table output (real loops still detected)
  • No threshold tuning or complex heuristics
  • Table detection is simple and fast (single pass over lines)

Changes

  • packages/opencode/src/session/prompt/text-ngram-detection.ts: Added isTableHeavy() helper and gate in TextNgramMonitor.append()
  • packages/opencode/test/session/text-ngram-detection.test.ts: Added 2 new tests covering streaming large tables and table-then-new-content scenarios

Test Results

All 25 n-gram detection tests pass (23 existing + 2 new). No regressions.

Closes #1441

…se positives

The sliding n-gram repetition detector triggers on markdown tables because
table rows naturally have high n-gram overlap due to pipe separators and
uniform row structure. This causes the agent to enter an infinite
retry loop when generating structured tabular output with 5+ rows.

Add isTableHeavy() check that detects when >40% of non-empty lines are
table rows. When detected, skip consecutive repeat detection entirely.

Adds 2 new tests: streaming large table (the exact scenario from XiaomiMiMo#1441)
and table-then-new-content to verify no false positives.

Closes XiaomiMiMo#1441
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Repetition detector false-positive on structured table output causes infinite retry loop

1 participant