Skip to content

fix(ir): the checked turn asks nothing (ENG-745) - #317

Merged
henokteixeira merged 8 commits into
mainfrom
henok/the-checked-turn-asks-nothing
Sep 8, 2026
Merged

henokteixeira merged 8 commits into
mainfrom
henok/the-checked-turn-asks-nothing

Conversation

@henokteixeira

@henokteixeira henokteixeira commented Sep 2, 2026 •

Copy link
Copy Markdown
Contributor

Linear: ENG-745

Context

R5 finding from the 09/02 afternoon round of the open flow (no new issue). No new issue yet; cite this PR as its origin.

When the analyst finds nothing and the evidence is sufficient, the route marks checked = True (app/api/internalization_room/back_translation.py) — the passage has been checked and there is no next turn. But closing_block(None) still returned CLOSING_PLAIN: "End with exactly one answerable question or invitation." The narrator obeyed: "Como vocês se sentem agora que terminaram de ensaiar essa parte da história?" — a question nobody was going to answer.

What changed

  • closing_block(finding, *, checked: bool = False) gains the one parameter it was missing — state.checked, which the route already computed before this call.

  • New block, CLOSING_CHECKED, chosen only when there is no finding and the passage was just checked:

    • Say plainly that the passage is told and checked, then stop there. Do not ask a question, do not invite them to answer anything, do not ask how the team feels, and do not say goodbye. There is no next turn on this passage — the screen takes the team on from here. Never a checklist, never a speech.

    Goes to the product owner for review, recorded here in full as the plan asked.

  • The router now calls room.closing_block(finding, checked=state.checked).

Code-review finding: a second contradiction of the same family

Code review found a static, unconditional line in the prompt template (prompts/backtranslation_verdict_system_prompt.md), right below the {{CLOSING}} slot, on every verdict turn:

- After the team acts on this one, they will finish the telling-back again.

With CLOSING_CHECKED, that line started appearing right after "there is no next turn on this passage" — the same kind of defect as the original finding, promising continuation on the one turn that has none. The line came out of the prompt's static text and now lives inside each closing that actually has a next round (CLOSING_PLAIN, CLOSING_SPOKEN, CLOSING_ON_SCREEN, CLOSING_MISSING_ON_SCREEN, CLOSING_MISSING_TO_REHEARSAL) — never in CLOSING_CHECKED. Approved by the Orchestrator to land in this same PR.

⚠️ Operational note — an already-seeded ir_prompts row does not change on its own

get_prompt_text reads IRPrompt.prompt from ir_prompts and only falls back to the .md (default_prompt) when no row exists for the key; seed_room_prompts is idempotent and does not update a row that already exists. Consequence: in any environment where BT_VERDICT_SPEAKER was already seeded before this PR, removing the static line from the .md does not propagate on its own — the old line keeps being served until someone updates that row in ir_prompts by hand. The CLOSING_CHECKED part of the fix itself is not affected by this: {{CLOSING}} is substituted at runtime with whatever closing_block() computes, so that part of the fix already holds for any stored row, old or new.

Scope respected

Nothing beyond this: the other closing blocks keep their text and decision order; the analyst was not touched.

Testing Plan

New file tests/test_ir_the_checked_turn_asks_nothing.py, shaped like tests/test_internalization_room_back_translation.py — imports the ENG-676 fixtures/doubles (patch_loop, patch_speaker) from the sibling file, the way tests/test_oc_review_flags.py → tests/test_oc_recording_description_rule.py already does.

  1. No finding and sufficient evidence: the closing does not ask a question — the narrator's system prompt is captured; absence of "answerable question"/"invitation", presence of "Do not ask", absence of the "finish the telling-back again" line. Failed before this PR.
  2. The validator sees the same closing — the validator's brief ({{ORDERED_CLOSING}}) carries CLOSING_CHECKED, not CLOSING_PLAIN. Failed before.
  3. An obedient narrator passes — used_fail_safe is False. The negative mirror (a narrator that still asks) was not written: the ENG-676 double ValidatorReadsOnlyItsOwnPrompt judges three things read from its own prompt (unsupported claim, uncited destination, unstated name), and none of them read the closing's "do not ask" instruction — so it cannot tell a working CLOSING_CHECKED apart from a broken one on that axis. Documented in the test's docstring rather than pretending coverage that does not exist.
  4. No finding, insufficient evidence: nothing changes (guard) — CLOSING_PLAIN, and it still contains the continuation line.
  5. With a finding: nothing changes (guard) — checked is ignored when there is a finding; it still contains the continuation line.

Mutation (paired, one per finding)

  • Temporarily restored an unconditional return CLOSING_PLAIN in closing_block: cases 1 and 2 fail.
  • Temporarily restored the unconditional static line in the .md: case 1 fails again (the new assertion about "finish the telling-back again").
  • Both reverted after confirming; the suite goes back to green.

Verification

JWT_SECRET_KEY=test-secret-for-pytest-only DATABASE_URL="sqlite+aiosqlite:///./test.db" uv run pytest tests/ -q
2455 passed, 5 skipped, 1 xfailed, 0 failed

ruff check . and uv run mypy app/ clean. Test-quality review and code self-review done by subagents before this PR; no additional finding beyond what is already recorded above.

No merge.

henokteixeira and others added 5 commits September 2, 2026 17:39
R5 (rodada de 02/09): quando o analista não acha nada e a evidência é
suficiente, a rota marca `checked = True` — a passagem está conferida e não
há próximo turno — mas `closing_block(None)` ainda devolvia CLOSING_PLAIN,
que manda o narrador terminar com uma pergunta. O narrador obedecia e
perguntava algo que ninguém ia responder.

closing_block(finding, *, checked=False) ganha o parâmetro mínimo que a
rota já calculava (`state.checked`), e um bloco novo, CLOSING_CHECKED, é
escolhido só quando não há achado e a passagem acabou de ser conferida:
diz que a passagem foi contada e conferida, e para — sem pergunta, sem
convite, sem perguntar como se sentem, sem despedida.

Revisão de código encontrou uma segunda contradição da mesma família: o
prompt do narrador tinha uma linha estática, incondicional, logo abaixo do
slot {{CLOSING}} — "After the team acts on this one, they will finish the
telling-back again." — em todo turno de verdict. Com CLOSING_CHECKED, essa
linha passou a aparecer bem depois de "there is no next turn on this
passage", prometendo continuação no único turno que não tem. A linha saiu
do texto estático do prompt e passou a viver dentro de cada fechamento que
de fato tem um próximo round (CLOSING_PLAIN, CLOSING_SPOKEN,
CLOSING_ON_SCREEN, CLOSING_MISSING_ON_SCREEN, CLOSING_MISSING_TO_REHEARSAL)
— e não em CLOSING_CHECKED.

Nada além disso: os outros blocos de fechamento mantêm seu texto e ordem
de decisão, e o analista não foi tocado.
🤖 Generated with [Nori](https://noriagentic.com)

Co-Authored-By: Nori <contact@tilework.tech>
Same fork as in #326: 20260901_room09 (this line's BT_CORRECTION prompt
key) and 20260902_room09 (main's drop of ir_prompts, PR #320) both
descend from 20260831_lang01 with nothing joining them.
🤖 Generated with [Nori](https://noriagentic.com)

Co-Authored-By: Nori <contact@tilework.tech>
# Conflicts:
#	alembic/versions/20260903_join_bt_correction_and_drop_ir_prompts.py
@henokteixeira henokteixeira changed the title ENG: o turno que confere a passagem não faz pergunta fix(ir): the checked turn asks nothing (ENG-745) Sep 4, 2026
@linear-code

linear-code Bot commented Sep 4, 2026

Copy link
Copy Markdown

ENG-745

#: process an ending. Left there it would have sat right after "there is no next turn" and
#: said the opposite in the same breath, so it now lives inside each closing that still has
#: a next round instead, and not in the one that does not.
_NEXT_ROUND = "After the team acts on this one, they will finish the telling-back again."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The move itself looks right to me, but the ⚠️ operational note on the description does not match what I see here:

get_prompt_text reads IRPrompt.prompt from ir_prompts and only falls back to the .md (default_prompt) when no row exists for the key

On head get_prompt_text is just return default_prompt(key)["prompt"] (app/services/internalization_room/prompts.py) — the committed file, no row read at all — and ir_prompts was dropped in alembic/versions/20260902_room09_drop_ir_prompts.py. So if I am reading it right the .md removal propagates by itself and nobody has to touch a row by hand, and the warning as written sends someone to a table that is not there anymore. Could you verify that and, if it is the case, drop the note from the description?

@little-joao little-joao Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nothing blocking, one point inline

@henokteixeira
henokteixeira changed the base branch from henok/the-closings-match-the-screen to main September 8, 2026 00:00
@henokteixeira
henokteixeira merged commit 27fc929 into main Sep 8, 2026
10 checks passed
@henokteixeira
henokteixeira deleted the henok/the-checked-turn-asks-nothing branch September 9, 2026 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant