Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/api/internalization_room/back_translation.py
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,7 @@ async def finish(

outcome = await room.run_verdict_turn(
findings_text=room.findings_block(finding),
closing=room.closing_block(finding),
closing=room.closing_block(finding, checked=state.checked),
scope=state.scope or session.pericope,
pericope_num=session.pericope,
messages=session.messages or [],
Expand Down
84 changes: 61 additions & 23 deletions app/services/internalization_room/back_translation.py
Original file line number Diff line number Diff line change
Expand Up @@ -581,44 +581,82 @@ def findings_block(finding: Finding | None) -> str:
return f"- {finding.kind}: {finding.note}"


CLOSING_ON_SCREEN = """- End by handing the choice to the screen, not by asking for a spoken \
#: What every closing below promises except `CLOSING_CHECKED`: the process goes on. It used
#: to be a static line in the prompt template itself, right under `{{CLOSING}}` and outside
#: any branch — true of every verdict turn there was, until `CLOSING_CHECKED` gave the
#: process an ending. Left there it would have sat right after "there is no next turn" and
#: said the opposite in the same breath, so it now lives inside each closing that still has
#: a next round instead, and not in the one that does not.
_NEXT_ROUND = "After the team acts on this one, they will finish the telling-back again."

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The move itself looks right to me, but the ⚠️ operational note on the description does not match what I see here:

get_prompt_text reads IRPrompt.prompt from ir_prompts and only falls back to the .md (default_prompt) when no row exists for the key

On head get_prompt_text is just return default_prompt(key)["prompt"] (app/services/internalization_room/prompts.py) — the committed file, no row read at all — and ir_prompts was dropped in alembic/versions/20260902_room09_drop_ir_prompts.py. So if I am reading it right the .md removal propagates by itself and nobody has to touch a row by hand, and the warning as written sends someone to a table that is not there anymore. Could you verify that and, if it is the case, drop the note from the description?


CLOSING_ON_SCREEN = (
"""- End by handing the choice to the screen, not by asking for a spoken \
answer. This stretch is on screen with its two voices side by side: theirs, in their own \
language, and the telling in {session_language}. Ask the boundary question above, then in one \
short sentence tell them they can listen to both and tap the microphone of the voice that has \
to speak again. Do not ask them to say the answer out loud, and do not offer any other next \
step — the screen offers exactly those two, and naming a third promises something they cannot \
do. Remaining findings wait for the next round. Never a checklist, never a speech."""
do. Remaining findings wait for the next round. """
+ _NEXT_ROUND
+ " Never a checklist, never a speech."
)

CLOSING_PLAIN = """- End with exactly one answerable question or invitation. Remaining \
findings wait for the next round. Never a checklist, never a speech."""
CLOSING_PLAIN = (
"- End with exactly one answerable question or invitation. Remaining findings wait for "
"the next round. " + _NEXT_ROUND + " Never a checklist, never a speech."
)
#: Word for word what this prompt closed with before the screen existed. A turn with no finding
#: at all affirms and names the badge; both other closings explain themselves in terms of *this
#: finding*, and there is none — `findings_block` is saying so in the same prompt.

CLOSING_SPOKEN = """- End with exactly one answerable question or invitation, and let them \
answer in words. This finding does not land on one stretch, so there is no stretch on screen \
and no two voices to choose between — the next conversational turn will respond to what they \
say. Remaining findings wait for the next round. Never a checklist, never a speech."""
CLOSING_CHECKED = """- Say plainly that the passage is told and checked, then stop there. Do \
not ask a question, do not invite them to answer anything, do not ask how the team feels, and \
do not say goodbye. There is no next turn on this passage — the screen takes the team on from \
here. Never a checklist, never a speech."""
#: The one turn with no finding that also has no next round: `state.checked` closes the
#: passage for good, so a question here would ask for an answer nobody will ever read — and,
#: unlike every other closing, this one may not carry `_NEXT_ROUND` either.

CLOSING_SPOKEN = (
"- End with exactly one answerable question or invitation, and let them answer in words. "
"This finding does not land on one stretch, so there is no stretch on screen and no two "
"voices to choose between — the next conversational turn will respond to what they say. "
"Remaining findings wait for the next round. "
+ _NEXT_ROUND
+ " Never a checklist, never a speech."
)

CLOSING_MISSING_ON_SCREEN = """- End by handing the choice to the screen, not by asking for a \
spoken answer. This stretch is on screen, and something the passage has is not in it. In one \
short sentence, tell them they can listen to both voices and then record this part again — \
the whole part, what they already told and what was missing. The screen offers exactly one \
microphone for that; do not name a second one, and do not ask them to say the answer out \
loud. Remaining findings wait for the next round. Never a checklist, never a speech."""
CLOSING_MISSING_ON_SCREEN = (
"- End by handing the choice to the screen, not by asking for a spoken answer. This "
"stretch is on screen, and something the passage has is not in it. In one short sentence, "
"tell them they can listen to both voices and then record this part again — the whole "
"part, what they already told and what was missing. The screen offers exactly one "
"microphone for that; do not name a second one, and do not ask them to say the answer out "
"loud. Remaining findings wait for the next round. "
+ _NEXT_ROUND
+ " Never a checklist, never a speech."
)

CLOSING_MISSING_TO_REHEARSAL = """- End by handing the choice to the screen, not by asking for \
a spoken answer. The end of the story has not been told yet — nothing they recorded is wrong, \
and nothing they recorded will be lost. In one short sentence, tell them they can go on and \
record what is still missing, and that what they already recorded stays. The screen offers \
exactly one microphone for that. Do not offer to settle it later, do not ask them to choose \
between voices, and do not ask them to say anything out loud. Never a checklist, never a \
speech."""
CLOSING_MISSING_TO_REHEARSAL = (
"- End by handing the choice to the screen, not by asking for a spoken answer. The end of "
"the story has not been told yet — nothing they recorded is wrong, and nothing they "
"recorded will be lost. In one short sentence, tell them they can go on and record what "
"is still missing, and that what they already recorded stays. The screen offers exactly "
"one microphone for that. Do not offer to settle it later, do not ask them to choose "
"between voices, and do not ask them to say anything out loud. "
+ _NEXT_ROUND
+ " Never a checklist, never a speech."
)


def closing_block(finding: Finding | None) -> str:
def closing_block(finding: Finding | None, *, checked: bool = False) -> str:
"""How the Speaker is told to end this turn: handing to the screen, or asking out loud.

`checked` is the caller's `state.checked` — whether this turn, with no finding, is also
the one that strikes the passage off the wheel for good. It only ever matters when
`finding` is `None`: a turn with a finding is not the checked turn, whatever `checked`
says, so the flag is read nowhere else in this function.

Chosen here rather than by the Speaker reading a branch, because the finding carries the
deciding fact and the prompt does not: `findings_block` sends kind and note, never the
address. A prompt that branched would be asking a model not to promise a choice the screen
Expand Down Expand Up @@ -646,7 +684,7 @@ def closing_block(finding: Finding | None) -> str:
again and reaches the model as literal braces.
"""
if finding is None:
return CLOSING_PLAIN
return CLOSING_CHECKED if checked else CLOSING_PLAIN
if finding.kind is FindingKind.MISSING:
return (
CLOSING_MISSING_ON_SCREEN
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,6 @@ recording is their work in their language; your ears only reach the telling.
found: too little to check is not a clean check.
- **Unclear:** just ask them to tell that piece again — no fuss.
{{CLOSING}}
- After the team acts on this one, they will finish the telling-back again.
- Never mention the map, findings, analysis, or any inner working. Ground everything in
*"a história conta / não conta"* and *"o que você me contou"*.

Expand Down
157 changes: 157 additions & 0 deletions tests/test_ir_the_checked_turn_asks_nothing.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
import pytest

from app.services.internalization_room.back_translation import (
CLOSING_CHECKED,
CLOSING_PLAIN,
Finding,
FindingKind,
closing_block,
findings_block,
segments_block,
)
from app.services.internalization_room.run_turn import run_verdict_turn
from tests.test_internalization_room_back_translation import (
SPEAKER,
VALIDATOR,
P,
_settings,
_told,
patch_loop,
patch_speaker,
)

#: Silences an unused-import lint warning: pytest discovers these fixtures by name because
#: they are imported into this module's namespace, not because anything here calls them
#: directly.
_FIXTURES = (patch_loop, patch_speaker)

#: The phrase every kind but this one still closes with. Its absence is half of what "asks
#: nothing" means here.
ANSWERABLE_QUESTION = "answerable question"

#: The other half of the old instruction's vocabulary — the word this closing may not use
#: even if it avoids a literal question mark.
INVITATION = "invitation"

#: The prompt's own promise of a next round, previously a static line under `{{CLOSING}}`
#: on every verdict turn. `CLOSING_CHECKED` has no next round, so this and it may not both
#: reach the Speaker on the same turn — found in code review, same class of defect as the
#: original bug: a signal that promises continuation on the one turn that has none.
CONTINUES_TELLING_BACK = "finish the telling-back again"


async def _checked_turn_for(draft: str, patch_speaker) -> str:
"""The Speaker's system prompt on a turn with no finding that closes `checked`."""
agent = patch_speaker(draft)
await run_verdict_turn(
session_language="Portuguese",
language_code="pt",
findings_text=findings_block(None),
closing=closing_block(None, checked=True),
scope=P,
pericope_num=P,
messages=[],
speaker_prompt=SPEAKER,
validator_prompt=VALIDATOR,
settings=_settings(),
)
return str(agent.seen[0])


async def _checked_turn_with_loop(draft: str, patch_loop):
"""Same turn, through the draft-and-gate loop so the Validator's own brief is visible."""
told = _told()
agent = patch_loop(draft, told)
outcome = await run_verdict_turn(
session_language="Portuguese",
language_code="pt",
findings_text=findings_block(None),
closing=closing_block(None, checked=True),
scope=P,
pericope_num=P,
messages=[],
telling_back=segments_block(told),
speaker_prompt=SPEAKER,
validator_prompt=VALIDATOR,
settings=_settings(),
)
return outcome, agent


@pytest.mark.asyncio
async def test_a_checked_turn_does_not_ask_a_question(patch_speaker) -> None:
"""Case 1. Sem achado e com evidência suficiente, o fechamento não pede pergunta.

R5: a rota marcava `checked = True` e ainda assim mandava o narrador terminar com
"exactly one answerable question or invitation" — o `CLOSING_PLAIN` de sempre. Não há
próximo turno depois de `checked`, então a pergunta não tinha para quem responder.
"""
spoken_to = await _checked_turn_for("A passagem foi contada e conferida.", patch_speaker)

assert ANSWERABLE_QUESTION not in spoken_to
assert INVITATION not in spoken_to
assert "Do not ask" in spoken_to
assert CLOSING_PLAIN not in spoken_to
assert CLOSING_CHECKED in spoken_to
assert CONTINUES_TELLING_BACK not in spoken_to


@pytest.mark.asyncio
async def test_the_validator_is_shown_the_checked_closing_not_the_plain_one(
patch_loop,
) -> None:
"""Case 2. O validador vê o mesmo fechamento que o narrador recebeu, não `CLOSING_PLAIN`."""
_, agent = await _checked_turn_with_loop("A passagem foi contada e conferida.", patch_loop)

assert CLOSING_CHECKED in agent.briefs[0]
assert CLOSING_PLAIN not in agent.briefs[0]
assert ANSWERABLE_QUESTION not in agent.briefs[0]


@pytest.mark.asyncio
async def test_an_obedient_narrator_passes_the_checked_turn(patch_loop) -> None:
"""Case 3, first half. A draft that only affirms told-and-checked is not refused.

This only proves the loop still runs end to end on this turn shape — none of the
Validator double's three rules read the closing at all, so this case cannot tell a
working `CLOSING_CHECKED` from a broken one either. Cases 1 and 2 carry that weight.

The mirror half of case 3 — a draft that still asks "Como vocês se sentem?" — is not
written here. `ValidatorReadsOnlyItsOwnPrompt` judges three things read straight out of
its own prompt: a claim about the telling-back with no evidence for it, a destination
the brief never named, and a name the team never said. None of its three rules read the
closing's own instruction not to ask, so it cannot fail a draft on that basis, and a test
asserting a fail-safe there would not be testing this change — it would be testing
against a rule the double does not have. Reported as a gap, not filled with a test that
would look like coverage of it.
"""
outcome, _ = await _checked_turn_with_loop("A passagem foi contada e conferida.", patch_loop)

assert outcome.used_fail_safe is False
assert outcome.speech == "A passagem foi contada e conferida."


def test_insufficient_evidence_keeps_the_plain_closing() -> None:
"""Case 4 (guard). Sem achado mas com evidência insuficiente, o fechamento não muda.

Há próximo turno — a equipe vai contar mais — então a pergunta continua tendo para quem
responder. `state.checked` só fica `True` com achado nenhum *e* evidência suficiente; sem
a segunda metade, o chamador nunca passa `checked=True` para `closing_block`.
"""
assert closing_block(None, checked=False) == CLOSING_PLAIN
assert closing_block(None) == CLOSING_PLAIN
assert CONTINUES_TELLING_BACK in closing_block(None, checked=False)


def test_a_finding_ignores_the_checked_flag() -> None:
"""Case 5 (guard). Com achado, nada muda — os fechamentos de achado continuam os de hoje.

Na rota real `state.checked` só é `True` quando não há achado, mas `checked` é
ignorado sempre que há um: a passagem não pode estar conferida no mesmo turno em que
o narrador está falando sobre algo que o analista encontrou.
"""
finding = Finding(kind=FindingKind.ADDITION, note="Orfa", segment_id="segmento-2")

assert closing_block(finding, checked=True) == closing_block(finding, checked=False)
assert closing_block(finding, checked=True) != CLOSING_CHECKED
assert CONTINUES_TELLING_BACK in closing_block(finding, checked=True)
Loading