Skip to content

Add scoped QR-hosted FableLoom sessions with half-duplex protagonist voice #5383

Description

@atomantic

Part of #5377. Blocked by #5382 and #5380. #5381 is an optional enhancement, not a blocker.

Goal

Add same-tailnet/same-LAN two-device hosted play: the computer runs authored pre-rendered video, a phone joins by QR code as the microphone/audience UI, and the off-screen protagonist answers in the approved character voice with no overlapping speech.

Proposed approach

  • Add explicit host-session creation with HTTPS/readiness preflight, short-lived high-entropy join tokens stored hashed, and a fragment-based QR URL.
  • Authenticate a dedicated hosted Socket.IO namespace; a guest token can access only one session and never becomes a general PortOS login.
  • Keep server story/session state authoritative, computer playback clock authoritative, and phone limited to microphone, transcript, and turn state.
  • Reuse local Whisper, character voice resolution, streaming/abort patterns, and a bounded hosted-character play-turn contract.
  • Implement LISTENING -> THINKING -> SPEAKING -> LISTENING, no barge-in, measured echo tail, one configured TTS output target, and runtime revalidation of the live-conversation gate.
  • Add reconnect snapshots, expiry/revocation, teardown, authored fallback lines, and machine-local/ephemeral transcript policy.

Acceptance criteria

  • No QR is offered unless the join URL is HTTPS and host/STT/LLM/TTS/playback readiness is green.
  • The token is scoped, short-lived, single-audience by default, absent from request logs/referrers, and rejected outside its hosted namespace/session.
  • The microphone accepts frames only in LISTENING; rendered character dialogue and live character TTS never overlap viewer input.
  • Live conversation is allowed only for a connected helper decision/hold node with an off-screen protagonist and a safe active hold asset.
  • Final transcript resolution and protagonist speech complete before a story transition is committed.
  • Host/phone reconnect restores one authoritative node, phase, and transcript without replaying a partial turn.
  • Session end aborts active STT/LLM/TTS work and revokes access.

Out of scope

  • Public internet relay, WebRTC, multiple audience devices, or real-time video.
  • FaceTime transport.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

architectureArchitecture and SOLIDarea:privacyPII vault, trusted-org registry, data-broker opt-outarea:story-builderStory Builder featurearea:voiceVoice stack: STT/TTS pipeline, proactive speech, voice tools, call bridgeeffort:maxDispatch reasoning effort: maximummodel:heavyDispatch capability: heavyplanTracked by /do:replanplan-featureFeature plan filed by the plan-feature brainstorm

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions