Skip to content

feat(models): add openaiStream() OpenAI-compatible SSE formatter#1106

Draft
heskew wants to merge 1 commit into
mainfrom
feat/models-openai-stream
Draft

feat(models): add openaiStream() OpenAI-compatible SSE formatter#1106
heskew wants to merge 1 commit into
mainfrom
feat/models-openai-stream

Conversation

@heskew

@heskew heskew commented Jun 2, 2026

Copy link
Copy Markdown
Member

Summary

Adds openaiStream() (resources/models/openaiStream.ts): a formatter that wraps an internal generateStream() AsyncIterable<GenerateChunk> into OpenAI-compatible chat.completion.chunk Server-Sent Events, terminated by the [DONE] sentinel. The yielded { data } messages pass through Harper's existing text/event-stream serializer (server/serverHelpers/contentTypes.ts) unchanged — an object datadata: {json}\n\n, the sentinel → data: [DONE]\n\n.

Why

Closes #514. This is the SSE formatting helper that #631's /v1/chat/completions streaming leg lists as a hard prerequisite. Splitting it out keeps #631 to thin protocol-translation Resources over a tested formatter. Part of #510.

Where to look / design decisions

  • Tool-call mapping (the subtle bit). Harper backends pre-parse tool-call arguments into objects and yield each complete ToolCall (id + name + object-args) exactly once — see components/openai/index.ts:180-221 (flushToolCallBuffer emits only when id && name). OpenAI's wire format instead streams function.arguments as string fragments the client concatenates. So the helper assembles per-id and emits each call's arguments as a single JSON.stringify'd blob — emitting incremental fragments would corrupt the client's concatenation ({"a":1} + {"b":2} → invalid JSON). Consequence: tool calls surface in one delta chunk rather than across many; standard SDKs accept this.
  • SSE shape. Emits data: only — no SSE event:/id: lines (OpenAI's stream is data-only; the completion id lives inside the JSON payload). Verified against the real serializer in the unit tests.
  • finish_reason fallback. finishReason ?? (toolAssembly.size ? 'tool_calls' : 'stop') covers the backend's tail-flush path (components/openai/index.ts:218-220), where tool calls are flushed with no explicit finishReason.

Testing

7 unit tests (unitTests/resources/models/openaiStream.test.js): content streaming, the [DONE] sentinel through the real SSE serializer, single + multi tool-call assembly and arg stringification, empty-stream (role + stop), and id stability. Build + lint + format clean.

⚠️ Not yet smoked end-to-end. openaiStream() has no standalone runtime surface — a real OpenAI-client smoke needs #631's /v1/chat/completions endpoint. This stays draft until it can be smoked alongside #631; the unit tests assert the exact wire bytes against Harper's serializer in the meantime.

Cross-model review (HEG step 9)

Both outside-model legs clean, no blockers:

  • Codex — no actionable correctness issues; ran the build + targeted test itself.
  • Gemini (agy) — all confirmations/suggestions, no blockers; ran mocha; confirmed OpenAI protocol fidelity, edge-case handling, and TypeStrip compliance.

🤖 Generated with Claude Code

…streaming

Wraps an internal generateStream() GenerateChunk iterator into OpenAI
chat.completion.chunk SSE messages consumed by Harper's existing
text/event-stream serializer, terminated by the [DONE] sentinel. Content
deltas stream inline; tool-call deltas are assembled per-id and emitted with
arguments stringified once (Harper backends pre-parse args to objects, so
incremental fragments would corrupt the OpenAI client's concatenation).

Backs #631's /v1/chat/completions streaming leg. Refs #514.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@heskew heskew requested a review from kriszyp June 2, 2026 21:22
@gemini-code-assist

Copy link
Copy Markdown

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@claude

claude Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add openaiStream() formatter helper for OpenAI-compatible SSE streaming

1 participant