feat(models): add openaiStream() OpenAI-compatible SSE formatter#1106
Draft
heskew wants to merge 1 commit into
Draft
feat(models): add openaiStream() OpenAI-compatible SSE formatter#1106heskew wants to merge 1 commit into
heskew wants to merge 1 commit into
Conversation
…streaming Wraps an internal generateStream() GenerateChunk iterator into OpenAI chat.completion.chunk SSE messages consumed by Harper's existing text/event-stream serializer, terminated by the [DONE] sentinel. Content deltas stream inline; tool-call deltas are assembled per-id and emitted with arguments stringified once (Harper backends pre-parse args to objects, so incremental fragments would corrupt the OpenAI client's concatenation). Backs #631's /v1/chat/completions streaming leg. Refs #514. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
Contributor
|
Reviewed; no blockers found. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
openaiStream()(resources/models/openaiStream.ts): a formatter that wraps an internalgenerateStream()AsyncIterable<GenerateChunk>into OpenAI-compatiblechat.completion.chunkServer-Sent Events, terminated by the[DONE]sentinel. The yielded{ data }messages pass through Harper's existingtext/event-streamserializer (server/serverHelpers/contentTypes.ts) unchanged — an objectdata→data: {json}\n\n, the sentinel →data: [DONE]\n\n.Why
Closes #514. This is the SSE formatting helper that #631's
/v1/chat/completionsstreaming leg lists as a hard prerequisite. Splitting it out keeps #631 to thin protocol-translation Resources over a tested formatter. Part of #510.Where to look / design decisions
ToolCall(id + name + object-args) exactly once — seecomponents/openai/index.ts:180-221(flushToolCallBufferemits only whenid && name). OpenAI's wire format instead streamsfunction.argumentsas string fragments the client concatenates. So the helper assembles per-id and emits each call's arguments as a singleJSON.stringify'd blob — emitting incremental fragments would corrupt the client's concatenation ({"a":1}+{"b":2}→ invalid JSON). Consequence: tool calls surface in one delta chunk rather than across many; standard SDKs accept this.data:only — no SSEevent:/id:lines (OpenAI's stream is data-only; the completion id lives inside the JSON payload). Verified against the real serializer in the unit tests.finish_reasonfallback.finishReason ?? (toolAssembly.size ? 'tool_calls' : 'stop')covers the backend's tail-flush path (components/openai/index.ts:218-220), where tool calls are flushed with no explicitfinishReason.Testing
7 unit tests (
unitTests/resources/models/openaiStream.test.js): content streaming, the[DONE]sentinel through the real SSE serializer, single + multi tool-call assembly and arg stringification, empty-stream (role + stop), and id stability. Build + lint + format clean.openaiStream()has no standalone runtime surface — a real OpenAI-client smoke needs #631's/v1/chat/completionsendpoint. This stays draft until it can be smoked alongside #631; the unit tests assert the exact wire bytes against Harper's serializer in the meantime.Cross-model review (HEG step 9)
Both outside-model legs clean, no blockers:
agy) — all confirmations/suggestions, no blockers; ran mocha; confirmed OpenAI protocol fidelity, edge-case handling, and TypeStrip compliance.🤖 Generated with Claude Code