Skip to content

feat: Add ChatCompletionsHTTPClient and support for non-streaming requests#1178

Open
copybara-service[bot] wants to merge 1 commit intomainfrom
test_902749082
Open

feat: Add ChatCompletionsHTTPClient and support for non-streaming requests#1178
copybara-service[bot] wants to merge 1 commit intomainfrom
test_902749082

Conversation

@copybara-service
Copy link
Copy Markdown

feat: Add ChatCompletionsHTTPClient and support for non-streaming requests

This is part of a larger chain of commits for adding chat completion API support to the Apigee model.

The HTTP client wraps payload construction (delegating to ChatCompletionsRequest.fromLlmRequest) and response parsing (delegating to ChatCompletionsResponse.ChatCompletion / ChatCompletionChunkCollection) for both non-streaming and streaming Server-Sent Events responses.
END_PUBLIC

Key behaviors:

  • Tri-state call timeout policy:

    • httpOptions == null OR timeout() empty: applies a default 5-minute call
      timeout to prevent indefinite hangs in the common unconfigured case.
    • httpOptions.timeout() == 0: respected as the explicit caller opt-in to
      infinite hang for long-running streams or batch jobs.
    • httpOptions.timeout() > 0: applied directly as the call timeout.
      This default intentionally diverges from the GenAI HttpOptions convention
      (which treats unset as infinite) as a defensive measure since this client
      does not yet have HTTP retry support.
  • SSE prefix handling accepts both "data: foo" (with space) and "data:foo"
    (without space) per the SSE spec, matching providers that omit the
    trailing space.

  • A single malformed JSON chunk in a streaming response is logged and
    skipped rather than aborting the entire stream. IOException
    (connection-level) still propagates as a stream error.

  • Content-Type is defensively forced to application/json by replacing
    rather than appending, preventing duplicate or conflicting headers if a
    caller supplies their own Content-Type.

  • Headers parameter accepts null (treated as no extra headers) and is
    stored as an ImmutableMap for thread-safe reuse across concurrent
    generateContent calls.

Test additions (16 total, +12 new):

  • HTTP error status (4xx/5xx) propagation for both streaming and
    non-streaming.
  • Empty body propagation.
  • Streaming continues past a single malformed chunk.
  • SSE "data:" prefix accepted with or without trailing space.
  • Custom headers reach the wire.
  • Caller-supplied Content-Type is overridden, not appended.
  • baseUrl with and without trailing slash.
  • Constructor tri-state timeout (null, zero=infinite, positive).
  • Constructor null headers parameter.
    All testSubscriber.await() calls bounded to 500ms to prevent test hangs.

…uests

This is part of a larger chain of commits for adding chat completion API support to the Apigee model.

The HTTP client wraps payload construction (delegating to ChatCompletionsRequest.fromLlmRequest) and response parsing (delegating to ChatCompletionsResponse.ChatCompletion / ChatCompletionChunkCollection) for both non-streaming and streaming Server-Sent Events responses.
END_PUBLIC

Key behaviors:

- Tri-state call timeout policy:
  * httpOptions == null OR timeout() empty: applies a default 5-minute call
    timeout to prevent indefinite hangs in the common unconfigured case.
  * httpOptions.timeout() == 0: respected as the explicit caller opt-in to
    infinite hang for long-running streams or batch jobs.
  * httpOptions.timeout() > 0: applied directly as the call timeout.
  This default intentionally diverges from the GenAI HttpOptions convention
  (which treats unset as infinite) as a defensive measure since this client
  does not yet have HTTP retry support.

- SSE prefix handling accepts both "data: foo" (with space) and "data:foo"
  (without space) per the SSE spec, matching providers that omit the
  trailing space.

- A single malformed JSON chunk in a streaming response is logged and
  skipped rather than aborting the entire stream. IOException
  (connection-level) still propagates as a stream error.

- Content-Type is defensively forced to application/json by replacing
  rather than appending, preventing duplicate or conflicting headers if a
  caller supplies their own Content-Type.

- Headers parameter accepts null (treated as no extra headers) and is
  stored as an ImmutableMap for thread-safe reuse across concurrent
  generateContent calls.

Test additions (16 total, +12 new):
- HTTP error status (4xx/5xx) propagation for both streaming and
  non-streaming.
- Empty body propagation.
- Streaming continues past a single malformed chunk.
- SSE "data:" prefix accepted with or without trailing space.
- Custom headers reach the wire.
- Caller-supplied Content-Type is overridden, not appended.
- baseUrl with and without trailing slash.
- Constructor tri-state timeout (null, zero=infinite, positive).
- Constructor null headers parameter.
All testSubscriber.await() calls bounded to 500ms to prevent test hangs.

PiperOrigin-RevId: 902749082
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant