[integrations][openai] Apply OpenAI native structured output#919
Open
weiqingy wants to merge 2 commits into
Open
[integrations][openai] Apply OpenAI native structured output#919weiqingy wants to merge 2 commits into
weiqingy wants to merge 2 commits into
Conversation
Carry the output schema to chat model connections as an explicit parameter and split the structured output strategy into a user policy and a per-model capability, so a later change can apply a provider's native structured output without threading the schema through the provider-facing model parameters. Java adds a concrete four argument chat overload whose default body ignores the schema and delegates to the existing abstract three argument chat, so connections with no native mechanism need no change. Python adds an explicit output_schema parameter to every connection, including the Java bridge connection, so the schema binds to a named parameter and cannot be forwarded into a provider request through kwargs. StructuredOutputStrategy expresses user intent (AUTO, NATIVE or PROMPT) on the setup and defaults to AUTO. Capability is separate: a connection side predicate over the effective model, since a provider supports its native mechanism only on some models. The base predicate returns false; a connection that adds a native translation overrides it. This is the mechanism only. No connection applies native structured output and no path passes a schema on the chat call yet, so behavior is unchanged.
Override the schema-carrying chat on the OpenAI completions connection to request the provider's native structured output when the caller supplies a schema the provider can honor natively. Native applies only when the schema is a POJO class (Java) or a BaseModel subclass (Python) and the effective model supports it. Capability is a connection-side predicate over the effective model, since the OpenAI json schema mode is model dependent: per the provider's structured output guide it is supported on gpt-4o-mini and the gpt-4o snapshots from 2024-08-06 onward, while gpt-4-turbo and gpt-3.5-turbo have json mode only. The predicate matches gpt-4o-mini by prefix (the whole family postdates the cutoff) and the capable gpt-4o snapshots exactly, so a pre-cutoff snapshot such as gpt-4o-2024-05-13 correctly falls back to the prompt path rather than failing at the provider. A RowTypeInfo schema stays on the prompt path. No path passes a schema on the chat call yet, so behavior is unchanged; native output is exercised by direct callers and the unit tests.
github-actions
Bot
added
doc-not-needed
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: #280
Stacked on #843 (the structured-output foundation). Until #843 merges, this PR's diff includes the foundation commit; it will rebase to a single commit once #843 lands. Please review #843 first.
Purpose of change
Applies OpenAI's native structured output on top of the foundation in #843.
The OpenAI completions connection overrides the schema-carrying
chatand requestsresponse_formatjson_schema strict when native applies, and declares the capability predicate over the effective model.Native applies only when the schema is a POJO class (Java) or a
BaseModelsubclass (Python) and the effective model supports it. Per OpenAI's structured-output guide, json_schema is supported ongpt-4o-miniand thegpt-4osnapshots from2024-08-06onward, whilegpt-4-turboandgpt-3.5-turbohave JSON mode only. Because "and later" is a snapshot cutoff rather than a name prefix, the predicate matchesgpt-4o-miniby prefix (its whole family post-dates the cutoff) and the capablegpt-4osnapshots exactly, so a pre-cutoff snapshot such asgpt-4o-2024-05-13falls back to the prompt path instead of failing at the provider. ARowTypeInfoschema stays on the prompt path.No path passes a schema on the chat call yet, so behavior is unchanged; native output is exercised by direct callers and the unit tests.
Tests
Java: the request carries
response_formatjson_schema strict when a capable model and a POJO schema are supplied; the capability predicate is model-dependent (an incapable and a pre-cutoff same-family snapshot both resolve to not-capable); null schema and non-POJO schema keep the prompt path.Python: the same coverage over the
BaseModelpath, including the pre-cutoff snapshot regression test and policy behavior.Java OpenAI module tests and the Python suite pass; lint and license checks are clean.
API
Touches the OpenAI connection: overrides the schema-carrying
chatand declares the capability predicate. No new user-facing surface beyond the foundation in #843.Documentation
doc-neededdoc-not-neededdoc-included