fix(example): use native dataset split fields - #230
Open
Gautam Sharma (GautamSharma99) wants to merge 1 commit into
Open
fix(example): use native dataset split fields#230Gautam Sharma (GautamSharma99) wants to merge 1 commit into
Gautam Sharma (GautamSharma99) wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
example create --splitthroughExampleNewParams.Splitexample list --splitthroughExampleListParams.Splitsso filtering happens server-side before paginationmetadata["split"]as specialsplitin structured list outputsplitCloses #222.
Problem
The CLI previously modeled dataset split membership as ordinary example metadata:
--splitintometadata["split"]metadata["split"]locallyLangSmith has first-class dataset split fields, and the generated Go SDK already exposes them on both create and list request types. Metadata with the same key is not equivalent to native split membership.
Besides producing examples that are not members of the requested LangSmith split, client-side filtering happened after pagination. A limited page could therefore appear empty even when matching examples existed on later server pages.
Implementation
Create
When
--splitis present, the command now sets:Metadata is passed through exactly as supplied. In particular, metadata such as
{"split":"source-label"}is no longer overwritten by--split test.List and pagination
When
--split testis present, the command now sets:The server applies the split constraint before offset pagination, so
--limitand--offsetoperate on the matching result set. The old post-fetch metadata filtering loop is removed.Output
The API returns native membership in a
splitresponse property, which can be either a string or a list. The current generated GoExampleresponse struct does not expose that property as a typed field, but it retains the original response JSON. The CLI reads only this retained field for presentation; API access still goes entirely through the generated SDK.splitpropertyN/Ain the table andnullin JSONmetadataBehavior change
Before, this command:
sent metadata equivalent to:
{"split":"test"}It now sends native membership and preserves metadata:
{ "split": "test", "metadata": {"split": "source-label"} }Tests
Added request-level tests with an HTTP test server that verify:
splits=teston the SDK query, including paginated requestssplitis preserved but not used as membershipsplitbody propertyLocal verification:
go test ./internal/cmd -run 'TestExample(List_UsesNativeSplitFilterAndOutput|Create_UsesNativeSplitWithoutChangingMetadata)|TestFormatExampleSplit'go test ./...go test -race ./...go vet ./...make buildgit diff --checkmake lintwas not available locally becausegolangci-lintis not installed.