Skip to content

fix(example): use native dataset split fields - #230

Open
Gautam Sharma (GautamSharma99) wants to merge 1 commit into
langchain-ai:mainfrom
GautamSharma99:fix/example-native-splits-222
Open

fix(example): use native dataset split fields#230
Gautam Sharma (GautamSharma99) wants to merge 1 commit into
langchain-ai:mainfrom
GautamSharma99:fix/example-native-splits-222

Conversation

@GautamSharma99

Copy link
Copy Markdown
Contributor

Summary

  • send example create --split through ExampleNewParams.Split
  • send example list --split through ExampleListParams.Splits so filtering happens server-side before pagination
  • render native split membership from the API response instead of treating metadata["split"] as special
  • expose native membership as split in structured list output
  • preserve user metadata unchanged, including an independent metadata key named split

Closes #222.

Problem

The CLI previously modeled dataset split membership as ordinary example metadata:

  • create inserted --split into metadata["split"]
  • list fetched unfiltered pages and then compared metadata["split"] locally
  • the pretty table displayed the metadata value

LangSmith has first-class dataset split fields, and the generated Go SDK already exposes them on both create and list request types. Metadata with the same key is not equivalent to native split membership.

Besides producing examples that are not members of the requested LangSmith split, client-side filtering happened after pagination. A limited page could therefore appear empty even when matching examples existed on later server pages.

Implementation

Create

When --split is present, the command now sets:

ExampleNewParams.Split = shared.UnionString(split)

Metadata is passed through exactly as supplied. In particular, metadata such as {"split":"source-label"} is no longer overwritten by --split test.

List and pagination

When --split test is present, the command now sets:

ExampleListParams.Splits = []string{"test"}

The server applies the split constraint before offset pagination, so --limit and --offset operate on the matching result set. The old post-fetch metadata filtering loop is removed.

Output

The API returns native membership in a split response property, which can be either a string or a list. The current generated Go Example response struct does not expose that property as a typed field, but it retains the original response JSON. The CLI reads only this retained field for presentation; API access still goes entirely through the generated SDK.

  • pretty output displays a single split or comma-separated split list
  • JSON output includes an additive split property
  • missing or malformed membership displays as N/A in the table and null in JSON
  • metadata remains available independently under metadata

Behavior change

Before, this command:

langsmith example create \
  --dataset evaluations \
  --inputs '{"question":"hello"}' \
  --metadata '{"split":"source-label"}' \
  --split test

sent metadata equivalent to:

{"split":"test"}

It now sends native membership and preserves metadata:

{
  "split": "test",
  "metadata": {"split": "source-label"}
}

Tests

Added request-level tests with an HTTP test server that verify:

  • list sends splits=test on the SDK query, including paginated requests
  • list returns the API's native multi-split membership
  • metadata named split is preserved but not used as membership
  • create sends the native split body property
  • create does not overwrite or discard metadata
  • pretty formatting handles string, list, missing, and invalid values

Local verification:

  • go test ./internal/cmd -run 'TestExample(List_UsesNativeSplitFilterAndOutput|Create_UsesNativeSplitWithoutChangingMetadata)|TestFormatExampleSplit'
  • go test ./...
  • go test -race ./...
  • go vet ./...
  • make build
  • git diff --check

make lint was not available locally because golangci-lint is not installed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

example --split uses metadata instead of LangSmith split fields

1 participant