Repository navigation
Conversation
Every PassthroughModel route rule had a hard-coded 120s request timeout, which cuts off long or slow generations from self-hosted servers, and the operator overwrote any hand patch on the next reconcile. Add an optional requestTimeout field (default 120s, same validation pattern as LLMModel's) that sets timeouts.request on the declared-model and catch-all rules of both endpoints. DefaultPassthroughRequestTimeout covers objects that never went through API defaulting. Regenerate the CRD, copy it into the chart, and document the field in the configuration reference and the self-hosted guide.
|
📄 Docs preview for |
jbouder
approved these changes
Nov 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #209
Change
PassthroughEndpoints.RequestTimeout(spec.endpoints.requestTimeout), optional, default120s, same Gateway API duration pattern asLLMModel's field.buildPassthroughRouteresolves the field, falling back to the newDefaultPassthroughRequestTimeout(120s) for CRs created before the field existed. The resolved value goes on both the declared-model rule and the catch-all rule, on both endpoints.charts/nebari-llm-serving/crds/passthroughmodel-crd.yaml.spec.endpointsreference, and the self-hosted trade-offs bullet now points at the field.The default stays
120s, as the issue recommends: existing PassthroughModels render identical routes after upgrading, and hosted providers keep a tight bound on hung requests.Verification
TestPassthroughRoutesRequestTimeoutchecks every rule on both routes, unset and set. Mutation check: hard-coding the catch-all rule back to"120s"fails the "set" case onexternal route rule 1andinternal route rule 1.make testpasses.golangci-lintreports nothing in the touched files (main has the same 51 pre-existing findings under the local 2.12.2).Updateof the generated spec (unstructured_apply.go:15-28).Not done: no live gateway test. Plain-Helm users need to apply the updated CRD before they can set the field (see "Upgrade the pack's CRDs with plain Helm"); Argo CD applies it on sync.