Is this related to a problem? Please describe.
The LLM routing demo needs a repeatable way to show request latency, token usage, routing readiness and collection health while an existing model keeps serving.
Describe the solution you'd like
Add an optional monitoring chart and commands to attach it to an existing routing stack. Provision metrics storage and a Grafana dashboard that discovers served models. Verify fresh scrape targets and request, latency and token counter increases using authenticated streaming and nonstreaming requests.
Support cached ARM64 images, retained metrics storage and explicit runtime metric targets.
Describe alternatives you've considered
Operators can configure a separate collector and dashboards manually, but the demo needs reproducible installation and verification commands.
Additional context
The deployment workflow is in #2322. The monitoring implementation is currently reviewed at estroz#4 and will move upstream.
Is this related to a problem? Please describe.
The LLM routing demo needs a repeatable way to show request latency, token usage, routing readiness and collection health while an existing model keeps serving.
Describe the solution you'd like
Add an optional monitoring chart and commands to attach it to an existing routing stack. Provision metrics storage and a Grafana dashboard that discovers served models. Verify fresh scrape targets and request, latency and token counter increases using authenticated streaming and nonstreaming requests.
Support cached ARM64 images, retained metrics storage and explicit runtime metric targets.
Describe alternatives you've considered
Operators can configure a separate collector and dashboards manually, but the demo needs reproducible installation and verification commands.
Additional context
The deployment workflow is in #2322. The monitoring implementation is currently reviewed at estroz#4 and will move upstream.