Skip to content

llm-routing: runtime guard exits after start when the cgroup swap file is missing #2334

Description

@kristinapathak

Describe the bug

charts/gguf-backend/files/runtime_guard.py (deploy/helm/llm-routing/spark; #2331 moves it under recipes/ without changing it) reads /sys/fs/cgroup/memory.swap.current in cgroup_memory(), but only inside the supervision loop, after it has started the runtime with Popen. On a host without cgroup v2 swap accounting, that file does not exist. FileNotFoundError then ends the guard, and its container, right after the RPC or model server starts.

Steps or code to reproduce bug

Run a runtime pod on a node where /sys/fs/cgroup/memory.swap.current is absent inside the container, for example with swap accounting disabled.

Expected behavior

Check that memory.current, memory.swap.current and memory.events exist before starting the runtime, and refuse to start with a clear message. Alternatively, treat a missing swap file as zero swap usage if that is safe.

Additional context

Introduced in #2322. Found during review of #2331.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingllm-stackLLM Gateway + Stargate Request Router + Vanity Gatewayneeds-triageIssue or PR awaiting maintainer triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions