| title | Introduction |
|---|---|
| description | What llm-helper is, and when to reach for it. |
llm-helper serves large models with vLLM, mlx-lm or llama.cpp on one GPU machine or several, and runs one model or several at once. It wraps an OpenAI-compatible inference server, vLLM by default, mlx-lm on Apple Silicon or llama.cpp for GGUF models, with saved model profiles, HuggingFace search, tested multi-machine recipes, and an optional Coder Agents sync. It was called vllm-helper until it served more than vLLM, and the old name still works.
It was built for unified-memory machines like the NVIDIA DGX Spark, where one machine holds one big model and a bigger model needs two or more working together. It works on any Linux machine with an NVIDIA GPU, and on Apple Silicon Macs on the GPU through vllm-metal.
- A profile is a small env file holding the settings that serve one model, for the provider the profile names.
- An instance is one running model: a profile served by its own service unit, a systemd unit on Linux or a launchd job on macOS.
- The CLI starts and stops instances, pulls models, and diagnoses the machine.
- The web panel is what
llm-helper serveserves: the same controls in a browser. - The endpoint is the inference server's own OpenAI-compatible API, one per instance, on port 8001 and up.
Inference traffic goes straight to the inference server on its own port. llm-helper is a control plane, so nothing here sits in the path of a request — a bug in it cannot break inference. The exceptions are opt-in: the model router, which gives several instances one address, and a pool, whose router sits in front of the replicas by design.
The binary is self-contained: one file to put on a machine, with the docs you are reading compiled in.
- Follow the Quickstart to set the machine up and serve a first model.
- See what a profile holds in Profiles.
- Run several models side by side with Several models at once.
- Serve with mlx-lm or llama.cpp instead of vLLM with Providers.
- Serve a model measured across several machines with Recipes.
- Pool replicas or shard one model across machines with More than one machine.
- Browse the CLI reference for every command.