FlowLLM: Build LLM applications with ease.

FlowLLM is a configuration-driven LLM application framework that organizes workflows, service entrypoints, and long-lived components with Service, Job, Step, and Component.

✨ Core Features

• Configuration-driven: Starts from flowllm/config/default.yaml, with config files and dot-notation overrides.
• Unified path: CLI / Client -> Service -> Application -> Job -> Step -> Component.
• Flexible Jobs: Supports sync, streaming, background, and scheduled tasks exposed through HTTP or MCP.
• Pluggable components: Extend Steps, Services, Clients, LLMs, Embeddings, Embedding Stores, and Agent Wrappers through registry R.

🧠 FlowLLM Development Skill

FlowLLM ships with a development Skill for coding agents. It explains the framework conventions, repository map, extension points, testing workflow, and review checklist for building or extending FlowLLM applications.

Start here when changing the codebase: FlowLLM Development Skill.

📝 Technical Notes

Note	Description
Context Management Design	Short-term agent context management design, including history compaction and tool-result truncation.
SELFCOMPACT and AgentScope PR #1938	Technical reading notes for the SELFCOMPACT paper, experiments, and the AgentScope implementation PR.

🚀 Quick Start

Installation

FlowLLM requires Python 3.11+.

Install from pip:

pip install flowllm

Install from source:

git clone https://github.com/flowllm-ai/flowllm.git
cd flowllm
pip install -e .

Start the Service

flowllm start

The default service address is 127.0.0.1:2333, and the default workspace is .flowllm/. Override configuration with dot notation:

flowllm start service.port=8181 enable_logo=false
flowllm start workspace_dir=/tmp/flowllm-demo service.host=127.0.0.1 service.port=8181

See the Quick Start for more startup and invocation examples.

🧩 Use FlowLLM

flowllm version
flowllm health_check
flowllm help
flowllm demo query="Hello FlowLLM" min_score=0.8
flowllm add a=1 b=2

CLI commands other than start call server-side Jobs with the same name. HTTP uses POST /<job_name>:

curl -s http://127.0.0.1:2333/add \
  -H 'Content-Type: application/json' \
  -d '{"a":1,"b":2}'

See Quick Start for service, CLI, HTTP, and streaming examples.

⚙️ Build Applications

FlowLLM applications are configured from flowllm/config/default.yaml or your own YAML/JSON config. Add a capability by registering a Step or Component, then exposing it as a Job:

jobs:
  reverse:
    backend: base
    description: "reverse text"
    steps:
      - backend: reverse_step

Configuration supports .env, ${VAR}, ${VAR:-default}, dot-notation overrides, and direct config files:

flowllm start config=/path/to/app.yaml

For implementation rules and examples, use the FlowLLM Development Skill. For the compact architecture reference, see Code Framework.

🆕 Minimal CLI Flow

For scripts that do not need the full service framework, flowllm.lite maps fl --action --field value to a Pydantic config and a small ordered BaseFlow. See FlowLLM Lite.

🗺️ Roadmap

Item	Description
Agent wrapper integration	Integrate FlowLLM into Agent wrappers so agents can develop FlowLLM Steps and Jobs from your ideas.
TypeScript frontend	Add a TypeScript frontend for building and developing FlowLLM applications through the UI.

📚 Documentation

• Quick Start
• Code Framework
• Contributing
• FlowLLM Development Skill

🤝 Open Source and Contributing

FlowLLM is licensed under Apache 2.0. Before contributing, read the contribution guide and development skill.

Install development dependencies from source:

pip install -e ".[dev]"

Before submitting changes, run:

pre-commit run --all-files
pytest

📄 License

This project is open source under the Apache License 2.0. See LICENSE for details.

⭐ Star History