Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 67 additions & 18 deletions guides/workflows/deploy-to-your-compute.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -51,36 +51,85 @@ tilebox runner start --cluster workflow-dev --debug

The runner watches its cluster, downloads missing release artifacts, starts the workflow runtime, and advertises the tasks it can execute. Updating a deployment changes what the runner can execute without rebuilding the runner process.

## Bundle the release runner in a container
## Run the official runner container

For cloud or Kubernetes deployments, package the release runner into a small container image. The image only needs Python, `uv`, the Tilebox command-line tool, and any system dependencies your workflow runtime needs. The workflow code itself comes from the deployed workflow release.
Tilebox publishes a ready-to-run release runner at [`ghcr.io/tilebox/runner`](https://github.com/orgs/tilebox/packages/container/package/runner) for Linux amd64 and arm64. The image starts `tilebox runner start` by default and includes the Tilebox CLI, `uv`, Python 3.12 through 3.14, Git, Git LFS, and build dependencies for common scientific and geospatial Python packages. Workflow code arrives through the releases deployed to the selected cluster, so you do not rebuild the image when a workflow changes.

```dockerfile Dockerfile
FROM python:3.13-slim
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
Export an API key, then start the runner for your cluster.

# Install system dependencies.
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates curl git git-lfs openssh-client \
&& rm -rf /var/lib/apt/lists/* \
&& apt-get clean
```bash
export TILEBOX_API_KEY="<API_KEY>"

docker run --rm \
--env TILEBOX_API_KEY \
--env TILEBOX_CLUSTER=workflow-dev \
ghcr.io/tilebox/runner:0.5.0
```

`TILEBOX_API_KEY` is required. `TILEBOX_CLUSTER` is optional; when omitted, the runner uses your default cluster. Provide credentials through your deployment system instead of including them in the image.

RUN curl -fsSL https://install.tilebox.com/cli.sh | TILEBOX_INSTALL_DIR=/usr/local/bin TILEBOX_NO_INSTALL_COMPLETIONS=1 sh
Use the official image directly when its runtime matches your workflow. Build a custom image from it when your code needs more operating system packages. The image does not include the NVIDIA CUDA toolkit, so CUDA extensions require a version-matched NVIDIA development image and GPU runtime.

# Required at runtime: set TILEBOX_CLUSTER to a valid cluster slug and
# TILEBOX_API_KEY to an API key that can read deployments and claim tasks.
ENV TILEBOX_CLUSTER=""
ENV TILEBOX_API_KEY=""
## Deploy the runner on Kubernetes

Run the image as a Kubernetes `Deployment` so the platform restarts the runner and lets you scale the number of processes. Store the API key in a `Secret`.

```bash
kubectl create secret generic tilebox-runner \
--from-literal=api-key="$TILEBOX_API_KEY"
```

CMD ["tilebox", "runner", "start"]
Save this manifest as `runner-deployment.yaml`.

```yaml runner-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: tilebox-runner
spec:
replicas: 1
selector:
matchLabels:
app: tilebox-runner
template:
metadata:
labels:
app: tilebox-runner
spec:
containers:
- name: runner
image: ghcr.io/tilebox/runner:0.5.0
env:
- name: TILEBOX_API_KEY
valueFrom:
secretKeyRef:
name: tilebox-runner
key: api-key
- name: TILEBOX_CLUSTER
value: workflow-dev
```

Build and publish this image with your normal container workflow. At runtime, provide `TILEBOX_CLUSTER` and `TILEBOX_API_KEY` through your deployment system rather than baking secrets into the image.
Apply the manifest.

In Kubernetes, run the image as a `Deployment` and store `TILEBOX_API_KEY` in a `Secret`. Set `TILEBOX_CLUSTER` through the pod environment and scale replicas to increase runner concurrency. In Google Cloud, run the same image on Cloud Run jobs, GKE, or Compute Engine depending on your workload constraints. In AWS, run it on ECS, EKS, or EC2 and inject the API key through your secret manager or task definition.
```bash
kubectl apply -f runner-deployment.yaml
```

The same image can run as a long-lived process on container or virtual machine services such as Amazon ECS, Amazon EKS, Amazon EC2, Google Kubernetes Engine, or Google Compute Engine. Inject `TILEBOX_API_KEY` with the platform's secret manager and set `TILEBOX_CLUSTER` in the container environment.

For a job-based container service that expects the process to exit, such as Cloud Run jobs, replace the default command with `tilebox runner start --stop-when-idling`. The runner processes available work, then exits when it becomes idle.

```bash
docker run --rm \
--env TILEBOX_API_KEY \
--env TILEBOX_CLUSTER=workflow-dev \
ghcr.io/tilebox/runner:0.5.0 \
tilebox runner start --stop-when-idling
```

## Scale runner processes

Scale the number of runner containers or virtual machine instances when you want more parallelism. In Kubernetes, increase the `Deployment` replica count. In GCP or AWS, use the scaling controls of the service that runs the container, such as Cloud Run, GKE, ECS, EKS, or an auto-scaling VM group. Each runner process connects to the same cluster and claims compatible tasks independently.
Scale the number of runner containers or virtual machine instances when you want more parallelism. In Kubernetes, increase the `Deployment` replica count. In GCP or AWS, use the scaling controls of the service that runs the container, such as GKE, ECS, EKS, or an auto-scaling VM group. Each runner process connects to the same cluster and claims compatible tasks independently.

As an alternative for local testing or constrained environments, you can run multiple runner processes inside one container or shell session. Use `tilebox parallel` only for that case.

Expand Down
4 changes: 4 additions & 0 deletions workflows/concepts/runners.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ The release runner can run releases from multiple workflows at the same time, bu
Release runners currently only support Python workflow projects. The Tilebox CLI invokes the Python runner environment from the published release artifact using `uv`.
</Info>

<Tip>
Tilebox publishes a ready-to-run release runner container at `ghcr.io/tilebox/runner`. See [Deploy to your compute](/guides/workflows/deploy-to-your-compute) for Docker and Kubernetes examples.
</Tip>

### Direct runners

A direct runner connects to the Tilebox API from your own code. Use it when you want full control over the process, deployment environment, dependencies, startup behavior, and scaling. You are responsible for deploying the script or binary, keeping it running, rolling out code changes, and rolling back when needed.
Expand Down