Skip to content

Add a container image and a multi-arch publish workflow - #20

Open
joh-klein wants to merge 1 commit into
nxcite:masterfrom
meddevo:docker-image
Open

joh-klein wants to merge 1 commit into
nxcite:masterfrom
meddevo:docker-image

Conversation

@joh-klein

Copy link
Copy Markdown
Contributor

Addresses the Docker half of #16.

What's here

  • Dockerfilerust:1-bookworm builder (not -slim: ring still needs a C compiler) into gcr.io/distroless/cc-debian12, which gives glibc plus the CA roots rustls needs to trust S3, and no shell. Cargo registry and target dir are BuildKit cache mounts; the target mount is keyed on TARGETPLATFORM so a single-builder multi-arch build can't have the two cargos racing on the same target/release/nx-cache-aws path.
  • .github/workflows/docker.yml — publishes to ghcr.io/${{ github.repository }}.
  • .dockerignore
  • README.md — a docker run example, the tag scheme, and a plain Deployment/Service for Kubernetes.

Why a matrix instead of one multi-platform build

build (matrix)   amd64 → ubuntu-24.04      ─┐  push-by-digest, untagged
                 arm64 → ubuntu-24.04-arm  ─┘
manifest         imagetools create → one multi-arch tag

Building both platforms in one job puts arm64 through QEMU, which is roughly an order of magnitude slower on a release build. Hosted arm runners are free on public repos, so each half compiles natively. Each build job pushes an untagged digest-only image and hands the digest to the manifest job via an artifact; that job stitches both digests into one tag. This is the pattern from the Docker docs for distributing a build across runners.

Details worth knowing:

  • PRs build but don't push (outputs: type=cacheonly), so PRs from forks work without registry credentials.
  • GHCR, not Docker Hub. GITHUB_TOKEN already has packages: write, so this works the moment it merges with nothing for you to configure. Docker Hub needs a repo created plus DOCKERHUB_USERNAME/DOCKERHUB_TOKEN secrets — happy to add that login step if you'd rather publish there, or in addition.
  • latest tracks the newest release, not the tip of master (metadata-action's latest=auto). master pushes publish master and sha-<short-sha>.
  • The concurrency group is keyed on the SHA: a release push updates both master and the vX.Y.Z tag, and this collapses the two runs into one build that publishes X.Y.Z.

No Helm chart

Deliberately left out. The server is one stateless container with no volumes, no sidecars, and about five environment variables — a chart would add a values schema, chart versioning, a chart repo to host, and CI to publish it, for something kubectl apply already does. The README section covers it in ~35 lines, including the note that on EKS you drop the AWS credential variables and attach an IAM role to the ServiceAccount. Glad to reconsider if you disagree.

Verified

  • docker build --platform linux/arm64 . succeeds on this tree (Apple Silicon).
  • actionlint clean on the workflow.
  • The workflow itself hasn't run against your repo yet — it needs your master, and it should go green on this PR (build-only, no push).

🤖 Generated with Claude Code

Refs nxcite#16 (the Docker half).

- Dockerfile: rust:1-bookworm builder (ring still wants a C compiler) into
  gcr.io/distroless/cc-debian12:nonroot — glibc plus the CA roots
  rustls-native-certs reads for S3, no shell, uid 65532.
- .github/workflows/docker.yml: one native runner per architecture (amd64 on
  ubuntu-24.04, arm64 on ubuntu-24.04-arm, both free on public repos) rather
  than one job emulating arm64 under QEMU. Each pushes a digest-only image;
  a final job stitches the digests into one multi-arch tag. PRs build without
  pushing, so fork PRs work. Publishes to ghcr.io/${{ github.repository }} —
  no registry secrets to configure.
- release.yml calls docker.yml after cutting the release. A `push: tags:`
  trigger would not work: the tag is created with `gh release create` using
  GITHUB_TOKEN, and GITHUB_TOKEN events do not start workflow runs.
- README: docker run example, tag scheme, and a plain Deployment/Service for
  Kubernetes.
@joh-klein

Copy link
Copy Markdown
Contributor Author

Force-pushed a revision after a review pass. Four fixes, one of which killed half the original PR:

1. Releases would never have published an image. The first version triggered on push: tags: ["v*"]. That never fires here: release.yml is workflow_dispatch-only and the tag is created by gh release create running with GITHUB_TOKEN, and events raised by GITHUB_TOKEN do not start new workflow runs. So type=semver would never have matched and :latest / :X.Y.Z would never have existed — only :master and :sha-…. (on: release: [published] wouldn't fix it either, for the same reason.) Fixed by making docker.yml a workflow_call workflow and adding an image job to release.yml that runs after create-tag-and-release and passes needs.validate.outputs.version. This is the one change that touches your existing release pipeline — 13 lines, no change to what it already does.

2. Concurrency was keyed wrong. group: docker-${{ github.sha }} puts every push in its own group, so two pushes to master in quick succession run in parallel and the older one can finish last, moving :master backwards. Now keyed on github.ref, with cancel-in-progress limited to pull requests — a superseded PR build is waste, but cancelling a release build would ship a release with no image.

3. Runs as non-root. gcr.io/distroless/cc-debian12:nonroot, uid 65532. Nothing else had to change: the server binds 3000 and writes nothing to disk.

4. README was wrong about credentials. Neither the docker run example nor the Deployment passed any, so a copy-paste run starts cleanly and then 500s on every request. Both now say what to pass and when you can omit it (ECS/EKS roles).

Also dropped ENV RUST_LOG=infotracing-subscriber is built without env-filter, so RUST_LOG is never read; the default level is already INFO.

Verified locally on this branch: actionlint clean on both workflows; docker build --platform linux/arm64 succeeds; the resulting image runs as uid 65532 and answers /health with OK, logging at INFO without RUST_LOG set.

One thing that needs a maintainer, not a commit: the first push creates the GHCR package as private, so anonymous docker pull ghcr.io/nxcite/nx-cache-server will 401 until someone flips the package to public in the repo's package settings.

Still no Helm chart, for the reasons in the description — happy to be talked out of that.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant