build: cache native release compilation

[djwhitt]

Summary

• compile each Rust NIF with cargo rustc before copying ordinary application source
• keep the architecture-specific Cargo target in a reusable Docker layer keyed only by Cargo and native inputs
• make the PR Docker check build the production Dockerfile rather than the alternate multi-step file
• update the touched Docker workflows to current checkout, Buildx, login, and build-push action majors

Motivation

The production Dockerfile currently compiles all native crates inside mix release after COPY . ./. Any Elixir source change therefore discards the native compilation cache.

The Docker Build Check previously built Dockerfile.multi-step, while production builds use the default Dockerfile. Because this PR changes the native compilation layers in the production Dockerfile, continuing to build the alternate file would not validate the affected production path. The new cache layout retains fast warm builds without relying on the alternate Dockerfile.

In main run 31831544506, mix release took 236.6s on arm64, with native compilation occupying roughly the first 216s. The dedicated layer uses the same per-package cargo rustc --release operation as Rustler so subsequent mix release invocations can reuse the final crate artifacts, not only downloaded dependencies.

Validation

• full local production image build passed
• cold dedicated native layer: 72.8s; following release step: 13.6s
• simulated Elixir-only source change kept the native layer cached and completed the release step in 15.3s
• fully warm image build reused both native and release layers
• manually dispatched GitHub production-image build passed on a fresh cache (run 31839947730); its immediate warm rerun reduced the Buildx step to 7s and the entire job to 21s (run 31840874212)
• final image smoke check verified the release executable, static manifest, VERSION, and LOGFLARE_COMMIT_SHA

Stack

• Depends on ci: isolate Docker build caches #3831
• Review this PR relative to ci/isolate-docker-build-caches

[amokan]

Approved with, what I view to be, non-blocking 🤖 items worth noting

[amokan]

Non-blocking 🤖

the crate list arrowipc_ex ch_compression_ex mapper_ex sqlparser_ex is hardcoded, duplicating Cargo.toml's members. A 5th NIF crate silently drops out of the prebuild with no signal; a renamed/removed crate is worse — cargo rustc --package <gone> exits non-zero and set -e hard-fails the production image build. Options: (a) RUN cargo build --workspace --release --locked — with no trailing rustc args it produces identical units/fingerprints to Rustler's cargo rustc and parallelizes instead of serializing; (b) derive from cargo metadata --no-deps; (c) guard the list against Cargo.toml members. Recommend (a).

[amokan]

Non-blocking 🤖

Switching the PR check from Dockerfile.multi-step to ./Dockerfile roughly doubles cold-cache cost against an unchanged timeout-minutes: 30. Dockerfile.multi-step starts FROM supabase/logflare:base (apt/nodejs/rustup, mix deps.compile, npm ci, cargo fetch already baked); the production Dockerfile redoes all of it. And the cache is cold more often than it looks: this workflow triggers only on pull_request/workflow_dispatch, never main, so cache-to: type=gha,scope=docker-build-check only ever writes PR-scoped caches readable by that PR alone. Every PR's first Docker Build Check is a full cold build on a 4-vCPU runner — plausibly 22–26 min against the 30-min cap. Options: (a) add push: branches: [main] so the shared default-branch cache gets populated; (b) raise timeout-minutes to 60; (c) use the Blacksmith runner the other Docker jobs use. Recommend (a)+(b).

[amokan]

Non-blocking 🤖

COPY . ./ re-stamps native/** mtimes, so the cached native layer won't survive on CI the way it does locally. BuildKit preserves build-context mtimes on COPY (verified empirically), and actions/checkout writes every file with the current time — so on a prebuild-layer cache hit, native/**/*.rs lands newer than the /app/target artifacts restored from that layer, and cargo's mtime freshness check recompiles + relinks all four local crates inside mix release. Registry deps keep Precalculated fingerprints so most of the win survives, but the "15.3s release step" was measured locally and won't reproduce in CI. Options: (a) normalize mtimes — find /app/native /app/Cargo.toml /app/Cargo.lock -exec touch -h -d @1 {} + before the cargo rustc step and again after COPY . ./ (safe: a genuine native change busts the COPY native native/ layer anyway); (b) RUN --mount=type=cache,target=/app/target (but cache-to: type=gha doesn't export cache mounts, losing the cross-runner win); (c) accept it and drop the 15.3s claim. Recommend (a).