Skip to content

package: configurable registries, search and mirrors - #1379

Draft
bigbes wants to merge 4 commits into
bigbes/tntp-9959-go-luarocks-nativefrom
bigbes/tntp-9960-registry
Draft

bigbes wants to merge 4 commits into
bigbes/tntp-9959-go-luarocks-nativefrom
bigbes/tntp-9960-registry

Conversation

@bigbes

@bigbes bigbes commented Sep 10, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #1377 — the base is that branch, so the diff here is only this
change. Re-target to v3 once #1377 merges.

What this adds

Configurable registries. One precedence, everywhere:
--registry > TT_REGISTRIES > [platform].registries > the built-in
defaults. A dependency's own registry key still wins for that dependency.
The layers replace one another rather than merging: a rock-server list is
ordered and first-found-wins, so appending a layer would change which server
answers instead of adding a fallback. An entry is an http(s) URL or a local
directory; a relative path resolves against the manifest for the manifest's
own list, and against the working directory otherwise. rocks.EffectiveRegistries
is the single function that settles it, and every command that queries a
server now goes through it — build, fetch, pack, resolve, add,
remove, update, search, download and registry list.

tt registry list [-o table|json|yaml]. The effective list in resolution
order, each server with the layer it came from, so an unexpected server can be
explained rather than guessed at.

tt package search <term> [-o …]. Every version every server offers.
Unlike resolution, which stops at the first server that has a rock, a search
asks all of them. A term nothing matches is not an error: the table writes
nothing to stdout and a line to stderr, the machine formats print an empty
list.

tt package download [<name>[@<version>]…] [--dir <path>]. A mirror
builder, not a file fetcher. It writes the LuaRocks manifest beside the rock
files it fetches, so the directory it leaves behind is a rock server. With
no arguments it mirrors the project's locked closure — every product's
dependencies plus the dev closure — at the exact locked versions. Re-running
overwrites and re-indexes, so a mirror is extended in place.

Deviations worth review

  • The lock records no server, so the no-argument download cannot honour a
    per-dependency registry: manifest.LockDependency carries name, version,
    source, checksum, path and content hash and nothing else. Every locked rock
    is fetched from the effective list. Related and pre-existing:
    build.materialize installs from the global server list too, so a rock
    resolved from a per-dependency registry is already installed from
    elsewhere. Not touched here.
  • rocks.EffectiveRegistries takes the manifest's list as []string rather
    than a *manifest.Manifest, so cli/manifest/rocks does not import the
    root package and the layering stays as the package docs describe it.
  • --registry is also on remove and pack, which were not in the original
    list: both re-resolve or build, so omitting them would leave two commands
    querying different servers from their neighbours.
  • cli/manifest/registry is added to the strict .golangci.yml adopted set.
    Verified by planting a deliberate violation and watching it get reported.

Verification

Go 1.26.0 (GOTOOLCHAIN=go1.26.0), golangci-lint 2.13.2 — the version
magefile.go and CI pin; the linter on the machine was 2.11.4 and was not used.

  • go build ./..., go vet ./..., mage lint:golang — clean.
  • Every one of the 8 commits in the stack builds on its own (loop over
    git rev-list, 8 iterations, 0 unbuildable).
  • mage unit: 48 packages ok. cli/configure's TestConfigureCli fails
    identically on the base commit (a macOS /private symlink assertion).
  • Go tests: 18 precedence/path subtests in cli/manifest/rocks, 5 lock-walk
    cases plus ref/format/render coverage in cli/manifest/registry.
  • pytest: test/integration/package + test/integration/registry — 75
    passed, 3 skipped (docker).

Mutations

mutation result
swap the flag and env layers TestEffectiveRegistriesPrecedence/flag_wins_over_everything and …LayersDoNotMerge go red
resolve manifest paths against the working directory …ResolvesRelativePaths/a_manifest_path_is_relative_to_the_project_directory goes red
make the mirror indexing step a no-op 3 pytest cases go red, including both offline builds
delete manifest from a built mirror the offline build fails with remote.FileRemoteIndex: load …: no manifest variant found where the control build succeeds

Every mutant was checked to build (go vet clean) before its test result was read.

The acceptance test is the one that matters:

tt package resolve
tt package download --dir ./mirror
<loopback server stopped>
TT_REGISTRIES=./mirror tt package build --locked

and .rocks/share/tarantool/stat/init.lua is there afterwards. The server is
stopped before the build, so a build that succeeds cannot have reached it.

Part of TNTP-9960

[platform].registries declares the ordered rock-server list a project
resolves against. An entry is an HTTP(S) URL or a local directory,
and a relative path is taken relative to the manifest.

rocks.EffectiveRegistries picks the list a command must use: the
first of the command line, the environment and the manifest that
names anything, falling back to the built-in defaults. The layers
replace one another rather than merging, because the list is ordered
and first-found-wins - appending would change which server answers
instead of adding a fallback.

Each entry carries the layer it came from, so a listing can say why
a server is being queried.

Part of TNTP-9960
tt package build, fetch, pack, resolve, add, remove and update now
take --registry, read TT_REGISTRIES, and fall back to the manifest
and then to the defaults, so the server list a run queries is settled
in one place instead of being the adapter default everywhere.

An unusable list is reported before the run starts rather than at the
first query, so a dependency edit is not written against servers that
were never reachable.

Part of TNTP-9960
A new package for the three commands that address rock servers rather
than a dependency graph.

Download is a mirror builder, not a file fetcher: it writes the
LuaRocks manifest beside the files it fetches, so the directory it
leaves behind is a rock server. Point a registry at it and the
project builds with no network. With no arguments it mirrors the
locked closure - every product's dependencies and the dev closure -
at the exact locked versions.

Indexing is the one operation still served by the Lua engine, since
the native one implements no admin command. It is local: a directory
walk, no network.

The package joins the strict linter's adopted set.

Part of TNTP-9960
tt registry list prints the effective server list in resolution order,
each server with the layer it came from, so a build querying an
unexpected server can be explained without guessing.

tt package search reports every version every server offers, since a
search asks what exists rather than picking one artifact. A term that
matches nothing is not an error.

tt package download builds a mirror. The acceptance test stops the
loopback server before building, so a build that succeeds cannot have
reached it.

Part of TNTP-9960
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant