fork: preserve canonical URLs in uv.lock across re-locks (v0.12.0) - #3
Closed
harupy wants to merge 3 commits into
Closed
fork: preserve canonical URLs in uv.lock across re-locks (v0.12.0)#3harupy wants to merge 3 commits into
harupy wants to merge 3 commits into
Conversation
Rewrite proxy registry URLs in `uv.lock` to their canonical counterparts via the `UV_INDEX_PROXIES` environment variable, so that lockfiles stay stable regardless of which mirror resolved the packages. Context: astral-sh#6349. When `UV_DEFAULT_INDEX` points at an internal mirror, upstream `uv lock` rewrites every `source.registry` URL in `uv.lock` to that mirror, creating noisy diffs and breaking portability across environments that use different mirrors. Set `UV_INDEX_PROXIES` with `canonical:proxy` mappings: UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simple After resolution, `Lock::rewrite_proxy_urls` replaces every matching proxy registry URL with its canonical counterpart. The canonical URLs are also injected into the `satisfies()` check so subsequent `uv lock` runs recognize the lockfile as up-to-date instead of re-resolving, and are mapped back to the proxy at install time so fetches hit a reachable mirror. All fork changes are tagged with `// fork:` comments so rebasers can find them quickly. Rebased from v0.11.14 onto 0.12.0. Notable adaptations: - Upstream deleted the `IndexUrls` type, so the commit patching `IndexUrls::default_index()` was dropped; the resolver now routes through `IndexLocations::default_index()`, which remains patched. - Upstream extracted index recording into `Lock::record_index` and moved the explicit-index pass into `satisfies_requires_dist`, so this fork only injects canonical URLs into the `remotes` set. - `UrlString::new` is now private, so mappings are built through the public API (`DisplaySafeUrl::parse` -> `UrlString::from`), which also rejects malformed `UV_INDEX_PROXIES` entries. - `std::env::set_var` is `unsafe` under edition 2024, so the env-reading entry points are split into thin wrappers over pure functions taking `&[ProxyMapping]`, which the tests call directly. This avoids `unsafe` and makes the tests parallel-safe. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
harupy
force-pushed
the
fork/preserve-lockfile-urls-v0.12.0
branch
from
July 30, 2026 07:03
5f77945 to
81d19dd
Compare
A proxy index advertises artifact URLs on its own host, so every wheel
and sdist URL in `uv.lock` pointed at the proxy, and every download went
through it. That makes the lockfile non-portable in the same way the
`source.registry` URLs were.
Add `UV_ARTIFACT_PROXIES`, mapping canonical artifact base URLs to proxy
base URLs:
UV_ARTIFACT_PROXIES=https://files.pythonhosted.org/packages:https://pypi-proxy.example.com/packages
The rewrite happens in `FileLocation::new`, the single point where both
`File::try_from_pypi` and `File::try_from_pyx` turn index-response URLs
into locations. That covers resolution-time downloads and the URLs
recorded in `uv.lock` at once, so no reverse mapping is needed at install
time.
Unlike the `UV_INDEX_PROXIES` mappings, these are prefix mappings, since
every artifact has a distinct path below the base. Matching respects path
boundaries, so a `/packages` base does not match `/packages-internal`.
Only absolute artifact URLs are rewritten. An index advertising relative
URLs resolves them against the index base at `to_url()` time and would
still point at the proxy.
Verified end-to-end against the internal mirror: `uv lock` records
`files.pythonhosted.org` URLs while `source.registry` stays on the proxy,
the recorded hashes match the proxy-served bytes, and the `.metadata`
sidecar is fetched from `files.pythonhosted.org`.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The `xattr -d com.apple.quarantine` line is self-explanatory; the comment just added noise to the copy-pasteable block. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Keep
uv.lockfree of proxy-specific URLs, so the same lockfile works across environments that reach PyPI through different mirrors. Rebased on top of upstream0.12.0.Context: astral-sh#6349. When
UV_DEFAULT_INDEXpoints at an internal mirror, upstreamuv lockbakes that mirror's host into both thesource.registryfield and every artifact URL, creating noisy diffs and breaking portability.Supersedes #2 (which targeted
0.11.14).Two independent mappings, because the canonical hosts differ — the index lives at
pypi.org/simplewhile artifacts live atfiles.pythonhosted.org/packages.UV_INDEX_PROXIES— registry URLsexport UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simpleAfter resolution,
Lock::rewrite_proxy_urlsreplaces every matching proxy registry URL with its canonical counterpart insource.registry. The canonical URLs are also injected into thesatisfies()check so subsequentuv lockruns recognize the lockfile as up-to-date instead of re-resolving, and are mapped back to the proxy at install time so fetches still hit a reachable mirror.UV_ARTIFACT_PROXIES— wheel and sdist URLsexport UV_ARTIFACT_PROXIES=https://files.pythonhosted.org/packages:https://pypi-proxy.example.com/packagesA proxy index advertises artifact URLs on its own host, so every wheel and sdist URL in
uv.lockpointed at the proxy and every download went through it.The rewrite happens in
FileLocation::new, the single point where bothFile::try_from_pypiandFile::try_from_pyxturn index-response URLs into locations. That covers resolution-time downloads and the URLs recorded inuv.lockat once, so — unlike the registry mapping — no reverse mapping is applied: artifacts are fetched directly from the canonical host.Two differences from the registry mapping worth knowing:
/packagesbase does not match/packages-internal/....to_url()time and would still point at the proxy.All fork changes are tagged with
// fork:comments so rebasers can find them quickly.Changes vs. #2, beyond the rebase
Upstream moved underneath this fork in four places:
fork: apply UV_INDEX_PROXIES to IndexUrls::default_index() too. Upstream deleted theIndexUrlstype entirely in 0.12 — it has zero remaining usages tree-wide. The resolver now routes throughIndexLocations::default_index(), which is still patched, so the behavior that commit existed for is retained.record_indexrefactor. The two duplicated index-recording blocks this fork used to patch are now a singleLock::record_indexhelper, and the "record explicit indexes" pass moved intosatisfies_requires_dist. This fork now only injects canonical URLs into theremotesset.UrlString::newis private in 0.12. Mappings are now built through the public API (DisplaySafeUrl::parse→UrlString::from), which also means malformed entries are rejected rather than producing an unusable mapping.std::env::set_varisunsafeunder edition 2024. Rather than wrap the tests inunsafeblocks, the env-reading entry points are split into thin wrappers over pure functions taking the parsed mappings, and tests call the pure functions. This removes theunsaferequirement and makes the tests parallel-safe — previously they mutated a process-global env var and could interfere with each other undercargo test's default threading.The
lock/mod.rsdiff shrank from ~130 lines of commented-out upstream code to +23/−8, which should make the next rebase materially easier.Diff surface
.github/workflows/fork-release.ymlcrates/uv-resolver/src/lock/url_preservation.rscrates/uv-distribution-types/src/artifact_proxies.rscrates/uv-distribution-types/src/index_url.rscrates/uv/src/commands/project/lock.rscrates/uv-resolver/src/lock/mod.rscrates/uv-distribution-types/src/file.rscrates/uv-distribution-types/src/lib.rsTest plan
cargo clippy -p uv-resolver -p uv-distribution-types --all-targets— clean, no warnings.cargo clippy -p uv --all-targets— clean, no warnings.cargo test -p uv-resolver --lib url_preservation— 6 passed.cargo test -p uv-distribution-types --lib— 64 passed (includes 6 newartifact_proxiestests).cargo fmt --check— clean.UV_DEFAULT_INDEXpointed at the proxy andUV_ARTIFACT_PROXIESset,uv lockrecordedfiles.pythonhosted.orgURLs for both the sdist and the wheel whilesource.registrystayed on the proxy. Verbose logs confirm the.metadatasidecar was fetched fromfiles.pythonhosted.org. The recorded wheel hash matches the proxy-served bytes (verified byte-identical,sha256:ff6d3f77…), so hashes stay valid either way.uv lockrun reportsExisting uv.lock satisfies workspace requirements(no re-resolve). Not verified.files.pythonhosted.orgreachable from CI runners. Not verified — see deployment note above.Known gap
There is no integration test under
crates/uv/tests/it/for either rewrite — coverage is unit tests plus the manual end-to-end run above.AGENTS.mdprefers integration tests; worth adding if this fork lives longer than expected.🤖 Generated with Claude Code