Skip to content

fork: preserve canonical URLs in uv.lock across re-locks (v0.12.0) - #3

Closed
harupy wants to merge 3 commits into
mainfrom
fork/preserve-lockfile-urls-v0.12.0
Closed

fork: preserve canonical URLs in uv.lock across re-locks (v0.12.0)#3
harupy wants to merge 3 commits into
mainfrom
fork/preserve-lockfile-urls-v0.12.0

Conversation

@harupy

@harupy harupy commented Jul 30, 2026

Copy link
Copy Markdown
Member

Summary

Keep uv.lock free of proxy-specific URLs, so the same lockfile works across environments that reach PyPI through different mirrors. Rebased on top of upstream 0.12.0.

Context: astral-sh#6349. When UV_DEFAULT_INDEX points at an internal mirror, upstream uv lock bakes that mirror's host into both the source.registry field and every artifact URL, creating noisy diffs and breaking portability.

Supersedes #2 (which targeted 0.11.14).

Two independent mappings, because the canonical hosts differ — the index lives at pypi.org/simple while artifacts live at files.pythonhosted.org/packages.

UV_INDEX_PROXIES — registry URLs

export UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simple

After resolution, Lock::rewrite_proxy_urls replaces every matching proxy registry URL with its canonical counterpart in source.registry. The canonical URLs are also injected into the satisfies() check so subsequent uv lock runs recognize the lockfile as up-to-date instead of re-resolving, and are mapped back to the proxy at install time so fetches still hit a reachable mirror.

UV_ARTIFACT_PROXIES — wheel and sdist URLs

export UV_ARTIFACT_PROXIES=https://files.pythonhosted.org/packages:https://pypi-proxy.example.com/packages

A proxy index advertises artifact URLs on its own host, so every wheel and sdist URL in uv.lock pointed at the proxy and every download went through it.

The rewrite happens in FileLocation::new, the single point where both File::try_from_pypi and File::try_from_pyx turn index-response URLs into locations. That covers resolution-time downloads and the URLs recorded in uv.lock at once, so — unlike the registry mapping — no reverse mapping is applied: artifacts are fetched directly from the canonical host.

Two differences from the registry mapping worth knowing:

  • These are prefix mappings, since every artifact has a distinct path below the base. Matching respects path boundaries, so a /packages base does not match /packages-internal/....
  • Only absolute artifact URLs are rewritten. An index advertising relative URLs resolves them against the index base at to_url() time and would still point at the proxy.

Deployment note: because artifacts are no longer fetched through the proxy, every environment running uv lock/uv sync must be able to reach files.pythonhosted.org directly. This was confirmed on a developer machine but not on CI runners — worth verifying there before relying on it, since a runner that cannot reach it will fail rather than fall back to the proxy.

All fork changes are tagged with // fork: comments so rebasers can find them quickly.

Changes vs. #2, beyond the rebase

Upstream moved underneath this fork in four places:

  • Dropped fork: apply UV_INDEX_PROXIES to IndexUrls::default_index() too. Upstream deleted the IndexUrls type entirely in 0.12 — it has zero remaining usages tree-wide. The resolver now routes through IndexLocations::default_index(), which is still patched, so the behavior that commit existed for is retained.
  • Adapted to upstream's record_index refactor. The two duplicated index-recording blocks this fork used to patch are now a single Lock::record_index helper, and the "record explicit indexes" pass moved into satisfies_requires_dist. This fork now only injects canonical URLs into the remotes set.
  • UrlString::new is private in 0.12. Mappings are now built through the public API (DisplaySafeUrl::parseUrlString::from), which also means malformed entries are rejected rather than producing an unusable mapping.
  • std::env::set_var is unsafe under edition 2024. Rather than wrap the tests in unsafe blocks, the env-reading entry points are split into thin wrappers over pure functions taking the parsed mappings, and tests call the pure functions. This removes the unsafe requirement and makes the tests parallel-safe — previously they mutated a process-global env var and could interfere with each other under cargo test's default threading.

The lock/mod.rs diff shrank from ~130 lines of commented-out upstream code to +23/−8, which should make the next rebase materially easier.

Diff surface

File Change
.github/workflows/fork-release.yml +165
crates/uv-resolver/src/lock/url_preservation.rs +341 (new, 6 unit tests)
crates/uv-distribution-types/src/artifact_proxies.rs +160 (new, 6 unit tests)
crates/uv-distribution-types/src/index_url.rs +30
crates/uv/src/commands/project/lock.rs +26/−8
crates/uv-resolver/src/lock/mod.rs +23/−8
crates/uv-distribution-types/src/file.rs +6/−1
crates/uv-distribution-types/src/lib.rs +2

Test plan

  • cargo clippy -p uv-resolver -p uv-distribution-types --all-targets — clean, no warnings.
  • cargo clippy -p uv --all-targets — clean, no warnings.
  • cargo test -p uv-resolver --lib url_preservation — 6 passed.
  • cargo test -p uv-distribution-types --lib — 64 passed (includes 6 new artifact_proxies tests).
  • cargo fmt --check — clean.
  • End-to-end against the internal mirror. With UV_DEFAULT_INDEX pointed at the proxy and UV_ARTIFACT_PROXIES set, uv lock recorded files.pythonhosted.org URLs for both the sdist and the wheel while source.registry stayed on the proxy. Verbose logs confirm the .metadata sidecar was fetched from files.pythonhosted.org. The recorded wheel hash matches the proxy-served bytes (verified byte-identical, sha256:ff6d3f77…), so hashes stay valid either way.
  • Second uv lock run reports Existing uv.lock satisfies workspace requirements (no re-resolve). Not verified.
  • files.pythonhosted.org reachable from CI runners. Not verified — see deployment note above.

Known gap

There is no integration test under crates/uv/tests/it/ for either rewrite — coverage is unit tests plus the manual end-to-end run above. AGENTS.md prefers integration tests; worth adding if this fork lives longer than expected.

🤖 Generated with Claude Code

Rewrite proxy registry URLs in `uv.lock` to their canonical counterparts
via the `UV_INDEX_PROXIES` environment variable, so that lockfiles stay
stable regardless of which mirror resolved the packages.

Context: astral-sh#6349. When `UV_DEFAULT_INDEX` points at an internal
mirror, upstream `uv lock` rewrites every `source.registry` URL in
`uv.lock` to that mirror, creating noisy diffs and breaking portability
across environments that use different mirrors.

Set `UV_INDEX_PROXIES` with `canonical:proxy` mappings:

    UV_INDEX_PROXIES=https://pypi.org/simple:https://pypi-proxy.example.com/simple

After resolution, `Lock::rewrite_proxy_urls` replaces every matching proxy
registry URL with its canonical counterpart. The canonical URLs are also
injected into the `satisfies()` check so subsequent `uv lock` runs
recognize the lockfile as up-to-date instead of re-resolving, and are
mapped back to the proxy at install time so fetches hit a reachable
mirror.

All fork changes are tagged with `// fork:` comments so rebasers can find
them quickly.

Rebased from v0.11.14 onto 0.12.0. Notable adaptations:

- Upstream deleted the `IndexUrls` type, so the commit patching
  `IndexUrls::default_index()` was dropped; the resolver now routes
  through `IndexLocations::default_index()`, which remains patched.
- Upstream extracted index recording into `Lock::record_index` and moved
  the explicit-index pass into `satisfies_requires_dist`, so this fork
  only injects canonical URLs into the `remotes` set.
- `UrlString::new` is now private, so mappings are built through the
  public API (`DisplaySafeUrl::parse` -> `UrlString::from`), which also
  rejects malformed `UV_INDEX_PROXIES` entries.
- `std::env::set_var` is `unsafe` under edition 2024, so the env-reading
  entry points are split into thin wrappers over pure functions taking
  `&[ProxyMapping]`, which the tests call directly. This avoids `unsafe`
  and makes the tests parallel-safe.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@harupy
harupy force-pushed the fork/preserve-lockfile-urls-v0.12.0 branch from 5f77945 to 81d19dd Compare July 30, 2026 07:03
A proxy index advertises artifact URLs on its own host, so every wheel
and sdist URL in `uv.lock` pointed at the proxy, and every download went
through it. That makes the lockfile non-portable in the same way the
`source.registry` URLs were.

Add `UV_ARTIFACT_PROXIES`, mapping canonical artifact base URLs to proxy
base URLs:

    UV_ARTIFACT_PROXIES=https://files.pythonhosted.org/packages:https://pypi-proxy.example.com/packages

The rewrite happens in `FileLocation::new`, the single point where both
`File::try_from_pypi` and `File::try_from_pyx` turn index-response URLs
into locations. That covers resolution-time downloads and the URLs
recorded in `uv.lock` at once, so no reverse mapping is needed at install
time.

Unlike the `UV_INDEX_PROXIES` mappings, these are prefix mappings, since
every artifact has a distinct path below the base. Matching respects path
boundaries, so a `/packages` base does not match `/packages-internal`.

Only absolute artifact URLs are rewritten. An index advertising relative
URLs resolves them against the index base at `to_url()` time and would
still point at the proxy.

Verified end-to-end against the internal mirror: `uv lock` records
`files.pythonhosted.org` URLs while `source.registry` stays on the proxy,
the recorded hashes match the proxy-served bytes, and the `.metadata`
sidecar is fetched from `files.pythonhosted.org`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@harupy harupy changed the title fork: preserve URLs in uv.lock across re-locks (v0.12.0) fork: preserve canonical URLs in uv.lock across re-locks (v0.12.0) Jul 30, 2026
The `xattr -d com.apple.quarantine` line is self-explanatory; the comment
just added noise to the copy-pasteable block.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@harupy harupy closed this Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant