Skip to content

Commit 592d5f3

Browse files
Merge remote-tracking branch 'upstream/master' into addsubmonoid-ssup
2 parents 8aea717 + 6bd5e54 commit 592d5f3

8,603 files changed

Lines changed: 310470 additions & 163236 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.devcontainer/devcontainer.json‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
{
22
"name": "Mathlib4 dev container",
3+
"remoteUser": "vscode",
34

45
"build": {
56
"dockerfile": "Dockerfile"
67
},
78

89
"onCreateCommand": "lake exe cache get!",
10+
"postCreateCommand": "gh auth status || gh auth login",
911

1012
"hostRequirements": {
1113
"cpus": 4,
@@ -16,5 +18,14 @@
1618
"vscode": {
1719
"extensions": ["leanprover.lean4"]
1820
}
19-
}
21+
},
22+
23+
"features": {
24+
// install `gh`
25+
"ghcr.io/devcontainers/features/github-cli:1": {}
26+
},
27+
28+
"mounts": [
29+
"source=${localEnv:HOME}/.gitconfig,target=/home/vscode/.gitconfig,type=bind,consistency=cached"
30+
]
2031
}

‎.git-blame-ignore-revs‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# 2024-05-20 replace `refine'` with one underscore by `refine` (#13059)
2+
7493b5f81b4c031b87877c5c124bea1ddc4e567d
3+
# 2024-05-24 replace many `refine'` with `refine` (#13166)
4+
fc48848e4374f13c796a7399bfccd2e228f776df
5+
# 2025-11-19 move Mathlib to the module system (#31786)
6+
6a54a80825b060ab20dc31751ebdce78b3a3b518
7+
# 2025-12-01 fix spelling in doc-strings (#32286)
8+
b728e1450a53133aa4171eeadc0bc8d3ee58415c

‎.github/PULL_REQUEST_TEMPLATE.md‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,5 +53,3 @@ using the following format:
5353
- [ ] depends on: #xyz [optional extra text]
5454
5555
-->
56-
57-
[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/from-referrer/)
Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
# Single source of truth mapping (repo, ref) → (upload container,
2+
# read fallback chain) for Mathlib's multi-container cache.
3+
#
4+
# Called by build, upload_cache, and post_steps in build_template.yml so
5+
# trust classification is decided in exactly one place. Lean-side cache
6+
# logic stays branch-agnostic; this composite is the seam where CI-only
7+
# trust policy lives.
8+
9+
name: Cache trust dispatch
10+
description: Compute the cache container target and read fallback for this job.
11+
12+
inputs:
13+
repo:
14+
description: GitHub repo full name (`owner/name`).
15+
required: true
16+
branch:
17+
description: |
18+
Branch name (`github.head_ref || github.ref_name`); on tag-triggered
19+
runs this is the tag name, distinguished via `github.ref_type` inside
20+
the dispatch step.
21+
required: true
22+
head-sha:
23+
description: |
24+
Head commit SHA for the ref being built. Used as the per-commit cache
25+
namespace (`MATHLIB_CACHE_REPO_SCOPE`) for fork-trust uploads, so a
26+
closed/hidden PR's poisoned artifacts cannot be served to a later
27+
honest PR from the same fork.
28+
required: true
29+
30+
# Outputs are mirrored to $GITHUB_ENV inside the step, which is what
31+
# downstream `cache get` / `cache put-staged` calls actually read. We
32+
# also expose them as action outputs for callers that need them in
33+
# `with:` blocks (e.g. constructing further `if:` conditions).
34+
outputs:
35+
primary:
36+
description: Container name for uploads (master, forks, nightly-testing, pr-toolchain-tests).
37+
value: ${{ steps.dispatch.outputs.primary }}
38+
read-chain:
39+
description: |
40+
Comma-separated read fallback chain for `MATHLIB_CACHE_FROM`. Empty
41+
when the job should use the cache tool's repo-level default.
42+
value: ${{ steps.dispatch.outputs.read-chain }}
43+
repo-scope:
44+
description: |
45+
Per-commit namespace suffix for `MATHLIB_CACHE_REPO_SCOPE`. Set to
46+
the head SHA when uploading to fork-trust containers; empty for
47+
master / nightly / pr-toolchain-tests uploads where scoping isn't
48+
applied.
49+
value: ${{ steps.dispatch.outputs.repo-scope }}
50+
51+
runs:
52+
using: composite
53+
steps:
54+
- name: Compute trust dispatch
55+
id: dispatch
56+
shell: bash
57+
env:
58+
REPO: ${{ inputs.repo }}
59+
BRANCH: ${{ inputs.branch }}
60+
HEAD_SHA: ${{ inputs.head-sha }}
61+
# Whether `branch` names a branch or a tag. Read from the run context
62+
# rather than an input: unlike `repo`/`branch`, the value does not
63+
# depend on which event shape (PR vs push) the caller handles.
64+
REF_TYPE: ${{ github.ref_type }}
65+
run: |
66+
PRIMARY=""
67+
READ_CHAIN=""
68+
REPO_SCOPE=""
69+
70+
# Security note: this dispatch is NOT the trust boundary for writes.
71+
# The real enforcement is the OIDC bearer token minted in upload_cache:
72+
# the token is scoped to a specific container, so a malicious actor
73+
# rewriting this case to `--container=master` from a fork build would
74+
# be 403'd by Azure regardless. The dispatch exists so the workflow
75+
# does the right thing in the honest case; defence in depth is RBAC.
76+
77+
# Privileged containers (master, nightly-testing, pr-toolchain-tests) are
78+
# writable only by a repo's own native CI, whose OIDC token is RBAC-scoped
79+
# to match. A cross-repo pull request is built by build_fork.yml with
80+
# fork-trust credentials whatever repo the PR head lives on (e.g. a `bump/*`
81+
# branch on the mathlib4-nightly-testing fork), so it can only write
82+
# `forks`. `GITHUB_REPOSITORY` is the repo the workflow runs as (the base
83+
# repo for `pull_request_target`), i.e. the one whose credentials this job
84+
# holds; when it differs from `$REPO`, the build is a fork and targets
85+
# `forks` regardless of the head repo's own trust class.
86+
if [ "$REPO" != "$GITHUB_REPOSITORY" ]; then
87+
PRIMARY="forks"
88+
else
89+
case "$REPO" in
90+
"leanprover-community/mathlib4")
91+
if [ "$REF_TYPE" = "tag" ]; then
92+
case "$BRANCH" in
93+
v4.*)
94+
# `v4.*` release tags: release_cache.yml rebuilds off-master
95+
# release commits (patch releases, patched release
96+
# candidates) and publishes them into `master`, the container
97+
# canonical checkouts read. Tag creation is restricted to
98+
# release managers by a tag ruleset, and the
99+
# `cache-upload-master` environment admits `v4.*` tag refs,
100+
# so these builds carry master trust. Reads use the tool's
101+
# default for the canonical repo, `master` alone, for the
102+
# same fill-in reason as the master/staging arm below.
103+
PRIMARY="master"
104+
;;
105+
*)
106+
# Other tags have no trust class of their own:
107+
# fork-equivalent, like the dev-branch arm below.
108+
PRIMARY="forks"
109+
READ_CHAIN="master,forks"
110+
;;
111+
esac
112+
else
113+
case "$BRANCH" in
114+
"master"|"staging")
115+
# Master, staging, and `v4.*` release tags (above) are the
116+
# only writers that feed `master` (`staging` is bors's merge
117+
# candidate, which fast-forwards to `master`). Reads use the
118+
# tool's default for the canonical repo, `master` alone, and
119+
# the chain must not widen: files the read chain serves are
120+
# skipped at stage time, so every file missing from `master`
121+
# is a miss that gets rebuilt and uploaded, and `master`
122+
# fills itself into a standalone cache. (Only PRIMARY=master
123+
# does this; other runs write to `forks` and widen the
124+
# chain.)
125+
PRIMARY="master"
126+
;;
127+
*)
128+
# `bors trying`, `ci-dev/*`, maintainer dev branches on the
129+
# canonical repo: trust level is fork-equivalent (the OIDC
130+
# token's RBAC scopes them to `forks`). Reads must widen
131+
# past the default [master] so the post-build verification
132+
# finds the just-uploaded fork-trust artifacts.
133+
PRIMARY="forks"
134+
READ_CHAIN="master,forks"
135+
;;
136+
esac
137+
fi
138+
;;
139+
"leanprover-community/mathlib4-nightly-testing")
140+
case "$BRANCH" in
141+
"nightly-testing"|"nightly-testing-green"|"staging"|bump/*)
142+
# Trusted nightly refs use the default [nightly-testing, forks].
143+
# It excludes `pr-toolchain-tests` so an upload from a
144+
# `batteries-pr-testing-*` branch never reaches a trusted-nightly
145+
# consumer.
146+
PRIMARY="nightly-testing"
147+
;;
148+
*)
149+
# `batteries-pr-testing-*`, etc.:
150+
# least-trusted (can build with arbitrary toolchains). Widen
151+
# reads to recover this branch's own previously-uploaded
152+
# artifacts; trusted-nightly stays preferred where hash
153+
# spaces happen to align.
154+
PRIMARY="pr-toolchain-tests"
155+
READ_CHAIN="pr-toolchain-tests,nightly-testing"
156+
;;
157+
esac
158+
;;
159+
*)
160+
# Foreign fork. The cache tool's default chain for a fork repo
161+
# is [master, forks] (master-first): master supplies the
162+
# bulk of unchanged upstream deps, forks supplies PR-specific
163+
# files. No widening needed, so MATHLIB_CACHE_FROM stays unset.
164+
PRIMARY="forks"
165+
;;
166+
esac
167+
fi
168+
169+
# Per-commit cache namespace, only for fork-trust uploads. Closes the
170+
# within-fork temporal replay attack: each commit's CI run gets its
171+
# own /f/{repo}/{sha}/... namespace, so artifacts from a closed/
172+
# hidden PR cannot be served to a later honest build on the same
173+
# fork. Master / nightly / pr-toolchain-tests uploads stay un-scoped:
174+
# master has a single writer (no replay risk), and the per-toolchain
175+
# hash partitioning isolates nightly and toolchain-test classes via
176+
# their root-hash inputs.
177+
if [ "$PRIMARY" = "forks" ]; then
178+
REPO_SCOPE="$HEAD_SHA"
179+
fi
180+
181+
echo "primary=$PRIMARY" >> "$GITHUB_OUTPUT"
182+
echo "read-chain=$READ_CHAIN" >> "$GITHUB_OUTPUT"
183+
echo "repo-scope=$REPO_SCOPE" >> "$GITHUB_OUTPUT"
184+
echo "MATHLIB_CACHE_PRIMARY=$PRIMARY" >> "$GITHUB_ENV"
185+
if [ -n "$READ_CHAIN" ]; then
186+
echo "MATHLIB_CACHE_FROM=$READ_CHAIN" >> "$GITHUB_ENV"
187+
fi
188+
if [ -n "$REPO_SCOPE" ]; then
189+
echo "MATHLIB_CACHE_REPO_SCOPE=$REPO_SCOPE" >> "$GITHUB_ENV"
190+
fi
191+
# Visible in CI logs so a glance at any cache-touching step shows
192+
# what trust class the job is operating under.
193+
SCOPE_NOTE=""
194+
if [ -n "$REPO_SCOPE" ]; then
195+
SCOPE_NOTE=", MATHLIB_CACHE_REPO_SCOPE=$REPO_SCOPE"
196+
fi
197+
if [ -n "$READ_CHAIN" ]; then
198+
echo "cache-trust-dispatch: REPO=$REPO BRANCH=$BRANCH → container=$PRIMARY, MATHLIB_CACHE_FROM=$READ_CHAIN$SCOPE_NOTE"
199+
else
200+
echo "cache-trust-dispatch: REPO=$REPO BRANCH=$BRANCH → container=$PRIMARY, MATHLIB_CACHE_FROM=<default>$SCOPE_NOTE"
201+
fi
Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# Get this commit's oleans, in two phases:
2+
# 1. Warm the cache from the `cache-snapshot` GitHub artifact (canonical repo only).
3+
# 2. Fetch this commit's oleans from the remote cache with the trusted master-built binary.
4+
# The fetch is HEAD-scoped (reads only this commit's own cache scope). The warm is fail-safe
5+
# (any failure → just the remote fetch) and its source is hardcoded, so nothing can redirect
6+
# the download off the trusted master pipeline.
7+
name: Get cache
8+
description: Get this commit's oleans into the local cache.
9+
inputs:
10+
working_directory:
11+
description: The lake project to fetch the cache for (e.g. the checked-out PR branch).
12+
required: true
13+
cache_bin:
14+
description: Path to the trusted `cache` binary, relative to `working_directory`.
15+
required: true
16+
runs:
17+
using: composite
18+
steps:
19+
# 1. Warm cache from the GitHub artifact. Resolve which snapshot to use: the one built
20+
# at this commit's merge-base with master (its unchanged files hash identically
21+
# there), else the newest still-retained one at-or-before it, else the latest (a
22+
# merge-base older than retention, or any failure, lands here). Canonical repo only.
23+
- name: Resolve cache snapshot
24+
id: resolve
25+
if: ${{ github.repository == 'leanprover-community/mathlib4' }}
26+
shell: bash
27+
env:
28+
GH_TOKEN: ${{ github.token }}
29+
HEAD_SHA: ${{ github.event.pull_request.head.sha || github.sha }}
30+
run: |
31+
set -uo pipefail
32+
runs="repos/leanprover-community/mathlib4/actions/workflows/build.yml/runs?branch=master&event=push&status=success"
33+
34+
# Each helper prints a matching successful master-push run-id, or empty.
35+
run_at() { gh api "${runs}&head_sha=$1" --jq '.workflow_runs[0].id // empty' 2>/dev/null || true; }
36+
latest_run() { gh api "${runs}&per_page=1" --jq '.workflow_runs[0].id // empty' 2>/dev/null || true; }
37+
# newest run created at-or-before date $1, but not older than cutoff $2
38+
newest_before() {
39+
gh api "${runs}&per_page=100" 2>/dev/null | jq -r --arg d "$1" --arg c "$2" \
40+
'[.workflow_runs[] | select(.created_at <= $d and ($c == "" or .created_at >= $c))][0].id // empty' \
41+
2>/dev/null || true
42+
}
43+
# True if run-id $1 actually carries a `cache-snapshot` artifact. Older runs
44+
# predate the feature and artifacts expire, so a successful run is not enough.
45+
# Filters server-side by name, but re-checks in jq in case `name` is ignored.
46+
has_snapshot() {
47+
[[ -n "$(gh api "repos/leanprover-community/mathlib4/actions/runs/$1/artifacts?name=cache-snapshot&per_page=100" \
48+
--jq '.artifacts[] | select(.name == "cache-snapshot") | .id' 2>/dev/null | head -1 || true)" ]]
49+
}
50+
51+
# This PR's merge-base with master (+ its commit date), and the cutoff below which
52+
# snapshots have expired (retention ~14d).
53+
mb_info=$(gh api "repos/leanprover-community/mathlib4/compare/master...${HEAD_SHA}" \
54+
--jq '.merge_base_commit | "\(.sha) \(.commit.committer.date)"' 2>/dev/null || true)
55+
read -r mb mb_date <<< "${mb_info}"
56+
cutoff=$(date -u -d '13 days ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || true)
57+
58+
# Prefer the merge-base's snapshot (while still retained), then the newest one
59+
# before it, then the latest of all.
60+
run_id=""
61+
if [[ -n "${mb}" && ( -z "${cutoff}" || "${mb_date}" > "${cutoff}" ) ]]; then
62+
run_id=$(run_at "${mb}")
63+
[[ -z "${run_id}" ]] && run_id=$(newest_before "${mb_date}" "${cutoff}")
64+
fi
65+
[[ -z "${run_id}" ]] && run_id=$(latest_run)
66+
67+
# The resolved run may carry no `cache-snapshot` artifact: an older merge-base
68+
# predating the feature, or one whose artifact already expired (older-than-today
69+
# runs often won't have one). Rather than let the download step hard-error on a
70+
# missing artifact, confirm it's present; if not, fall back to the latest master
71+
# run, and warm only if that one has it.
72+
if [[ -n "${run_id}" ]] && ! has_snapshot "${run_id}"; then
73+
echo "Run ${run_id} has no cache-snapshot artifact; falling back to latest master run."
74+
run_id=$(latest_run)
75+
if [[ -n "${run_id}" ]] && ! has_snapshot "${run_id}"; then
76+
run_id=""
77+
fi
78+
fi
79+
80+
echo "Resolved cache-snapshot run_id: '${run_id}' (merge-base: ${mb:-unknown})"
81+
echo "run_id=${run_id}" >> "$GITHUB_OUTPUT"
82+
83+
- name: Warm cache from GitHub artifact
84+
if: ${{ steps.resolve.outputs.run_id != '' }}
85+
continue-on-error: true # fail-safe: fall back to the remote fetch (step 2)
86+
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
87+
with:
88+
name: cache-snapshot
89+
path: /home/lean/.cache/mathlib
90+
repository: leanprover-community/mathlib4
91+
run-id: ${{ steps.resolve.outputs.run_id }}
92+
github-token: ${{ github.token }}
93+
94+
# 2. Fetch this commit's oleans from the remote cache with the trusted `cache` binary.
95+
# Runs on every repo; the warm above just gives the canonical repo a local head
96+
# start. HEAD-scoped: reads only this commit's own cache scope.
97+
# Earlier steps give PR code write access to `.lake/`, so it can hold symlinks that
98+
# point outside it. The fetch runs inside landrun, which denies every write that
99+
# resolves outside `.lake/` and the local cache directory.
100+
- name: Fetch cache from remote
101+
shell: bash
102+
env:
103+
WORKDIR: ${{ inputs.working_directory }}
104+
CACHE_BIN: ${{ inputs.cache_bin }}
105+
CACHE_REPO: ${{ github.event.pull_request.head.repo.full_name || github.repository }}
106+
run: |
107+
set -eo pipefail
108+
# Use the trusted toolchain's `lean`/`leantar` (see setup-build-env).
109+
export PATH="${TRUSTED_LEAN_PREFIX:?trusted toolchain sysroot not set}/bin:${PATH}"
110+
cd "${WORKDIR}"
111+
# This shell writes the log outside the sandbox, so the log must not be under `.lake/`.
112+
log="${RUNNER_TEMP:-/tmp}/cache-get.log"
113+
# --repo so fork PRs also read their own repo-namespaced cache (master is read flat,
114+
# so this still gets the master bulk); for in-repo runs it resolves to the same repo.
115+
# `--rwx` on the local cache directory: `cache` can run a `curl` that it downloads there.
116+
landrun --unrestricted-network --rox /usr --rox /etc --rw /dev \
117+
--rox /home/lean/.elan --rox /home/lean/actions-runner/_work \
118+
--rwx /home/lean/.cache/mathlib/ --rw .lake/ \
119+
--env PATH --env HOME --env CI --env TERM --env LEAN_SRC_PATH \
120+
--env MATHLIB_CACHE_FROM --env MATHLIB_CACHE_REPO_SCOPE \
121+
--env MATHLIB_CACHE_GET_URL --env MATHLIB_CACHE_BASE_URL \
122+
--env CACHE_BIN --env CACHE_REPO \
123+
-- bash -euo pipefail -c \
124+
'rm -rf .lake/build/lib/lean/Mathlib; exec "${CACHE_BIN}" --repo="${CACHE_REPO}" get' \
125+
2>&1 | tee "${log}"
126+
# Warmth = how much the snapshot covered HEAD: files already cached locally
127+
# (just decompressed) vs downloaded from Azure. Parsed best-effort from the log.
128+
warm=$(grep -oE 'Decompressing [0-9]+ already-cached' "${log}" | grep -oE '[0-9]+' | head -1 || true)
129+
cold=$(grep -oE 'Attempting to download [0-9]+' "${log}" | grep -oE '[0-9]+' | head -1 || true)
130+
echo "Cache warmth: ${warm:-0} already-cached (warm) / ${cold:-0} downloaded from Azure (cold)"

‎.github/actions/get-mathlib-ci/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ then use the local action:
2525

2626
```yaml
2727
- name: Checkout local actions
28-
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
28+
uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
2929
with:
3030
ref: ${{ github.workflow_sha }}
3131
fetch-depth: 1

0 commit comments

Comments
 (0)