Skip to content

[None][infra] Bake pre-commit hook cache into tritondevel CI image - #17530

Closed
brnguyen2 wants to merge 1 commit into
NVIDIA:mainfrom
brnguyen2:brnguyen/precommit-hook-cache-baking
Closed

[None][infra] Bake pre-commit hook cache into tritondevel CI image#17530
brnguyen2 wants to merge 1 commit into
NVIDIA:mainfrom
brnguyen2:brnguyen/precommit-hook-cache-baking

Conversation

@brnguyen2

Copy link
Copy Markdown
Collaborator

Description

Pre-populate the pre-commit hook store in the tritondevel CI image at build time, so the Run type check stage no longer clones hooks from github.com at test time.

Problem. The type-check stage runs python3 -m pre_commit run type-check --all-files (jenkins/L0_Test.groovy). Even though type-check is a repo: local hook that touches no network, pre-commit enumerates every remote repo in .pre-commit-config.yaml — it clones each one (git fetch origin --tags) to read its hook manifest — before it runs the selected hook. So a single failed clone of e.g. github.com/pycqa/isort fails the whole stage:

[INFO] Initializing environment for https://github.com/pycqa/isort.
CalledProcessError: ('git', 'fetch', 'origin', '--tags') return code: 128
fatal: could not read Username for 'https://github.com': No such device or address

This is transient infra flakiness on test nodes that lack reliable github access, not a real type-check failure.

Fix. Build nodes do reach github anonymously (see install_mooncake.sh, install_ucx.sh, install_nixl.sh, which already git clone https://github.com/... at build time). Cloning the hooks once at build leaves the runtime store warm and fully offline.

  • docker/common/install_precommit_hooks.sh (new): seeds a throwaway git repo with .pre-commit-config.yaml and runs pre-commit install-hooks into a fixed PRE_COMMIT_HOME, cloning every hook repo and building every hook env.
  • docker/Dockerfile.multi (tritondevel stage): ENV PRE_COMMIT_HOME=/opt/pre-commit-cache + a RUN that bind-mounts the config and invokes the script.

Design notes.

  • Fixed absolute PRE_COMMIT_HOME (not the default ~/.cache/pre-commit): hook virtualenvs bake absolute shebang paths, so build-time and runtime paths must be identical. Set once as an ENV so runtime pre_commit reads the same store.
  • tritondevel only, not devel: scopes the cache to the CI image, keeping it out of the shipped release image (so no ATTRIBUTIONS change — nothing new is redistributed).
  • pre-commit is still pip installed from requirements-dev.txt at test time; the build-time install only populates the store.

Caveat. The store is keyed by (repo, rev). A PR that bumps a hook rev: misses the cache for that repo and falls back to the current github-clone behavior for that PR only, until the dep image is rebuilt. A follow-up can add a build-time git config --global url.<internal-mirror>.insteadOf https://github.com/<owner>/ to close even that gap.

Test Coverage

Mechanics verified locally against pre-commit 4.6.2: after install-hooks populates PRE_COMMIT_HOME, a subsequent pre-commit run reuses the cache with zero network (no "Initializing environment" line). End-to-end validation requires a tritondevel image rebuild (the Build-Docker-Images helper job) and a jenkins/current_image_tags.properties repin; the resulting Run type check stage should show no "Initializing environment for https://github.com" line.

PR Checklist

  • Please check this after reviewing the above items as appropriate for this PR.

Pre-populate the pre-commit hook store at image-build time so the CI
type-check stage no longer clones hooks from github.com at test time.

pre-commit enumerates every remote repo in .pre-commit-config.yaml (cloning
each to read its hook manifest) before running the selected hook, so a single
'git fetch origin --tags' against github.com can fail the whole 'Run type
check' stage on test nodes without reliable github access. Build nodes do
reach github (see install_mooncake.sh / install_ucx.sh), so cloning the hooks
once at build time leaves the runtime store warm and offline.

- add docker/common/install_precommit_hooks.sh: seeds a throwaway git repo
  with the config and runs 'pre-commit install-hooks' into a fixed
  PRE_COMMIT_HOME
- tritondevel stage: set ENV PRE_COMMIT_HOME=/opt/pre-commit-cache and invoke
  the script (bind-mounting .pre-commit-config.yaml)

Cache is keyed by (repo, rev); a PR that bumps a hook rev misses the cache
until the dep image is rebuilt, falling back to the current github-clone
behavior for that PR only. CI-only: the cache lives in tritondevel, not in
the shipped release image, so no ATTRIBUTIONS change.

Signed-off-by: Brian Nguyen <brnguyen@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant