Repository navigation
Wait out longer CloudKit outages when stapling - #9230
Merged
Merged
Conversation
Apple's ticket-delivery endpoint failed stapling with NSURLErrorDomain -1005 for longer than retry-transient.sh's default three attempts (~90s of waiting) on the v10.0.4-beta.3 release run and again on the 2026-10-28 nightly, after notarization had already been accepted both times. Add a -b flag for the backoff base and give the stapler six attempts at a 60s base, so it rides out up to 15 minutes of CloudKit flakiness instead of failing the whole platform build. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approved
No active review issues remain for the stapling retry changes.
No open issues 🎉
What changed in this PR
This PR makes macOS release builds more resilient to CloudKit outages after notarization by extending retries for stapling and validation.
Changes:
- Adds configurable retry backoff with
-b, preserving the existing 30-second default. - Configures stapler operations for six attempts with 15 minutes of total waiting per operation.
- Rejects leading-zero values for attempts and backoff to avoid Bash octal arithmetic issues.
| File | Description |
|---|---|
scripts/ci/retry-transient.sh |
Adds configurable backoff, validates numeric options, and updates usage documentation. |
Makefile |
Sets stapler retries to six attempts with a 60-second base backoff. |
Review effort: Balanced
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Apple's CloudKit ticket-delivery endpoint has now failed
xcrun stapler staplewithNSURLErrorDomain -1005after notarization was already accepted twice in four days — thev10.0.4-beta.3release run (Oct 25, run 40008674087) and the Oct 28 nightly dispatch (run 40765793487). Both outages outlastedretry-transient.sh's default three attempts, which only wait ~90 seconds in total, so the whole macOS build failed and each release needed a manualgh run rerun --failed.scripts/ci/retry-transient.shgains a-b BACKOFFflag: the wait before attempt N+1 isN*BACKOFFseconds (default 30, the previously hardcoded value, so the gomobile call site is unchanged).STAPLER_RETRYbecomes-n 6 -b 60: six attempts waiting 60/120/180/240/300s between them — 15 minutes of patience for a staple/validate that is idempotent and already past notarization, instead of failing the platform build.-nand-b(a round-1 gate catch):$((...))arithmetic reads010as octal 8 and chokes on08, so-b 08would have broken the retry loop instead of waiting 8 seconds.Testing
bash -nclean; exercised the script directly: success passthrough, retry loop with-n 2 -b 1(1s elapsed, correct messages), non-matching-mfailure exits immediately with the command's code, and0/08/010/x/empty are rejected for both flags with exit 2.make -n notarize-darwinconfirms both staple and validate expand toretry-transient.sh -n 6 -b 60 -- xcrun stapler ....Risk
retry-transient.sh— the other call site (gomobile bind) passes neither-nnor-band keeps the exact previous behavior (3 attempts, 30s base).How to revert
Revert the commit; no state to undo.
Pre-PR review (local Codex gate)
round=2 range=origin/main...HEADCODEX GATE: verdict=approve critical=0 important=0 minor=0 preexisting=0 reviewed=573edcbbdbapprove(high reasoning effort; the gate read both call sites, the workflows that invoke the Makefile targets, and the option-parsing edge cases)scripts/ci/retry-transient.sh—-bvalidation accepted leading-zero values that$((...))reads as octal (-b 08would error,-b 010would wait 8s); both-nand-bnow reject leading zeros.AGENTS.mdadditions: none🤖 Generated with Claude Code
Summary by CodeRabbit