Skip to content

feat(chunk-post): fault-domain-aware quorum + soft-preferred fallback (B2)#806

Closed
vilenarios wants to merge 1 commit into
feat/ip-fault-domainfrom
feat/chunk-post-fault-domain-quorum
Closed

feat(chunk-post): fault-domain-aware quorum + soft-preferred fallback (B2)#806
vilenarios wants to merge 1 commit into
feat/ip-fault-domainfrom
feat/chunk-post-fault-domain-quorum

Conversation

@vilenarios

Copy link
Copy Markdown
Contributor

Stacked on #805 (ipFaultDomain) — base will retarget to develop once #805 merges.

What

Makes chunk fanout fault-domain-aware and removes the single-fault-domain tips SPOF: all five tip-*.arweave.xyz resolve into one 38.29.227.0/24, so today's hard CHUNK_POST_MIN_PREFERRED_SUCCESS_COUNT=2 means a tips outage hard-fails every chunk POST even when independent discovered peers accepted it. Both new behaviors are opt-in and default to the exact legacy quorum.

Changes

  • Distinct-domain counting — buckets each successful peer by IP /24·/48 (ipFaultDomain). CHUNK_POST_MIN_DISTINCT_DOMAINS (default 0 = off) adds a soft target among successes that degrades gracefully when the eligible peer set can't supply N (never hard-fails on scarcity).
  • Soft-preferred fallbackCHUNK_POST_PREFERRED_SOFT_FALLBACK (default false) lets a strong distinct-domain discovered quorum satisfy a POST only when no tip was eligible (tips down/over-queue) — never when tips merely failed, so tips stay the hard primary on a real outage only.
  • Centralized verdictevaluateChunkBroadcastVerdict is the single source of truth for "seeded sufficiently?"; the handler reads result.succeeded instead of recomputing thresholds (removes latent drift). With both flags at defaults it reduces exactly to the legacy rule — covered by a regression test.
  • Placement-evidence contract (plan §18)BroadcastChunkResult now carries distinctDomainCount / preferredEligibleCount / succeeded, and POST /chunk emits the X-AR-IO-Chunk-Placement-Domains header, so the bundler can consume where a chunk landed instead of collapsing the result to pass/fail. (Bundler-side consumption is a separate cross-repo proposal.)
  • Metricschunk_post_distinct_domains (histogram), chunk_post_preferred_shortfall_total{reason} (emitted even when the fallback makes the POST succeed → a tips outage is never silently masked), chunk_post_domain_shortfall_total{reason}.

Safety / rollout

  • Default-off: MIN_DISTINCT_DOMAINS=0 + PREFERRED_SOFT_FALLBACK=false ⇒ byte-identical to today's behavior (regression-tested).
  • Peer→domain bucketing is in-process (no hot-path DNS — the postChunk peer list is IP-literal after DNS resolution).
  • Soak gate before any default flip: a simulated tips outage now succeeds via fallback, chunk_post_distinct_domains rises, no regression with flags off.

Tests

config 20/20 · composite-client 24/24 (verdict logic exhaustively: default-off regression, fallback fires only on tips_unavailable, domain soft-degrade) · ip-utils 50/50 · lint clean.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Jun 30, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.51185% with 58 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.77%. Comparing base (c07d6db) to head (ce714b7).

Files with missing lines Patch % Lines
src/arweave/composite-client.ts 64.19% 58 Missing ⚠️
Additional details and impacted files
@@                   Coverage Diff                    @@
##           feat/ip-fault-domain     #806      +/-   ##
========================================================
- Coverage                 78.79%   78.77%   -0.02%     
========================================================
  Files                       132      132              
  Lines                     50189    50395     +206     
  Branches                   3803     3814      +11     
========================================================
+ Hits                      39548    39701     +153     
- Misses                    10592    10645      +53     
  Partials                     49       49              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Makes chunk fanout fault-domain-aware and removes the single-fault-domain
tips SPOF (all five tip-*.arweave.xyz resolve into one 38.29.227.0/24, so a
tips outage hard-fails every POST today). Both behaviors are opt-in and
default to the legacy quorum exactly.

- Distinct-domain counting: buckets each successful peer by IP /24 (v4) /48
  (v6) via ipFaultDomain; CHUNK_POST_MIN_DISTINCT_DOMAINS (default 0=off) adds
  a soft target among successes that degrades gracefully when the eligible
  peer set can't supply N (never hard-fails on scarcity).
- Soft-preferred fallback: CHUNK_POST_PREFERRED_SOFT_FALLBACK (default false)
  lets a strong distinct-domain discovered quorum satisfy a POST when NO tip
  was eligible (tips down/over-queue) — but never when tips merely failed,
  so tips stay the hard primary on-ranp on a real outage only.
- Centralized verdict: evaluateChunkBroadcastVerdict is the single source of
  truth for "seeded sufficiently?"; the handler reads result.succeeded instead
  of recomputing the thresholds (removes latent drift). With both flags at
  defaults it reduces EXACTLY to the legacy rule (regression test).
- Placement-evidence contract: BroadcastChunkResult now carries
  distinctDomainCount/preferredEligibleCount/succeeded, and POST /chunk emits
  X-AR-IO-Chunk-Placement-Domains — so the bundler can consume where a chunk
  landed instead of collapsing the result to pass/fail (see plan §18).
- Metrics: chunk_post_distinct_domains (histogram),
  chunk_post_preferred_shortfall_total{reason}, chunk_post_domain_shortfall_total{reason}.
  Preferred shortfall is emitted even when the fallback makes the POST succeed,
  so a tips outage is never silently masked.

Depends on #805 (ipFaultDomain). Default-off; soak before any default flip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XQPK4TXcVoXoyFp6sNW2Lr
@vilenarios

Copy link
Copy Markdown
Contributor Author

Superseded by #812 — consolidated into the feature/chunk-fanout integration branch (F + B1 + B2 + dashboard as one coherent PR). All commits from this branch are included there; closing to review as a unit.

@vilenarios vilenarios closed this Jul 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant