Repository navigation
[DS-4501] BlobPump and state transition staleness age guards, record compression - #280
Merged
Merged
Conversation
Use QuorumByzQuorum for ObservationQuorum and the StateTransition invariant through one helper. Move the KV-free observation checks into fetchBlobsDecodeAndValidateObservation, shared by ValidateObservation and StateTransition, and fail the round on an observation that does not pass them instead of dropping it. Log requested and observed stream counts for a parked blob snapshot.
product-security-plaid-production
Bot
requested a review
from yaroslav-glukhov-chainlink
October 28, 2026 20:01
Carry the seqNr a blob pump cycle started in inside the blob payload (LLOBlobPayloadProto) and keep the agreed observation timestamps of the last seven rounds in the hot state. Per stream, aggregate only the contributions whose round age and time age are within MaxStreamValueAgeRounds, and fall back to every contribution when the fresh ones cannot reach the contribution floor. Estimate the round interval from the agreed timestamps, capped at a multiple of the configured one. Reject blobs without a cycle start, observations whose blobs name different cycles, and cycle starts ahead of the round. Count stale contributions and fallbacks in metrics and outcome telemetry.
Remove the pump's round gate and wall clock age gate with their knobs MaxSnapshotRounds, MaxBlobSnapshotAge and MaxRoundPeriod. Freshness is judged at the state transition from replicated data, so Take keeps one local check: the snapshot's blob must stay fetchable for BlobFetchMarginRounds past the round that references it. Bound the blob lifetime to [MinBlobLifetimeRounds, MaxBlobLifetimeRounds] and raise the default to BlobFetchMarginRounds + SnapshotReuseRounds, so a snapshot can be offered for the whole reuse horizon.
…arks Keep the newest parked snapshot and offer it to every round while its blob stays fetchable, so a cycle that outlasts a round or fails costs no round its stream values. A round that already referenced the snapshot waits up to inFlightWait for the cycle in flight before reusing it. Count reuses and report them in the observation log.
Expose cycles, misses and reuses as llo_blob_pump_*_total counters labelled by donID, next to the freshness metrics.
brunotm
force-pushed
the
DS-4501-blob-pump-state-transition-age
branch
from
October 29, 2026 08:33
8a667fc to
7ca1ba7
Compare
Share the blob payload framing as payloadcodec.go and apply it to the ReportsPlusPrecursor. Pin the zstd encoder options and freeze the compressed output in the precursor golden, since libocr signs a digest of the framed bytes and every oracle must produce the same ones. Keep admission bounding the raw precursor under the libocr limit. Export raw and framed precursor size as llo_precursor_size_bytes metric.
Frame c/defs, c/codecs, r/agg and the history header and chunks with the payload codec shared by blobs and the precursor. The fixed-width keys stay raw. libocr builds the state root from these bytes, so the pinned encoder options apply and a full history chunk golden freezes the compressed output. Treat an undecodable history frame as corrupt history so the window resets. Keep admission bounding the raw records under the per-key limit.
…rvations Round 1 accepts only empty observations since the byzantine quorum check. Report the stored hot state size per round.
Embed the c/defs record bytes instead of the decoded definitions, so StateTransition no longer re-encodes them and Reports resolves them through the channel cache, decoding only on a miss. Keep the record in each ChannelGeneration so the bytes always match the definitions they were decoded from.
Encode single segment frames so small payloads carry their content size, and reject frames that omit it. Size the decode buffer to the declared size instead of the whole budget, which small KV records hit every round.
Add a trimmed copy of klauspost/compress v1.20.3 zstd encoder.
denis-chernov-smartcontract
approved these changes
Nov 2, 2026
brunotm
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.