Repository navigation
feat(blocks): record every block since genesis - #16
Merged
Merged
Conversation
…ep, rather than stall
…, recheck below the floor alone
… gap and the chain reset
…ch refusal and the pause reset on a chain change
…k test by its outcome
…d the batch of one
…an show its oldest block
…nswer between refusals and its report
…ating report, and word the pause exactly
…f one never completes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
gnockpit records every block since genesis in its SQLite database, at about 100 bytes a block (about 130 MB for onyx's 1.25M blocks). It fills the history back from the first
-rpcin paced JSON-RPC batches, then follows the chain head. Each block keeps:What it changes:
/api/stats' windows) come from this history. They are complete as soon as the fill covers the window, rather than counted from gnockpit's start. The forward-only, 31-day signing history is removed. The live 100-block window counts a precommit for another block as missed, as the history does./api/historygives how far the history reaches and the fill rate;?from=&to=the blocks of up to 1,000 heights with their validator sets;?days=Nthe daily vote-delay histograms.--history-paceand--history-batchset the fill's load on the source, per network.Why: the validator page and the ranking (next steps) are computed from these facts.
Changes
RPC: commits read in JSON-RPC batches through the single-call path; validator sets with priorities; block events. The live window keeps final commits across refreshes, forgets an earlier chain's, and counts a precommit for another block as missed. —
8cdb764,65507e2,ffb962d,d908213,9a2a1de,51b85bd,4989e61Block history: a port of tm2's proposer priorities, the per-block record, the store and the fill loop. —
5f2dafa,321e080,b9bee92,16dd053,9c9a568Compact store, about 11× smaller: each validator address stored once, keyed tables without row ids and no secondary index; a window's misses counted from per-day counters, its first block found by bisecting heights. —
c151134,b55967f,ae27f4f,6465d36,838a35fMissed-block counts from the history, and
/api/historywith every fact the history keeps. —7aa57db,644e762,92b3a3a,314908eThe fill flags. —
8a76bd6A fill that holds behind load balancers and rate limits:
Retry-Afteron every read; catching up after downtime goes at the pace;—
7955682,e4609e3,1ac89dd,518fb52,08f7864,e437a55,16e45ac,ef0057b,9fb499b,85156c4,b04535a,879cdd1,d34af06,6e1215d,bc699fe,4312b7eThe sync indicator in Network State's header, with a server-side fill rate. —
4e5f691,2c03e25,f28ddc4,d29759aDocs: README (block history, flags,
/api/history) and AGENT.md. —fe35a22,8390283,fa4b0edTesting
go test ./...covers:EXPLAIN QUERY PLAN) and its size;On first start, the old
signing_blocksandsigning_missestables are dropped and the fill begins: at the defaults, at most about 100 blocks a second, so 1.6M blocks take at least 4.5 hours.--db-pathshould be persistent.Screenshots