feat(genesis-writer): rebuild rewards from table state, not replayed transactions - #504
Merged
Merged
Conversation
…transactions Reward pools and rewards were migrated by scanning ~66M rows of core_transactions for reward transactions by protobuf wire tag, so the original signed bytes could be replayed verbatim. Once the transactions are signed fresh, those bytes have no role — and core_reward_pools and core_rewards already carry every field a new transaction needs. Two SELECTs against two small tables replace the scan, and with it the chunked block_id windows, the false-positive confirmation, the wire-tag constants, and the xmin-horizon care that reading a live production validator demanded. The legacy-to-modern conversion path goes too: there is nothing to convert when the transactions are built from table state. Both signatures are real. The envelope is signed by one of the pool's own authorities, satisfying the signer-membership check, and rm_owner_signature is ed25519 over the same body by the reward manager keypair. Both derive from operator-supplied launchpad secrets, taken from the environment rather than flags because a flag reaches argv. Two secrets are required because the pool set spans a rotation: reward managers were initialized under the original secret and cannot move, while current claim authorities derive from its replacement. Each pool is matched against both generations and a pool neither derives stops the run rather than being emitted with a key that controls nothing. That generational split is also how a phantom pool is identified. A pool whose reward manager only derives from the rotated secret, for a mint that also has an original-generation pool, names a Solana account that was never created — reward creation re-derived the manager after the secret changed. Emitting a create for it would make that dangling reference canonical, so it is dropped and its rewards move to the manager their mint actually has. The qualifier matters: a mint launched after the rotation legitimately has a rotated-generation manager and keeps it. Rewards are verified for functional equivalence rather than row equality: the offer (reward_id, name, amount) is preserved, every reward names a reward manager a pool covers, and both signatures verify the way the validator checks them. Tests go red against the previous behavior — disabling phantom detection fails with the phantom pool listed among those to create. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The writer inserts blocks straight into postgres and never runs them through ABCI, so anything FinalizeBlock would have written has to be written here instead. That is already true of the auth tables, which projectBlockAuthState handles. core_reward_pools and core_rewards are core tables populated the same way, and nothing was projecting them. Without this a migrated chain carries reward transactions in core_transactions while both reward tables stay empty, and nothing downstream repairs it: the bootstrap node treats those blocks as already committed, and other nodes state-sync from its tables. Entity tables are unaffected because the ETL materializes those by reading the chain; these are not ETL tables. ProjectMigrationRewardState reproduces the rows finalize writes, including core_rewards.address, which is derived from (txhash, chain id, height, message index) and so can only be computed once a transaction has a place in a block — not at synthesis time. The sender is recovered rather than assumed, because it is API-visible and must be the address that actually signed. Admission control is deliberately not re-run: signature, deadline and authorization were settled when the writer built these transactions. The tests assert ROWS. A test that counts transactions cannot detect this at all — the transactions are present either way, which is exactly why the gap was easy to miss. Verified red: stubbing the projection to a no-op fails with "the pool row must exist; transactions alone are not migrated state". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The writer read GENESIS_LAUNCHPAD_SECRET / GENESIS_LAUNCHPAD_SECRET_ROTATED while rewards_cli's rotate-launchpad-secret reads LAUNCHPAD_OLD_SECRET / LAUNCHPAD_NEW_SECRET. Four names for the same two secrets, with identical meanings, across two tools an operator uses in the same sitting. Use the rewards_cli spelling in both, and LAUNCHPAD_MINTS to match. Both secrets are now required up front rather than the rotated one being optional. Either alone derives half the key material -- the original owns the reward managers, the rotated one owns the authorities the pools currently carry -- and a run missing one previously failed much later with a per-pool "no mint derives this reward manager", which reads like an incomplete mint list rather than a missing secret. Failing at load names the variable instead. Neither set at all still returns no keys, since whether that is fatal depends on whether rewards are being migrated, and the caller already decides that. Verified red: tolerating a missing secret fails with "setting only LAUNCHPAD_OLD_SECRET was accepted; it derives half the key material". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Reward pools and rewards were migrated by scanning ~66M rows of
core_transactionsfor reward transactions by protobuf wire tag, so the original signed bytes could be replayed verbatim.Once the transactions are signed fresh, those bytes have no role — and
core_reward_poolsandcore_rewardsalready carry every field a new transaction needs:CreateRewardPoolrewards_manager_pubkey,authoritiescore_reward_poolsCreateRewardreward_id,name,amount,rewards_manager_pubkeycore_rewardsTwo
SELECTs replace the scan, and with it the chunkedblock_idwindows, the false-positive confirmation, the wire-tag constants and their tests, and the xmin-horizon care that reading a live production validator demanded. The legacy-to-modern conversion path goes too — there is nothing to convert when transactions are built from table state.Columns deliberately not read:
address,id,index,tx_hash,sender,raw_message,block_height, timestamps. The chain derives all of them.senderbecomes whichever key signs here, which is intended.Signatures are real
validateCreateRewardPool's signer-membership checkrm_owner_signature— ed25519 over the same body by the reward manager keypairBoth derive from operator-supplied launchpad secrets, read from the environment rather than flags because a flag reaches
argvand is kept in shell history.Why two secrets
The pool set spans a secret rotation. Reward managers are initialized once on Solana and cannot move, so they stay bound to the secret that created them, while current claim authorities derive from the replacement. Each pool is matched against both generations, and a pool that neither derives stops the run rather than being emitted with a key that controls nothing.
That generational split is also how a phantom pool is identified. A pool whose reward manager derives only from the rotated secret, for a mint that also has an original-generation pool, names a Solana account that was never created — reward creation re-derived the manager after the secret changed. Emitting a create for it would make that dangling reference canonical on the new chain, so it is dropped and its rewards move to the manager their mint actually has.
The qualifier is load-bearing: a mint launched after the rotation legitimately has a rotated-generation manager, it does exist on Solana, and dropping it would silently discard that mint's rewards.
Verification
Rewards are checked for functional equivalence, not row equality — this migration deliberately improves on the source state rather than reproducing it:
reward_id,name,amount) is preserved exactlyrm_owner_signatureverifies viaed25519.Verifyagainst the reward manager pubkey itselfProtoRecover, is a member of the pool's authoritiesVerified red against the previous behavior: disabling phantom detection fails with the phantom pool listed among those to create, and fails the remap assertion.
Operator-facing changes
--core-cmt-home,--core-scan-chunk,--core-scan-dry-runremoved.--core-dsnis the only reward source.--launchpad-mints(file, one mint per line). Required for rewards: every launchpad key is a function of(secret, mint), so the secrets alone derive nothing.GENESIS_LAUNCHPAD_SECRETandGENESIS_LAUNCHPAD_SECRET_ROTATED.🤖 Generated with Claude Code