feat(solana-indexer): PR 2 — add types module scaffolding #4506
feat(solana-indexer): PR 2 — add types module scaffolding #4506tilacog wants to merge 23 commits into
types module scaffolding #4506Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces the solana-indexer crate, which defines domain types for a Solana settlement indexer, including channel messages, commitment tracking, dead-letter handling, errors, events, metrics, recovery flows, and transaction helpers. A high-severity issue was identified in channel.rs where the PartialEvent struct is missing the half: PartialHalf field described in its documentation, which is required for the watchdog to reconstruct events. Adding this field also means PartialEvent must derive Clone instead of Copy due to heap-allocated types in PartialHalf.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| #[derive(Debug, Clone, Copy)] | ||
| pub struct PartialEvent { | ||
| /// Slot the partial was observed at. | ||
| pub slot: u64, | ||
| /// Transaction signature the partial corresponds to. | ||
| pub signature: Signature, | ||
| } |
There was a problem hiding this comment.
The doc comment for PartialEvent states that "each delivery carries the half that just landed," but the struct definition is missing the half: PartialHalf field. Without this field, the watchdog cannot receive or hold the actual data halves to match and reconstruct the full event. Additionally, since PartialHalf contains heap-allocated Box types, PartialEvent cannot derive Copy and should only derive Debug, Clone.
| #[derive(Debug, Clone, Copy)] | |
| pub struct PartialEvent { | |
| /// Slot the partial was observed at. | |
| pub slot: u64, | |
| /// Transaction signature the partial corresponds to. | |
| pub signature: Signature, | |
| } | |
| #[derive(Debug, Clone)] | |
| pub struct PartialEvent { | |
| /// Slot the partial was observed at. | |
| pub slot: u64, | |
| /// Transaction signature the partial corresponds to. | |
| pub signature: Signature, | |
| /// The half that just landed. | |
| pub half: PartialHalf, | |
| } |
types module scaffolding
Local tombi was 0.9.0; CI uses 1.1.0 which produces different output.
|
This pull request has been marked as stale because it has been inactive a while. Please update this pull request or it will be automatically closed. |
| #[error("replay window exceeded")] | ||
| ReplayWindowExceeded { | ||
| /// The slot the subscriber attempted to resume from. | ||
| attempted_slot: u64, | ||
| /// The earliest slot the provider can still serve. | ||
| earliest_replayable_slot: u64, | ||
| }, |
There was a problem hiding this comment.
aren't you losing the slot u64's here?
There was a problem hiding this comment.
Thanks, I was so focused on the types that I completely forgot to include the fields in the error message. Fixed in bfb4dcf.
| /// String label used in `solana.dead_letter.reason`. | ||
| pub fn as_str(self) -> &'static str { |
There was a problem hiding this comment.
this would be more accurate, i see "as_str" like a "const to_string" and I'd expect it to return the same as the label itself e.g. DecoderError -> "DecoderError"
| /// String label used in `solana.dead_letter.reason`. | |
| pub fn as_str(self) -> &'static str { | |
| /// String label used in `solana.dead_letter.reason`. | |
| pub fn as_label(self) -> &'static str { |
| /// String label used in `solana.*` `commitment` columns. | ||
| pub fn as_str(self) -> &'static str { | ||
| match self { | ||
| Self::Confirmed => "confirmed", | ||
| Self::Finalized => "finalized", | ||
| Self::RolledBack => "rolled_back", | ||
| } | ||
| } |
There was a problem hiding this comment.
| /// String label used in `solana.*` `commitment` columns. | |
| pub fn as_str(self) -> &'static str { | |
| match self { | |
| Self::Confirmed => "confirmed", | |
| Self::Finalized => "finalized", | |
| Self::RolledBack => "rolled_back", | |
| } | |
| } | |
| /// String label used in `solana.*` `commitment` columns. | |
| pub fn as_label(self) -> &'static str { | |
| match self { | |
| Self::Confirmed => "confirmed", | |
| Self::Finalized => "finalized", | |
| Self::RolledBack => "rolled_back", | |
| } | |
| } |
| use {crate::types::Signature, solana_sdk::pubkey::Pubkey}; | ||
|
|
||
| /// On-chain commitment of a transaction or row. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq)] |
There was a problem hiding this comment.
Do we actually need all these now? IMO we should just start with debug + clone (and maybe copy i guess) but the eq's will make more sense when actually used
There was a problem hiding this comment.
You're right, they were added "just in case," but future PRs only use this type in match expressions, so the derives aren't needed.
Removed in 0fef0ed
| pub enum PartialHalf { | ||
| /// Transaction-update half. | ||
| Tx(Box<SubscribeUpdateTransactionInfo>), | ||
| /// Account-update half. | ||
| Account(Box<SubscribeUpdateAccountInfo>), | ||
| } |
There was a problem hiding this comment.
why do these need to be boxed?
There was a problem hiding this comment.
They are big types, and clippy caught that in its large_enum_variant lint.
SubscribeUpdateAccountInfo: 128 bytesSubscribeUpdateTransactionInfo: 520 bytes (determines the enum size)
Mind that those values are in the hot path, so we expect them to be moved/copied a lot.
| /// at. The org file names the channel payload "Event"; the spec defines that | ||
| /// type as `StreamUpdate`, and that is what this crate uses. |
There was a problem hiding this comment.
Please update those comments. AI always writes doc comments which reference context that is not present in the PR or the final code base.
There was a problem hiding this comment.
Thanks for catching that, it shouldn't have been in there.
Removed in f57158f.
I do want to move the spec into the repo eventually, but I haven't landed on the right format yet.
| /// An account-update slot message. | ||
| Account { | ||
| /// Slot the message was observed at. | ||
| slot: u64, |
There was a problem hiding this comment.
Is slots solana lingo for blocks?
There was a problem hiding this comment.
Precisely.
A slot can be seen as a spot reserved for a block, for which the validator can include a block, or not.
More details here.
| //! Commitment-tracking types: confirmation state, signature status, and the row | ||
| //! shapes consumed by the finalization worker. |
There was a problem hiding this comment.
Maybe an explainer on what a commitment is in this context is more helpful.
There was a problem hiding this comment.
I've expanded the module docs in 399c2ec to give more context.
|
|
||
| use {crate::types::Signature, solana_sdk::pubkey::Pubkey}; | ||
|
|
||
| /// On-chain commitment of a transaction or row. |
There was a problem hiding this comment.
Does "row" mean a DB row here? In that case it seems like this should maybe not live in the fundamental "types" directory.
There was a problem hiding this comment.
Yeah, it's a DB row.
I figured it was fine to keep this type here since it's only meant for the finalization worker component.
It will be used only to bump the commitment level on most rows in the database (every solana.* row comes from a transaction).
|
|
||
| /// On-chain commitment of a transaction or row. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq)] | ||
| pub enum Commitment { |
There was a problem hiding this comment.
commitment needs a better explanation here. It sounds like this is just the outcome of a tx but it's impossible to tell based on the existing information.
There was a problem hiding this comment.
I've expanded the comments in 72146c4 to explain a bit about each commitment level
| /// Transaction signature. | ||
| tx_signature: Signature, | ||
| /// Slot the settlement was observed at. | ||
| slot: u64, |
There was a problem hiding this comment.
This code could be more self explanatory with Slot(u64) and observed_at: Slot.
There was a problem hiding this comment.
Agreed.
Introduced the Slot type in 57b5c76
| /// Transaction signature. | ||
| tx_signature: Signature, | ||
| /// Index of the instruction within the transaction. | ||
| ix_index: u8, |
There was a problem hiding this comment.
Is the tx index used often enough that a new type would make sense for this?
There was a problem hiding this comment.
I don't think so.
That info is mostly useful for auditing.
The SolverInteraction variant is still something we want to keep track of and index, but we don't really have a way to interpret it since it's basically an arbitrary instruction.
The upside is that, if we ever decide to try to retroactively decode a subset of those, we can refer to this info.
| /// Real owner of the order. | ||
| real_owner: Pubkey, | ||
| /// Order UID. | ||
| order_uid: [u8; 32], |
There was a problem hiding this comment.
GIven that intent hash and order uid have the same data "shape" they should definitely get their own separate types.
There was a problem hiding this comment.
I've introduced a new OrderUid type in c7936f4 to represent both intent_hash and order_uid fields, since they represent the same concept.
| OrderEnabled { | ||
| /// Custodial PDA. | ||
| custodial_pda: Pubkey, | ||
| /// Address that enabled the order. |
There was a problem hiding this comment.
What does "enabled" here mean? Is this a contract that calls a function to activate the order or perhaps an account that pays for the gas or something else entirely?
There was a problem hiding this comment.
I've expanded the doc-comments in 271539d to clarify those roles.
You intuition is correct, the enabler is an unprivileged actor that subsidizes the the order creation.
| #[metric(subsystem = "solana_indexer")] | ||
| pub struct Metrics { | ||
| /// Slots between the chain tip and the indexed watermark. | ||
| pub indexer_lag_slots: GenericGauge<AtomicI64>, |
There was a problem hiding this comment.
Does it make sense that a lag gets a signed integer gauge?
Same applies to other metrics which all seem to work just fine with unsigned ints.
There was a problem hiding this comment.
That's true, it was an oversight from my part.
After considering this other review comment, I decided to not include the metrics in this PR, but I'll keep the signing in mind when reintroducing them.
types module scaffoldingtypes module scaffolding
types module scaffoldingtypes module scaffolding
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
… DeadLetterReason
No usage site compares Commitment values with == — all code uses match.
squadgazzz
left a comment
There was a problem hiding this comment.
LGTM in overall. Any discrepancies can be adjusted in further PRs.
| /// Prometheus metrics for the solana-indexer. | ||
| #[derive(prometheus_metric_storage::MetricStorage)] | ||
| #[metric(subsystem = "solana_indexer")] | ||
| pub struct Metrics { |
There was a problem hiding this comment.
Not sure if metrics need to be a part of this PR. Maybe it makes sense to introduce them only when actually used? Right now, it is hard to judge whether the types and labels are correctly chosen.
There was a problem hiding this comment.
That's a fair point. I've removed them in 6b2e8a1.
We'll add them in the follow up PRs, as needed.
| /// Transaction signature. | ||
| tx_signature: Signature, | ||
| /// Index of the instruction within the transaction. | ||
| ix_index: u8, |
There was a problem hiding this comment.
According to the docs, it should be u16. Which one should be used?
There was a problem hiding this comment.
Thanks, let's go w/ u16.
Fixed in 47f8edf
I don't think we'll ever go beyond 256 instructions, but I don't want to be caught by surprise here.
| #[derive(Debug, Clone)] | ||
| pub struct PdaSnapshot { | ||
| /// Order UID. | ||
| pub order_uid: [u8; 32], |
There was a problem hiding this comment.
As @MartinquaXD already suggested for other types, maybe we need an OrderUid type here as well.
There was a problem hiding this comment.
I've introduced a new OrderUid type in c7936f4 to represent both intent_hash and order_uid fields, since they represent the same concept.
Switches raw/serialized byte fields from Vec<u8> to bytes::Bytes so cheap clones are available from the start. Fields changed: AccountInfo.data, DeadLetterEntry.raw_bytes, ResolvedInstruction.data. ResolvedInstruction.accounts stays Vec<u8> because it stores account indices, not opaque bytes.
Introduces `types::Slot`, a thin newtype wrapper around `u64`, and replaces the bare `u64` slot fields in the event, channel, transaction, commitment, and dead-letter types with it.
…fiers Add a single `OrderUid(pub [u8; 32])` newtype in `types/mod.rs` and use it in place of raw `[u8; 32]` for every order-identifier field across `TradeDelta`, the settlement-program lifecycle events, the SolFlow custody events, and `PdaSnapshot`.
- Change all domain types from pub to pub(crate) — these are internal implementation details that shouldn't leak into the public API - Add #![expect(dead_code)] to each types module file to suppress warnings during bootstrap phase when not all types are wired up yet - Add #[allow(clippy::enum_variant_names)] on SolFlowEvent to suppress false positive from semantically correct variant names
…dules Move Slot to types/slot.rs and OrderUid to types/order.rs, leaving types/mod.rs as pure module declarations + the Signature re-export. Consistent with the crate's one-concept-per-file pattern.
Spec §9 Domain event taxonomy types SolverInteraction.ix_index as u16. Bump both the event field (events.rs) and the upstream decoder view (ResolvedInstruction.ix_index) to u16 for consistent typing through the pipeline; the single downcast from the grpc u32 lands at the future construction site.
Description
Adds the type definitions for the
types/module listed in the crate-layout spec.This is PR 2 in the sequence; later PRs (traits (PR 3), then component structs (PR 4)) consume these types.
Changes
Added
crates/solana-indexer/src/types/with one file per module from the spec:wire.rs: wire-format types, for the Yellowstone gRPC protobuf typeschannel.rs: payload types for messages exchanged between the Ingester and Decoder componentevents.rs: domain event taxonomycommitment.rs: types for tracking Solana commitment levelsdead_letter.rs: dead-letter primitivesrecovery.rs: recovery stateerrors.rs: error types for this cratetx.rs: transaction contextDeclared
pub mod types;inlib.rsso the module compiles.How to test
cargo check -p solana-indexercargo +nightly fmt --all -- --checkcargo clippy --locked --workspace --all-features --all-targets -- -D warningsNo unit tests are included: the modules contain only type definitions, with no logic to exercise until the traits in PR 3 are introduced.