getLogs improvements #876

Activity

1. LukaszRozmej commented on Sep 7, 2026

   

   LukaszRozmej

   on Sep 7, 2026

   More actions

   1. Not sure how would this work on stateless endpoint. You do have reorg support through eth_subscribe and removed flag.
   2. I think there is work around it, but that still may be somewhat true that it is under-speced.
   3. This would need new endpoint with this feature (may be worth specing)
   4. Depends what do you mean by that. If you are doing long range scans (like millions of blocks), or are you querying very dense logs (only hundreds of blocks, but very large amount of logs returned). For former both Nethermind and Geth have optional index structures. Second is just somewhat limited of just how much data you can push through JSON-RPC.
   5. This is somewhat true - not only for this endpoint. Could be speced with 3 on new endpont. But can be mitigated by the client, first query current head, then query logs based on hash (or number).
2. sakulstra commented on Sep 7, 2026

   

   sakulstra

   on Sep 7, 2026

   Last edited by sakulstra

   Author

   More actions

   I mostly wanted to get the discussion started as the recent crypto projects I've been involved in eventually all pivoted away from pure rpc indexing, but to my surprise I basically did not find any issues addressing any of their problems except #617.

   I personally think it would be much better to have a new eth_getLogsV2 vs a bloated eth_getLogs. Also I doubt that pagination can be introduced within the existing eth_getLogs.

   1* A stateless endpoint could just return a NOT_CANONICAL_CHAIN error i assume? Re eth_subscribe imho is no answer to that problem: It only works for indexers that constantly consume live data. It only works with some rpcs. Often has different limitations regarding filter topics. Also it does not support fromHash or similar either, so with a broken stream you have kinda the same issue on continuity.

   4* sorry, that was unclear — i didn't mean node side scan performance. i meant json-rpc request/response is just the wrong shape for a backfill. you want to say "give me everything from here" once and have it streamed, rather than n round trips where you also have to guess n and handle provider specific errors(2).

   How i imagine an ideal api to look like(largely stolen from envio):

   Request

   {
     "fromBlockHash": "0x..", // <- for reorg safe start
     "toBlock": "latest",
     "address": ["0x.."],
     "topics": [],
     "fields": ["blockNumber", "address", "data"] // <- as in a lot of cases the client does not care about huge chunks of the data
   }

   response / stream chunk

   {
     "logs": [],
     "toBlock": "0x..", "toBlockHash": "0x..",
     "headBlock": "0x..", "headBlockHash": "0x.."
   }

   the server truncates wherever it likes, the client feeds toBlockHash back in as the next fromBlockHash until it catches up with head:

   • no range guessing and no provider specific error handling
   • an orphaned anchor is just an error
   • if implemented as a stream it's the same thing. Just that it doesn't stop, and a dropped connection resumes from the last toBlockHash.

   Probably is not as easy as that.
   I assume the cursor might need to be more complex so it can point "mid-block".
   Envio does some deduplication via block/txn fields which i personally like, just not so sure how complex the rpc native endpoint should be 🤷

   "field_selection": {
       "block": ["number", "timestamp", "hash"],
       "transaction": ["block_number", "transaction_index", "hash", "from", "to"]
    }
   
3. bomanaps commented on Sep 9, 2026

   

   bomanaps

   on Sep 9, 2026

   Contributor

   More actions

   The mid-block cursor you flagged connects directly to an open question in #855 where the cursor needs a block-global log index to work correctly and right now geth, erigon, and reth all return different values for that index so that needs to be settled before the cursor design can move forward.
4. sakulstra commented on Sep 22, 2026

   

   sakulstra

   on Sep 22, 2026

   Author

   More actions

   Seems like monad is doing sth on that front monad-crypto/MIPs#88