Ability to backfill state information

[thomasj02]

Users who do fast sync or snap sync, even when pruning is disabled, are only able to access state information after the pivot block. There's no way for a user to access information even one block before the pivot without syncing an archive node.

I'd like to propose a feature to backfill state information. From a UI perspective, the user can specify a block number N, and all state information for blocks N to the latest block are downloaded.

I'm a new geth developer, but I would be willing to help with implementing this feature if someone can provide guidance.

[s1na]

The issue is we can't download state entries for blocks before the pivot block. That's because other nodes on the network prune their state and won't serve us those entries. That said even with a full (non-archive) sync geth maintains "state checkpoints" every once in a while so that you can re-generate the state for block N by re-executing blocks from the last checkpoint.

[thomasj02]

Thanks, that makes sense. I assume that archive nodes or nodes that are keeping significant history are rare enough that it's just not practical?

Re-generating the state seems totally reasonable if a non-full user can get the state checkpoint data. Are the state checkpoints you're referring to the same thing as snapshots?

[s1na]

Thanks, that makes sense. I assume that archive nodes or nodes that are keeping significant history are rare enough that it's just not practical?

Yep pretty much infeasible AFAIK.

No snapshots are an accelerating data structure where you store only the leaves of the state trie and read them via single disk access. These checkpoints are blocks for which you have the full state trie and exactly which blocks you have is different based on a couple of factors and they're not the same between users (you can see your nodes checkpoints by debug_getAccessibleState, see #23646). So you can't really expect multiple nodes to be able to feed you state for the same old block.