Skip to content

3.0 snapshotter has gap in block files. integrity must catch such case #16525

@AskAlexSharov

Description

@AskAlexSharov

3.0 eth-mainnet snapshotter:

ubuntu@snapshotter-bm-e3-ethmainnet-n1:~/erigon$ go run ./cmd/erigon seg integrity --check=HeaderNoGaps --datadir=/erigon-data
INFO[08-10|04:38:27.588] logging to file system                   log dir=/home/ubuntu/.local/share/erigon/logs file prefix=erigon log level=info json=false
INFO[08-10|04:38:27.589] logging to file system                   log dir=/erigon-data/logs file prefix=erigon log level=info json=false
INFO[08-10|04:38:27.591] [db] open                                label=chaindata sizeLimit=1TB pageSize=4KB
INFO[08-10|04:38:27.905] [snapshots:block] Stat                   blocks=18.00M indices=18.00M alloc=460.4MB sys=486.0MB
INFO[08-10|04:38:28.755] [snapshots:caplin] Stat                  blocks=12.31M indices=12.31M
INFO[08-10|04:38:28.755] [snapshots:bor] Stat                     blocks=1 indices=1 alloc=1.5GB sys=1.6GB
INFO[08-10|04:38:32.079] [snapshots:history] Stat                 blocks=23.10M txNum2blockNum="1024=14760K,1536=19814K,1792=22202K,1856=22736K,1888=22982K,1904=23088K,1906=23101K,1907=23108K" txs=2.97B first_history_idx_in_db=23101521 alloc=4.3GB sys=4.4GB
INFO[08-10|04:38:32.079] [downloads]                              locked=true at="04 Aug 25 09:02 2025"
INFO[08-10|04:38:32.193] [integrity] snapshots are publishable
ubuntu@snapshotter-bm-e3-ethmainnet-n1:~/erigon$ go run ./cmd/integration print_stages --datadir /erigon-data --chain mainnet
INFO[08-10|04:38:44.128] logging to file system                   log dir=/erigon-data/logs file prefix=integration log level=info json=false
INFO[08-10|04:38:44.130] [db] open                                label=chaindata sizeLimit=1TB pageSize=4KB
INFO[08-10|04:38:44.130] [dbg] cfg                                syncCfg="{LoopThrottle:0s ExecWorkerCount:5 ReconWorkerCount:0 BodyCacheLimit:256MB BodyDownloadTimeoutSeconds:2 BreakAfterStage: LoopBlockLimit:0 ParallelStateFlushing:true UploadLocation: UploadFrom:earliest FrozenBlockLimit:0 ChaosMonkey:false AlwaysGenerateChangesets:false KeepExecutionProofs:false PersistReceiptsV1:0 PersistReceiptsCacheV2:true}"
INFO[08-10|04:38:44.256] [downloads]                              locked=true at="04 Aug 25 09:02 2025"
DBUG[08-10|04:38:44.267] [snapshots] see gap                      type=headers from=18000000
DBUG[08-10|04:38:44.267] [snapshots] see gap                      type=bodies from=18000000
DBUG[08-10|04:38:44.267] [snapshots] see gap                      type=transactions from=18000000
INFO[08-10|04:38:45.388] [snapshots:caplin] Stat                  blocks=12.31M indices=12.31M
INFO[08-10|04:38:47.681] [snapshots:blocks] Stat                  blocks=18.00M indices=18.00M alloc=4.5GB sys=4.8GB
INFO[08-10|04:38:47.681] [snapshots:bor] Stat                     blocks=1 indices=1 alloc=4.5GB sys=4.8GB
INFO[08-10|04:38:47.682] [snapshots:history] Stat                 blocks=23.10M txNum2blockNum="1024=14760K,1536=19814K,1792=22202K,1856=22736K,1888=22982K,1904=23088K,1906=23101K,1907=23108K" txs=2.97B first_history_idx_in_db=23101521 alloc=4.5GB sys=4.8GB
Note: prune_at doesn't mean 'all data before were deleted' - it just mean stage.Prune function were run to this block. Because 1 stage may prune multiple data types to different prune distance.

 			 stage_at 	 prune_at
OtterSync 		 23108295 	 0
Headers 		 23108296 	 0
PolygonSync 		 0 		 0
BlockHashes 		 23108296 	 0
Bodies 			 23108296 	 0
Senders 		 23108296 	 0
Execution 		 23108296 	 23108296
CustomTrace 		 0 		 0
Translation 		 0 		 0
TxLookup 		 23108296 	 23099976
Finish 			 23108296 	 0
--
prune distance: archive

blocks: segments=17999999, indices=17999999
blocks.bor: segments=0, indices=0
state.history: idx steps: 1.00, TxNums_Index(23108296,2979745030)

sequence: EthTx=2984922027

in db: first header 23099976, last header 23108296, first body 22895882, last body 23108296
--

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions