feat: blockBrokers can control block validation #285
Conversation
SgtPooki
left a comment
SgtPooki
left a comment
There was a problem hiding this comment.
self review.. things look good.
| validateFn: async (block: Uint8Array): Promise<void> => { | ||
| await validateFn(block) | ||
| blocksWereValidated = true | ||
| } | ||
| }) | ||
|
|
||
| // verify block | ||
| const hash = await hasher.digest(block) | ||
|
|
||
| if (!uint8ArrayEquals(hash.digest, cid.multihash.digest)) { | ||
| throw new CodeError('Hash of downloaded block did not match multihash from passed CID', 'ERR_HASH_MISMATCH') | ||
| if (!blocksWereValidated) { | ||
| // the blockBroker either did not throw an error when attempting to validate the block | ||
| // or did not call the validateFn at all. We should validate the block ourselves | ||
| await validateFn(block) | ||
| } |
There was a problem hiding this comment.
We could have a callback method where we tell blockBrokers that they gave us an invalid block, and then wait for them to handle it however they see fit, but that would require us to do another try block, and ends up in some weird "back-and-forth" control scenario when blockProviders have complex retry flows.
We don't want to get in the business of handling those retry flows, so exposing a function they can call to determine whether blocks are valid or not prior to giving control back to us is the most simple & flexible method I can think of right now.
We can adjust if we see problems with this, but I think this is fairly flexible, and won't end up with duplicate validations (except where js-ipfs-bitswap is doing so internally)
SgtPooki
deleted the
fix/block-broker-testing-invert-blockValidation-control
branch
Built on top of #281 & #284
This PR adds the
validateFnoption toBlockRetriever.retrieveoptions and slightly modifieshow
raceBlockRetrieversinside ofNetworkedStorageis handled. Some callouts:validateFnto call when they retrieve blocks, and MAY call this method prior to returning the blockvalidateFn, whenvalidateFnand given MultihashHashers)retrievemethod (e.g. try other gateways, other peers/transports/etc, whatever they want)retrievemethod and otherBlockBrokerswill continue toraceBlockRetrieversas expectedvalidateFn, we will call it for them, and thenvalidateFnand given MultihashHashers)retrievemethod and otherBlockBrokerswill continue toraceBlockRetrieversas expectedretrievemethod inBitswapBlockBrokerwere modified so avalidateFncan be handled there, but no work towards that effort should be done until feat: stop automatically verifying bytes js-ipfs-bitswap#603 is handled.