Gloas envelope processing merge containers #8384
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…upernode (sigp#8265) This PR adds backfill functionality to nodes switching to become a supernode or semi-supernode. Please note that we currently only support a CGC increase, i.e. if the node's already custodying 67 columns, switching to semi-supernode (64) will have no effect. From @eserilev > if a node's cgc increases on start up, we just need two things for custody backfill to do its thing > > - data column custody info needs to be updated to reflect the cgc change > - `CustodyContext::validator_registrations::epoch_validator_custody_requirements` needs to be updated to reflect the cgc change - [x] Add tests - [x] Test on devnet-3 - [x] switch to supernode - [x] switch to semisupernode - [x] Test on live testnets - [x] Update docs (functions) Co-Authored-By: Jimmy Chen <[email protected]>
N/A We were publishing columns all columns that we didn't already have in the da cache when reconstructing. This is unnecessary outbound bandwidth for the node that is supposed to sample fewer columns. This PR changes the behaviour to publish only columns that we are supposed to sample in the topics that we are subscribed to. Co-Authored-By: Pawan Dhananjay <[email protected]>
We are seeing some crazy IO utilisation on Holesky now that data columns have started to expire. Our previous approach of _iterating the entire blobs DB_ doesn't seem to be scaling. New blob pruning algorithm that uses a backwards block iterator from the epoch we want to prune, stopping early if an already-pruned slot is encountered. Co-Authored-By: Michael Sproul <[email protected]>
Open PRs to include for the release - sigp#7907 - sigp#8247 - sigp#8251 - sigp#8253 - sigp#8254 - sigp#8265 - sigp#8269 - sigp#8266 Co-Authored-By: Jimmy Chen <[email protected]> Co-Authored-By: Jimmy Chen <[email protected]>
…#8124) Co-Authored-By: 0x19dG87 <[email protected]>
Testing non finalized checkpoint sync noticed this log that dumps blob data in Debug format to the logs. Log only block root and commitment of each blob Co-Authored-By: dapplion <[email protected]>
sigp#7756 introduces a logging issue, where the relevant log: https://github.com/sigp/lighthouse/blob/da5b2317205efc72b98cdc922289acefe3f76a13/beacon_node/network/src/network_beacon_processor/rpc_methods.rs#L380-L385 obtains the `block_root` from `slots_by_block_root.keys()`. If the `block_root` is empty (block not found in the data availability checker), then the log will not show any block root: `DEBUG BlobsByRoot outgoing response processed peer_id: 16Uiu2HAmCBxs1ZFfsbAfhSA98rUUL8Q1egLPb6WpGdKZxX6HqQYX, block_root: [], returned: 4` This PR revises to return the `block_root` in the request as a vector of block root Co-Authored-By: Tan Chee Keong <[email protected]>
Closes: - sigp#8249 New `/lighthouse/custody` API including: - [x] Earliest custodied data column slot - [x] Node CGC - [x] Custodied columns Co-Authored-By: Michael Sproul <[email protected]>
Co-Authored-By: Tan Chee Keong <[email protected]> Co-Authored-By: chonghe <[email protected]>
…RU_CAPACITY_NON_ZERO` (sigp#8234) - Renames `OVERFLOW_LRU_CAPACITY` to `OVERFLOW_LRU_CAPACITY_NON_ZERO` to follow naming convention of `STATE_LRU_CAPACITY_NON_ZERO` - Makes `OVERFLOW_LRU_CAPACITY_NON_ZERO` and `STATE_LRU_CAPACITY_NON_ZERO` private since they are only used in this module - Moves `STATE_LRU_CAPACITY` into test module since it is only used for tests Co-Authored-By: Kevaundray Wedderburn <[email protected]>
Testing non-finality checkpoint synced these logs showed up in my INFO grep and were noisy. INFO should only include the notifier and exceptional events. I don't see why the user would care about this info. Downgrade to debug Co-Authored-By: dapplion <[email protected]>
Co-Authored-By: kevaundray <[email protected]>
Co-Authored-By: Tan Chee Keong <[email protected]>
sigp#8012 Replace all instances of `VariableList::from` and `FixedVector::from` to their `try_from` variants. While I tried to use proper error handling in most cases, there were certain situations where adding an `expect` for situations where `try_from` can trivially never fail avoided adding a lot of extra complexity. Co-Authored-By: Mac L <[email protected]> Co-Authored-By: Michael Sproul <[email protected]> Co-Authored-By: Michael Sproul <[email protected]>
Use the recently published `compare_fields` and remove it from Lighthouse https://crates.io/crates/compare_fields Co-Authored-By: Mac L <[email protected]>
Currently the `eth2` crate lib file is a large monolith of almost 3000 lines of code. As part of the bosun migration we are trying to increase code readability and modularity in the lighthouse crates initially, which then can be transferred to bosun Co-Authored-By: hopinheimer <[email protected]> Co-Authored-By: hopinheimer <[email protected]>
Co-Authored-By: hopinheimer <[email protected]>
Partially addresses sigp#8248 Run the beacon chain, http and network tests only for recent forks instead of everything from phase 0. Also added gloas also to the recent forks list. I thought that would be a good way to know if changes in the current fork affect future forks. Not completely sure if we should run for future forks, but added it so that we can discuss here. Co-Authored-By: Pawan Dhananjay <[email protected]> Co-Authored-By: Jimmy Chen <[email protected]>
Update the EF spec tests to v1.6.0-beta.1 There are a few new light client tests (which we pass), and some for progressive containers, which we haven't implemented (we ignore them). Co-Authored-By: Michael Sproul <[email protected]>
Fixes sigp#8268 Switch `est_time` from time until DA boundary slot, to time to finish total custody work from the original earliest data-column slot down to the DA boundary Co-Authored-By: PoulavBhowmick03 <[email protected]>
…rk (sigp#8259) Addresses this comment: sigp#8254 (comment) We're currently using `subscribe_all_data_column_subnets` here to subscribe to all subnets https://github.com/sigp/lighthouse/blob/522bd9e9c6ac167f2231525e937c9ebbcb86cf6e/beacon_node/lighthouse_network/src/types/topics.rs#L82-L92 But its unnecessary because the else path also works for supernode (uses `sampling_subnets` instead) The big diffs will disappear once sigp#8254 is merged. Co-Authored-By: Jimmy Chen <[email protected]>
Bump gas limit to 60M as part of Fusaka mainnet release. Co-Authored-By: Jimmy Chen <[email protected]> Co-Authored-By: Jimmy Chen <[email protected]>
Co-Authored-By: Tan Chee Keong <[email protected]>
Co-Authored-By: Eitan Seri- Levi <[email protected]>
During custody backfill sync there could be an edge case where we update CGC at the same time where we are importing a batch of columns which may cause us to incorrectly overwrite values when calling `backfill_validator_custody_requirements`. To prevent this race condition, the expected cgc is now passed into this function and is used to check if the expected cgc == the current validator cgc. If the values arent equal, this probably indicates that a very recent CGC occurred so we do not prune/update values in the `epoch_validator_custody_requirements` map. Co-Authored-By: Eitan Seri-Levi <[email protected]>
sigp#8311 Removes the `git_version` crate from `lighthouse_version` and implements git `HEAD` tracking manually. This removes the (mostly) broken dirty tracking but prevents spurious recompilation of the `lighthouse_version` crate. This also reworks the way crate versions are handled by utilizing workspace version inheritance and Cargo environment variables. This means the _only_ place where Lighthouse's version is defined is in the top level `Cargo.toml` for the workspace. All relevant binaries then inherit this version. This largely makes the `change_version.sh` script useless so I've removed it, although we could keep a version which just alters the workspace version (if we need to maintain compatibility with certain build/release tooling. ### When is a Rebuild Triggered? 1. When the build.rs file is changed. 2. When the HEAD commit changes (added, removed, rebased, etc) 3. When the branch changes (this includes changing to the current branch, and creating a detached HEAD) Note that working/staged changes will not trigger a recompile of `lighthouse_version`. Co-Authored-By: Mac L <[email protected]> Co-Authored-By: Michael Sproul <[email protected]>
sigp#8135 mainnet config PR: eth-clients/mainnet#11 Co-Authored-By: Eitan Seri-Levi <[email protected]> Co-Authored-By: Michael Sproul <[email protected]> Co-Authored-By: Tan Chee Keong <[email protected]>
The beacon API spec wasn't updated to use the Fulu definition of `dependent_root` for the proposer duties endpoint. No other client updated their logic, so to retain backwards compatibility the decision has been made to continue using the block root at the end of epoch `N - 1`, and introduce a new v2 endpoint down the track to use the correct dependent root. Eth R&D discussion: https://discord.com/channels/595666850260713488/598292067260825641/1433036715848765562 Change the behaviour of the v1 endpoint back to using the last slot of `N - 1` rather than the last slot of `N - 2`. This introduces the possibility of dependent root false positives (the root can change without changing the shuffling), but causes the least compatibility issues with other clients. Co-Authored-By: Michael Sproul <[email protected]>
…igp#8339) Custody backfill sync has a bug when we request columns from more than one peer per batch. The fix here ensures we wait for all requests to be completed before performing verification and importing the responses. I've also added an endpoint `lighthouse/custody/backfill` that resets a nodes earliest available data column to the current epoch so that custody backfill can be triggered. This endpoint is needed to rescue any nodes that may have missing columns due to the custody backfill sync bug without requiring a full re-sync. Co-Authored-By: Eitan Seri- Levi <[email protected]> Co-Authored-By: Eitan Seri-Levi <[email protected]> Co-Authored-By: Jimmy Chen <[email protected]> Co-Authored-By: Michael Sproul <[email protected]>
N/A Includes the following unmerged PRs: - sigp#8344 - sigp#8335 - sigp#8339 This PR should be merged after all above PRs are merged. Co-Authored-By: Jimmy Chen <[email protected]> Co-Authored-By: Jimmy Chen <[email protected]>
`beacon-chain-tests` is now regularly taking 1h+ on CI since Fulu fork was added. This PR attemtpts to reduce the test time by bringing down the number of blobs generated in tests - instead of generating 0..max_blobs, the generator now generates 0..1 blobs by default, and this can be modified by setting `harness.execution_block_generator.set_min_blob_count(n)`. Note: The blobs are pre-generated and doesn't require too much CPU to generate however processing a larger number of them on the beacon chain does take a lot of time. This PR also include a few other small improvements - Our slowest test (`chain_segment_varying_chunk_size`) runs 3x faster in Fulu just by reusing chain segments - Avoid re-running fork specific tests on all forks - Fix a bunch of tests that depends on the harness's existing random blob generation, which is fragile beacon chain test time on test machine is **~2x** faster: ### `unstable` ``` Summary [ 751.586s] 291 tests run: 291 passed (13 slow), 0 skipped ``` ### this branch ``` Summary [ 373.792s] 291 tests run: 291 passed (2 slow), 0 skipped ``` The next set of tests to optimise is the ones that use [`get_chain_segment`](https://github.com/sigp/lighthouse/blob/77a9af96de0f693127055e381ece3e98dceea0a8/beacon_node/beacon_chain/tests/block_verification.rs#L45), as it by default build 320 blocks with supernode - an easy optimisation would be to build these blocks with cgc = 8 for tests that only require fullnodes. Co-Authored-By: Jimmy Chen <[email protected]> Co-Authored-By: Jimmy Chen <[email protected]>
This is an optimisation targeted at Fulu networks in non-finality. While debugging on Holesky, we found that `state_root_at_slot` was being called from `prepare_beacon_proposer` a lot, for the finalized state: https://github.com/sigp/lighthouse/blob/2c9b670f5d313450252c6cb40a5ee34802d54fef/beacon_node/http_api/src/lib.rs#L3860-L3861 This was causing `prepare_beacon_proposer` calls to take upwards of 5 seconds, sometimes 10 seconds, because it would trigger _multiple_ beacon state loads in order to iterate back to the finalized slot. Ideally, loading the finalized state should be quick because we keep it cached in the state cache (technically we keep the split state, but they usually coincide). Instead we are computing the finalized state root separately (slow), and then loading the state from the cache (fast). Although it would be possible to make the API faster by removing the `state_root_at_slot` call, I believe it's simpler to change `state_root_at_slot` itself and remove the footgun. Devs rightly expect operations involving the finalized state to be fast. Co-Authored-By: Michael Sproul <[email protected]>
Remove all Windows-related CI jobs Co-Authored-By: antondlr <[email protected]>
Co-Authored-By: Tan Chee Keong <[email protected]> Co-Authored-By: Michael Sproul <[email protected]>
while working on this sigp#7892 @michaelsproul pointed it might be a good metric to measure the delay from start of the slot instead of the current `slot_duration / 3`, since the attestations duties start before the `1/3rd` mark now with the change in the link PR. Co-Authored-By: hopinheimer <[email protected]> Co-Authored-By: hopinheimer <[email protected]>
### Downgrade a non error to `Debug` I noticed this error on one of our hoodi nodes: ``` Nov 04 05:13:38.892 ERROR Error during data column reconstruction block_root: 0x4271b9efae7deccec3989bd2418e998b83ce8144210c2b17200abb62b7951190, error: DuplicateFullyImported(0x4271b9efae7deccec3989bd2418e998b83ce8144210c2b17200abb62b7951190) ``` This shouldn't be logged as an error and it's due to a normal race condition, and it doesn't impact the node negatively. ### Remove spammy logs This logs is filling up the log files quite quickly and it is also something we'd expect during normal operation - getting columns via EL before gossip. We haven't found this debug log to be useful, so I propose we remove it to avoid spamming debug logs. ``` Received already available column sidecar. Ignoring the column sidecar ``` In the process of removing this, I noticed we aren't propagating the validation result, which I think we should so I've added this. The impact should be quite minimal - the message will stay in the gossip memcache for a bit longer but should be evicted in the next heartbeat. Co-Authored-By: Jimmy Chen <[email protected]>
Another good candidate for publishing separately from Lighthouse is `sensitive_url` as it's a general utility crate and not related to Ethereum. This PR prepares it to be spun out into its own crate. I've made the `full` field on `SensitiveUrl` private and instead provided an explicit getter called `.expose_full()`. It's a bit ugly for the diff but I prefer the explicit nature of the getter. I've also added some extra tests and doc strings along with feature gating `Serialize` and `Deserialize` implementations behind the `serde` feature. Co-Authored-By: Mac L <[email protected]>
This compiles, is there any reason to keep `ecdsa`? CC @jxs Co-Authored-By: Michael Sproul <[email protected]>
Self hosted GitHub Runners review and improvements local testnet workflow now uses warpbuild ci runner Co-Authored-By: lemon <[email protected]> Co-Authored-By: antondlr <[email protected]>
Use the recently published `sensitive_url` and remove it from Lighthouse Co-Authored-By: Mac L <[email protected]>
Fixes sigp#7001. Mostly mechanical replacement of `derivative` attributes with `educe` ones. ### **Attribute Syntax Changes** ```rust // Bounds: = "..." → (...) #[derivative(Hash(bound = "E: EthSpec"))] #[educe(Hash(bound(E: EthSpec)))] // Ignore: = "ignore" → (ignore) #[derivative(PartialEq = "ignore")] #[educe(PartialEq(ignore))] // Default values: value = "..." → expression = ... #[derivative(Default(value = "ForkName::Base"))] #[educe(Default(expression = ForkName::Base))] // Methods: format_with/compare_with = "..." → method(...) #[derivative(Debug(format_with = "fmt_peer_set_as_len"))] #[educe(Debug(method(fmt_peer_set_as_len)))] // Empty bounds: removed entirely, educe can infer appropriate bounds #[derivative(Default(bound = ""))] #[educe(Default)] // Transparent debug: manual implementation (educe doesn't support it) #[derivative(Debug = "transparent")] // Replaced with manual Debug impl that delegates to inner field ``` **Note**: Some bounds use strings (`bound("E: EthSpec")`) for superstruct compatibility (`expected ','` errors). Co-Authored-By: Javier Chávarri <[email protected]> Co-Authored-By: Mac L <[email protected]>
…into gloas-containers
ethDreamer
merged commit e354038
into
sigp:gloas-envelope-processing
20 of 25 checks passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
15 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
merging latest
gloas-containers