mirror of
https://github.com/sigp/lighthouse.git
synced 2026-03-03 00:31:50 +00:00
b8072c5b7799f66ae7977e367b8ececdc2fb2682
58 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Michael Sproul
|
b8072c5b77 |
Gloas payload bid consensus (#8801)
- [x] Consensus changes for execution payload bids - [x] EF tests for bids (and `block_header` -- no changes required). Co-Authored-By: Michael Sproul <michael@sigmaprime.io> |
||
|
Michael Sproul
|
7e275f8dc2 |
Gloas envelope consensus and more operations tests (#8781)
- Implement new `process_execution_payload` (as `process_execution_payload_envelope`). - Implement new processing for deposit requests, including logic for adding new builders to the registry with index reuse. - Enable a bunch more operations EF tests (most of them except bid processing/payload attestations/etc which we don't have code for yet). Co-Authored-By: Michael Sproul <michael@sigmaprime.io> |
||
|
Michael Sproul
|
f8cfaa4251 |
Gloas consensus logic for attestations (#8760)
Co-Authored-By: Michael Sproul <michael@sigmaprime.io> |
||
|
Michael Sproul
|
7df2972d41 |
Run all ssz_static tests for Gloas (#8755)
- Ensure all ssz_static tests are running and passing for Gloas 🎉 - Refine file ignores for Gloas EF tests Co-Authored-By: Michael Sproul <michael@sigmaprime.io> Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com> |
||
|
Eitan Seri-Levi
|
39727aa406 |
Move KZG commitments from payload envelope to payload bid and spec alpha.2 (#8725)
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com> Co-Authored-By: Michael Sproul <michael@sigmaprime.io> Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com> Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com> |
||
|
Jimmy Chen
|
21cabba1a2 |
Updated consensus types for Gloas 1.7.0-alpha.1 (#8688)
Pulling out consensus type changes from #8677. This PR covers all type changes for spec 1.7.0-alpha.1 (except for `DataColumnSidecar` changes, which is covered in @eserilev's PR #8682) Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com> Co-Authored-By: Michael Sproul <michael@sigmaprime.io> |
||
|
Michael Sproul
|
f70c650d81 |
Update spec tests to v1.6.0-beta.1 (#8263)
Update the EF spec tests to v1.6.0-beta.1 There are a few new light client tests (which we pass), and some for progressive containers, which we haven't implemented (we ignore them). Co-Authored-By: Michael Sproul <michael@sigmaprime.io> |
||
|
Eitan Seri-Levi
|
c7492f1c27 |
Update to 1.6.0 alpha.6 spec (#7967)
Upgrade `rust_eth_kzg` library to `0.9` to support the new cell index sorting tests in `recover_cells_and_kzg_proofs` https://github.com/ethereum/consensus-specs/releases https://github.com/crate-crypto/rust-eth-kzg/compare/v0.8.1...v0.9.0 |
||
|
Barnabas Busa
|
2b33fe6620 |
Update to spec v1.6.0-alpha.5 (#7910)
- https://github.com/ethereum/consensus-specs/pull/4508 |
||
|
Jimmy Chen
|
b4704eab4a |
Fulu update to spec v1.6.0-alpha.4 (#7890)
Fulu update to spec [v1.6.0-alpha.4](https://github.com/ethereum/consensus-specs/releases/tag/v1.6.0-alpha.4). - Make `number_of_columns` a preset - Optimise `get_custody_groups` to avoid computing if cgc = 128 - Add support for additional typenum values in type_dispatch macro |
||
|
ethDreamer
|
2d9fc34d43 |
Fulu EF tests v1.6.0-alpha.0 (#7540)
Update to EF tests v1.6.0-alpha.0 |
||
|
Pawan Dhananjay
|
33e41f7249 |
Consensus spec tests beta4 (#7231)
N/A Run latest consensus spec tests on CI https://github.com/ethereum/consensus-specs/releases/tag/v1.5.0-beta.4 |
||
|
Michael Sproul
|
ff739d56be |
Fix light client merkle proofs (#7007)
Fix a regression introduced in this PR: - https://github.com/sigp/lighthouse/pull/6361 We were indexing into the `MerkleTree` with raw generalized indices, which was incorrect and triggering `debug_assert` failures, as described here: - https://github.com/sigp/lighthouse/issues/7005 - Convert `generalized_index` to the correct leaf index prior to proof generation. - Add sanity checks on indices used in `BeaconState::generate_proof`. - Remove debug asserts from `MerkleTree::generate_proof` in favour of actual errors. This would have caught the bug earlier. - Refactor the EF tests so that the merkle validity tests are actually run. They were misconfigured in a way that resulted in them running silently with 0 test cases, and the `check_all_files_accessed.py` script still had an ignore that covered the test files, so this omission wasn't detected. |
||
|
Michael Sproul
|
ceb5ecf349 |
Update EF tests to spec v1.5.0-beta.2 (#6958)
Update spec tests for recent v1.5.0-beta.2 release. There are no substantial changes for Electra and earlier, and the Fulu test updates to be made are tracked here: - https://github.com/sigp/lighthouse/issues/6957 - Add `SingleAttestation` SSZ tests - Add new `deposit_with_reorg` fork choice tests - Update tag to v1.5.0-beta.2 - Ignore Fulu tests |
||
|
Jimmy Chen
|
70194dfc6a |
Implement PeerDAS Fulu fork activation (#6795)
Addresses #6706 This PR activates PeerDAS at the Fulu fork epoch instead of `EIP_7594_FORK_EPOCH`. This means we no longer support testing PeerDAS with Deneb / Electrs, as it's now part of a hard fork. |
||
|
Michael Sproul
|
06e4d22d49 |
Electra spec changes for v1.5.0-beta.0 (#6731)
* First pass * Add restrictions to RuntimeVariableList api * Use empty_uninitialized and fix warnings * Fix some todos * Merge branch 'unstable' into max-blobs-preset * Fix take impl on RuntimeFixedList * cleanup * Fix test compilations * Fix some more tests * Fix test from unstable * Merge branch 'unstable' into max-blobs-preset * Implement "Bugfix and more withdrawal tests" * Implement "Add missed exit checks to consolidation processing" * Implement "Update initial earliest_exit_epoch calculation" * Implement "Limit consolidating balance by validator.effective_balance" * Implement "Use 16-bit random value in validator filter" * Implement "Do not change creds type on consolidation" * Rename PendingPartialWithdraw index field to validator_index * Skip slots to get test to pass and add TODO * Implement "Synchronously check all transactions to have non-zero length" * Merge remote-tracking branch 'origin/unstable' into max-blobs-preset * Remove footgun function * Minor simplifications * Move from preset to config * Fix typo * Revert "Remove footgun function" This reverts commit |
||
|
Eitan Seri-Levi
|
9d069a9588 |
Fix electra light client types (#6361)
* persist light client updates * update beacon chain to serve light client updates * resolve todos * cache best update * extend cache parts * is better light client update * resolve merge conflict * initial api changes * add lc update db column * fmt * added tests * add sim * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * fix some weird issues with the simulator * tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * test changes * merge conflict * testing * started work on ef tests and some code clean up * update tests * linting * noop pre altair, were still failing on electra though * allow for zeroed light client header * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * merge unstable * remove unwraps * remove unwraps * fetch bootstrap without always querying for state * storing bootstrap parts in db * mroe code cleanup * test * prune sync committee branches from dropped chains * Update light_client_update.rs * merge unstable * move functionality to helper methods * refactor is best update fn * refactor is best update fn * improve organization of light client server cache logic * fork diget calc, and only spawn as many blcoks as we need for the lc update test * resovle merge conflict * add electra bootstrap logic, add logic to cache current sync committee * add latest sync committe branch cache * fetch lc update from the cache if it exists * fmt * Fix beacon_chain tests * Add debug code to update ranking_order ef test * Fix compare code * merge conflicts * merge conflict * add better error messaging * resolve merge conflicts * remove lc update from basicsim * rename sync comittte variable and fix persist condition * refactor get_light_client_update logic * add better comments, return helpful error messages over http and rpc * pruning canonical non checkpoint slots * fix test * rerun test * update pruning logic, add tests * fix tests * fix imports * fmt * refactor db code * Refactor db method * Refactor db method * lc electra changes * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * add additional comments * testing lc merkle changes * lc electra * update struct defs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * fix merge * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-bootstrap * fix merge * linting * merge conflict * prevent overflow * enable lc server for http api tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * get tests working: * remove related TODOs * fix test lint * Merge branch 'persist-light-client-bootstrap' of https://github.com/eserilev/lighthouse into light-client-electra * fix tests * fix conflicts * remove prints * Merge branch 'persist-light-client-bootstrap' of https://github.com/eserilev/lighthouse into light-client-electra * remove warning * resolve conflicts * merge conflicts * linting * remove comments * cleanup * linting * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * pre/post electra light client cached data * add proof type alias * move is_empty_branch method out of impl * add ssz tests for all forks * refactor beacon state proof codepaths * rename method * fmt * clean up proof logic * refactor merkle proof api * fmt * Merge branch 'unstable' into light-client-electra * Use superstruct mapping macros * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * rename proof to merkleproof * fmt * Resolve merge conflicts * merge conflicts |
||
|
Michael Sproul
|
a732a87846 |
Remove TTD flags and safe-slots-to-import-* (#6489)
* Delete SAFE_SLOTS_TO_IMPORT_OPTIMISTICALLY * Update fork choice tests * Remove TTD related flags * Add deprecation warning * Remove more dead code * Delete EF on_merge_block tests * Remove even more dead code * Address Mac's review comments |
||
|
kevaundray
|
653126f42e |
chore: update and enable PeerDAS tests to latest spec test release (1.5.0-alpha-5) (#6312)
* enable DAS tests * update spec testing code * Update PeerDAS kzg tests input structures to latest spec. * Update `ef_tests` ignore files. |
||
|
Lion - dapplion
|
f75a2cf65b |
PeerDAS implementation (#5683)
* 1D PeerDAS prototype: Data format and Distribution (#5050) * Build and publish column sidecars. Add stubs for gossip. * Add blob column subnets * Add `BlobColumnSubnetId` and initial compute subnet logic. * Subscribe to blob column subnets. * Introduce `BLOB_COLUMN_SUBNET_COUNT` based on DAS configuration parameter changes. * Fix column sidecar type to use `VariableList` for data. * Fix lint errors. * Update types and naming to latest consensus-spec #3574. * Fix test and some cleanups. * Merge branch 'unstable' into das * Merge branch 'unstable' into das * Merge branch 'unstable' into das # Conflicts: # consensus/types/src/chain_spec.rs * Add `DataColumnSidecarsByRoot ` req/resp protocol (#5196) * Add stub for `DataColumnsByRoot` * Add basic implementation of serving RPC data column from DA checker. * Store data columns in early attester cache and blobs db. * Apply suggestions from code review Co-authored-by: Eitan Seri-Levi <eserilev@gmail.com> Co-authored-by: Jacob Kaufmann <jacobkaufmann18@gmail.com> * Fix build. * Store `DataColumnInfo` in database and various cleanups. * Update `DataColumnSidecar` ssz max size and remove panic code. --------- Co-authored-by: Eitan Seri-Levi <eserilev@gmail.com> Co-authored-by: Jacob Kaufmann <jacobkaufmann18@gmail.com> * feat: add DAS KZG in data col construction (#5210) * feat: add DAS KZG in data col construction * refactor data col sidecar construction * refactor: add data cols to GossipVerifiedBlockContents * Disable windows tests for `das` branch. (c-kzg doesn't build on windows) * Formatting and lint changes only. * refactor: remove iters in construction of data cols * Update vec capacity and error handling. * Add `data_column_sidecar_computation_seconds` metric. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Merge branch 'unstable' into das # Conflicts: # .github/workflows/test-suite.yml # beacon_node/lighthouse_network/src/types/topics.rs * fix: update data col subnet count from 64 to 32 (#5413) * feat: add peerdas custody field to ENR (#5409) * feat: add peerdas custody field to ENR * add hash prefix step in subnet computation * refactor test and fix possible u64 overflow * default to min custody value if not present in ENR * Merge branch 'unstable' into das * Merge branch 'unstable' into das-unstable-merge-0415 # Conflicts: # Cargo.lock # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/data_availability_checker/availability_view.rs # beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs # beacon_node/beacon_chain/src/data_availability_checker/processing_cache.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/network/src/network_beacon_processor/mod.rs # beacon_node/network/src/sync/block_lookups/tests.rs # crypto/kzg/Cargo.toml * Merge remote-tracking branch 'sigp/unstable' into das * Merge remote-tracking branch 'sigp/unstable' into das * Fix merge conflicts. * Send custody data column to `DataAvailabilityChecker` for determining block importability (#5570) * Only import custody data columns after publishing a block. * Add `subscribe-all-data-column-subnets` and pass custody column count to `availability_cache`. * Add custody requirement checks to `availability_cache`. * Fix config not being passed to DAChecker and add more logging. * Introduce `peer_das_epoch` and make blobs and columns mutually exclusive. * Add DA filter for PeerDAS. * Fix data availability check and use test_logger in tests. * Fix subscribe to all data column subnets not working correctly. * Fix tests. * Only publish column sidecars if PeerDAS is activated. Add `PEER_DAS_EPOCH` chain spec serialization. * Remove unused data column index in `OverflowKey`. * Fix column sidecars incorrectly produced when there are no blobs. * Re-instate index to `OverflowKey::DataColumn` and downgrade noisy debug log to `trace`. * DAS sampling on sync (#5616) * Data availability sampling on sync * Address @jimmygchen review * Trigger sampling * Address some review comments and only send `SamplingBlock` sync message after PEER_DAS_EPOCH. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Merge branch 'unstable' into das # Conflicts: # Cargo.lock # Cargo.toml # beacon_node/beacon_chain/src/block_verification.rs # beacon_node/http_api/src/publish_blocks.rs # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/protocol.rs # beacon_node/lighthouse_network/src/types/pubsub.rs # beacon_node/network/src/sync/block_lookups/single_block_lookup.rs # beacon_node/store/src/hot_cold_store.rs # consensus/types/src/beacon_state.rs # consensus/types/src/chain_spec.rs # consensus/types/src/eth_spec.rs * Merge branch 'unstable' into das * Re-process early sampling requests (#5569) * Re-process early sampling requests # Conflicts: # beacon_node/beacon_processor/src/work_reprocessing_queue.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/network/src/network_beacon_processor/rpc_methods.rs * Update beacon_node/beacon_processor/src/work_reprocessing_queue.rs Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Add missing var * Beta compiler fixes and small typo fixes. * Remove duplicate method. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Merge remote-tracking branch 'sigp/unstable' into das * Fix merge conflict. * Add data columns by root to currently supported protocol list (#5678) * Add data columns by root to currently supported protocol list. * Add missing data column by roots handling. * Merge branch 'unstable' into das # Conflicts: # Cargo.lock # Cargo.toml # beacon_node/network/src/sync/block_lookups/tests.rs # beacon_node/network/src/sync/manager.rs * Fix simulator tests on `das` branch (#5731) * Bump genesis delay in sim tests as KZG setup takes longer for DAS. * Fix incorrect YAML spacing. * DataColumnByRange boilerplate (#5353) * add boilerplate * fmt * PeerDAS custody lookup sync (#5684) * Implement custody sync * Lint * Fix tests * Fix rebase issue * Add data column kzg verification and update `c-kzg`. (#5701) * Add data column kzg verification and update `c-kzg`. * Fix incorrect `Cell` size. * Add kzg verification on rpc blocks. * Add kzg verification on rpc data columns. * Rename `PEER_DAS_EPOCH` to `EIP7594_FORK_EPOCH` for client interop. (#5750) * Fetch custody columns in range sync (#5747) * Fetch custody columns in range sync * Clean up todos * Remove `BlobSidecar` construction and publish after PeerDAS activated (#5759) * Avoid building and publishing blob sidecars after PeerDAS. * Ignore gossip blobs with a slot greater than peer das activation epoch. * Only attempt to verify blob count and import blobs before PeerDAS. * #5684 review comments (#5748) * #5684 review comments. * Doc and message update only. * Fix incorrect condition when constructing `RpcBlock` with `DataColumn`s * Make sampling tests deterministic (#5775) * PeerDAS spec tests (#5772) * Add get_custody_columns spec tests. * Add kzg merkle proof spec tests. * Add SSZ spec tests. * Add remaining KZG tests * Load KZG only once per process, exclude electra tests and add missing SSZ tests. * Fix lint and missing changes. * Ignore macOS generated file. * Merge remote branch 'sigp/unstable' into das * Merge remote tracking branch 'origin/unstable' into das * Implement unconditional reconstruction for supernodes (#5781) * Implement unconditional reconstruction for supernodes * Move code into KzgVerifiedCustodyDataColumn * Remove expect * Add test * Thanks justin * Add withhold attack mode for interop (#5788) * Add withhold attack mode * Update readme * Drop added readmes * Undo styling changes * Add column gossip verification and handle unknown parent block (#5783) * Add column gossip verification and handle missing parent for columns. * Review PR * Fix rebase issue * more lint issues :) --------- Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com> * Trigger sampling on sync events (#5776) * Trigger sampling on sync events * Update beacon_chain.rs * Fix tests * Fix tests * PeerDAS parameter changes for devnet-0 (#5779) * Update PeerDAS parameters to latest values. * Lint fix * Fix lint. * Update hardcoded subnet count to 64 (#5791) * Fix incorrect columns per subnet and config cleanup (#5792) * Tidy up PeerDAS preset and config values. * Fix broken config * Fix DAS branch CI (#5793) * Fix invalid syntax. * Update cli doc. Ignore get_custody_columns test temporarily. * Fix failing test and add verify inclusion test. * Undo accidentally removed code. * Only attempt reconstruct columns once. (#5794) * Re-enable precompute table for peerdas kzg (#5795) * Merge branch 'unstable' into das * Update subscription filter. (#5797) * Remove penalty for duplicate columns (expected due to reconstruction) (#5798) * Revert DAS config for interop testing. Optimise get_custody_columns function. (#5799) * Don't perform reconstruction for proposer node as it already has all the columns. (#5806) * Multithread compute_cells_and_proofs (#5805) * Multi-thread reconstruct data columns * Multi-thread path for block production * Merge branch 'unstable' into das # Conflicts: # .github/workflows/test-suite.yml # beacon_node/network/src/sync/block_lookups/mod.rs # beacon_node/network/src/sync/block_lookups/single_block_lookup.rs # beacon_node/network/src/sync/network_context.rs * Fix CI errors. * Move PeerDAS type-level config to configurable `ChainSpec` (#5828) * Move PeerDAS type level config to `ChainSpec`. * Fix tests * Misc custody lookup improvements (#5821) * Improve custody requests * Type DataColumnsByRootRequestId * Prioritize peers and load balance * Update tests * Address PR review * Merge branch 'unstable' into das * Rename deploy_block in network config (`das` branch) (#5852) * Rename deploy_block.txt to deposit_contract_block.txt * fmt --------- Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com> * Merge branch 'unstable' into das * Fix CI and merge issues. * Merge branch 'unstable' into das # Conflicts: # beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs # lcli/src/main.rs * Store data columns individually in store and caches (#5890) * Store data columns individually in store and caches * Implement data column pruning * Merge branch 'unstable' into das # Conflicts: # Cargo.lock * Update reconstruction benches to newer criterion version. (#5949) * Merge branch 'unstable' into das # Conflicts: # .github/workflows/test-suite.yml * chore: add `recover_cells_and_compute_proofs` method (#5938) * chore: add recover_cells_and_compute_proofs method * Introduce type alias `CellsAndKzgProofs` to address type complexity. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Update `csc` format in ENR and spec tests for devnet-1 (#5966) * Update `csc` format in ENR. * Add spec tests for `recover_cells_and_kzg_proofs`. * Add tests for ENR. * Fix failing tests. * Add protection against invalid csc value in ENR. * Fix lint * Fix csc encoding and decoding (#5997) * Fix data column rpc request not being sent due to incorrect limits set. (#6000) * Fix incorrect inbound request count causing rate limiting. (#6025) * Merge branch 'stable' into das # Conflicts: # beacon_node/network/src/sync/block_lookups/tests.rs # beacon_node/network/src/sync/block_sidecar_coupling.rs # beacon_node/network/src/sync/manager.rs # beacon_node/network/src/sync/network_context.rs # beacon_node/network/src/sync/network_context/requests.rs * Merge remote-tracking branch 'unstable' into das * Add kurtosis config for DAS testing (#5968) * Add kurtosis config for DAS testing. * Fix invalid yaml file * Update network parameter files. * chore: add rust PeerdasKZG crypto library for peerdas functionality and rollback c-kzg dependency to 4844 version (#5941) * chore: add recover_cells_and_compute_proofs method * chore: add rust peerdas crypto library * chore: integrate peerdaskzg rust library into kzg crate * chore(multi): - update `ssz_cell_to_crypto_cell` - update conversion from the crypto cell type to a Vec<u8>. Since the Rust library defines them as references to an array, the conversion is simply `to_vec` * chore(multi): - update rest of code to handle the new crypto `Cell` type - update test case code to no longer use the Box type * chore: cleanup of superfluous conversions * chore: revert c-kzg dependency back to v1 * chore: move dependency into correct order * chore: update rust dependency - This version includes a new method `PeerDasContext::with_num_threads` * chore: remove Default initialization of PeerDasContext and explicitly set the parameters in `new_from_trusted_setup` * chore: cleanup exports * chore: commit updated cargo.lock * Update Cargo.toml Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * chore: rename dependency * chore: update peerdas lib - sets the blst version to 0.3 so that it matches whatever lighthouse is using. Although 0.3.12 is latest, lighthouse is pinned to 0.3.3 * chore: fix clippy lifetime - Rust doesn't allow you to elide the lifetime on type aliases * chore: cargo clippy fix * chore: cargo fmt * chore: update lib to add redundant checks (these will be removed in consensus-specs PR 3819) * chore: update dependency to ignore proofs * chore: update peerdas lib to latest * update lib * chore: remove empty proof parameter --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Update PeerDAS interop testnet config (#6069) * Update interop testnet config. * Fix typo and remove target peers * Avoid retrying same sampling peer that previously failed. (#6084) * Various fixes to custody range sync (#6004) * Only start requesting batches when there are good peers across all custody columns to avoid spaming block requests. * Add custody peer check before mutating `BatchInfo` to avoid inconsistent state. * Add check to cover a case where batch is not processed while waiting for custody peers to become available. * Fix lint and logic error * Fix `good_peers_on_subnet` always returning false for `DataColumnSubnet`. * Add test for `get_custody_peers_for_column` * Revert epoch parameter refactor. * Fall back to default custody requiremnt if peer ENR is not present. * Add metrics and update code comment. * Add more debug logs. * Use subscribed peers on subnet before MetaDataV3 is implemented. Remove peer_id matching when injecting error because multiple peers are used for range requests. Use randomized custodial peer to avoid repeatedly sending requests to failing peers. Batch by range request where possible. * Remove unused code and update docs. * Add comment * chore: update peerdas-kzg library (#6118) * chore: update peerDAS lib * chore: update library * chore: update library to version that include "init context" benchmarks and optional validation checks * chore: (can remove) -- Add benchmarks for init context * Prevent continuous searchers for low-peer networks (#6162) * Merge branch 'unstable' into das * Fix merge conflicts * Add cli flag to enable sampling and disable by default. (#6209) * chore: Use reference to an array representing a blob instead of an owned KzgBlob (#6179) * add KzgBlobRef type * modify code to use KzgBlobRef * clippy * Remove Deneb blob related changes to maintain compatibility with `c-kzg-4844`. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Store computed custody subnets in PeerDB and fix custody lookup test (#6218) * Fix failing custody lookup tests. * Store custody subnets in PeerDB, fix custody lookup test and refactor some methods. * Merge branch 'unstable' into das # Conflicts: # beacon_node/beacon_chain/src/beacon_chain.rs # beacon_node/beacon_chain/src/block_verification_types.rs # beacon_node/beacon_chain/src/builder.rs # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs # beacon_node/beacon_chain/src/data_column_verification.rs # beacon_node/beacon_chain/src/early_attester_cache.rs # beacon_node/beacon_chain/src/historical_blocks.rs # beacon_node/beacon_chain/tests/store_tests.rs # beacon_node/lighthouse_network/src/discovery/enr.rs # beacon_node/network/src/service.rs # beacon_node/src/cli.rs # beacon_node/store/src/hot_cold_store.rs # beacon_node/store/src/lib.rs # lcli/src/generate_bootnode_enr.rs * Fix CI failures after merge. * Batch sampling requests by peer (#6256) * Batch sampling requests by peer * Fix clippy errors * Fix tests * Add column_index to error message for ease of tracing * Remove outdated comment * Fix range sync never evaluating request as finished, causing it to get stuck. (#6276) * Merge branch 'unstable' into das-0821-merge # Conflicts: # Cargo.lock # Cargo.toml # beacon_node/beacon_chain/src/beacon_chain.rs # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs # beacon_node/beacon_chain/src/data_column_verification.rs # beacon_node/beacon_chain/src/kzg_utils.rs # beacon_node/beacon_chain/src/metrics.rs # beacon_node/beacon_processor/src/lib.rs # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/config.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/lighthouse_network/src/rpc/outbound.rs # beacon_node/lighthouse_network/src/rpc/rate_limiter.rs # beacon_node/lighthouse_network/src/service/api_types.rs # beacon_node/lighthouse_network/src/types/globals.rs # beacon_node/network/src/network_beacon_processor/mod.rs # beacon_node/network/src/network_beacon_processor/rpc_methods.rs # beacon_node/network/src/network_beacon_processor/sync_methods.rs # beacon_node/network/src/sync/block_lookups/common.rs # beacon_node/network/src/sync/block_lookups/mod.rs # beacon_node/network/src/sync/block_lookups/single_block_lookup.rs # beacon_node/network/src/sync/block_lookups/tests.rs # beacon_node/network/src/sync/manager.rs # beacon_node/network/src/sync/network_context.rs # consensus/types/src/data_column_sidecar.rs # crypto/kzg/Cargo.toml # crypto/kzg/benches/benchmark.rs # crypto/kzg/src/lib.rs * Fix custody tests and load PeerDAS KZG instead. * Fix ef tests and bench compilation. * Fix failing sampling test. * Merge pull request #6287 from jimmygchen/das-0821-merge Merge `unstable` into `das` 20240821 * Remove get_block_import_status * Merge branch 'unstable' into das * Re-enable Windows release tests. * Address some review comments. * Address more review comments and cleanups. * Comment out peer DAS KZG EF tests for now * Address more review comments and fix build. * Merge branch 'das' of github.com:sigp/lighthouse into das * Unignore Electra tests * Fix metric name * Address some of Pawan's review comments * Merge remote-tracking branch 'origin/unstable' into das * Update PeerDAS network parameters for peerdas-devnet-2 (#6290) * update subnet count & custody req * das network params * update ef tests --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> |
||
|
Michael Sproul
|
e09fe5a372 |
Implement Electra spec v1.5.0-alpha.5 (#6305)
* Implement spec v1.5.0-alpha.5 * Merge remote-tracking branch 'origin/unstable' into spec-1.5.0-alpha.5 * Ignore broken Electra light client tests |
||
|
ethDreamer
|
9a295d09b4 |
Implement Electra devnet-1 changes (#5937)
* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix genesis from eth1 for electra * Fix Electra Fork Choice Tests (#5764) * add new operation tests * Fix Electra Fork Choice Tests (#5764) * Fix Electra Fork Choice Tests (#5764) * Fix Electra Fork Choice Tests (#5764) * add epoch processing new tests * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix ef test compilation * disable and enable ef test for epoch processing appropriately * add consolidations to merkle calc for inclusion proof * fix should_override_fc tests * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * get ssz tests compiling * ssz merkle validity for only deneb * im dumb af * Merge remote-tracking branch 'upstream/ef-tests-electra' into ef-tests-electra * enable tests * Fix Consolidation Sigs & Withdrawals * Merge pull request #5766 from ethDreamer/two_fixes Fix Consolidation Sigs & Withdrawals * Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * comment out consolidation operation ef tests until they are added * add signed consolidation ssz tests * add as_iter to all List and Vector fields in the beacon state * remove dbg * Send unagg attestation based on fork * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Fix Epoch Processing Consolidations * Fix Final Spec Tests! * Fix ser/de * Merge branch 'electra-engine-api' into beacon-api-electra * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * update electra readiness with new endpoints * don't block mev boost till genesis finalization * Publish all aggregates * just one more check bro plz.. * Fix Bug In Block Processing with 0x02 Credentials * fix slashing handling * Fix Bug In Block Processing with 0x02 Credentials * Merge remote-tracking branch 'upstream/unstable' * Send unagg attestation based on fork * Publish all aggregates * just one more check bro plz.. * Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable Merge `unstable` into `electra_attestation_changes` * Merge pull request #5835 from realbigsean/fix-validator-logic Fix validator logic * Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling Electra slashing handling * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Rename deploy block electra (#5853) * Rename deploy_block.txt to deposit_contract_block.txt * fmt * Change for prater too * fix: serde rename camle case for execution payload body (#5846) * Merge branch 'electra-engine-api' into beacon-api-electra * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Electra attestation changes rm decode impl (#5856) * Remove Crappy Decode impl for Attestation * Remove Inefficient Attestation Decode impl * Implement Schema Upgrade / Downgrade * Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Fix failing attestation tests and misc electra attestation cleanup (#5810) * - get attestation related beacon chain tests to pass - observed attestations are now keyed off of data + committee index - rename op pool attestationref to compactattestationref - remove unwraps in agg pool and use options instead - cherry pick some changes from ef-tests-electra * cargo fmt * fix failing test * Revert dockerfile changes * make committee_index return option * function args shouldnt be a ref to attestation ref * fmt * fix dup imports --------- Co-authored-by: realbigsean <seananderson33@GMAIL.com> * fix some todos (#5817) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * add consolidations to merkle calc for inclusion proof * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Remove Duplicate KZG Commitment Merkle Proof Code (#5874) * Remove Duplicate KZG Commitment Merkle Proof Code * s/tree_lists/fields/ * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix compile * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Fix slasher tests (#5906) * Fix electra tests * Add electra attestations to double vote tests * Update superstruct to 0.8 * Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes * Small cleanup in slasher tests * Clean up Electra observed aggregates (#5929) * Use consistent key in observed_attestations * Remove unwraps from observed aggregates * move committee bits to the last index in attestation (#5924) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * De-dup attestation constructor logic * Remove unwraps in Attestation construction * Dedup match_attestation_data * Remove outdated TODO * Use ForkName Ord in fork-choice tests * Use ForkName Ord in BeaconBlockBody * Make to_electra not fallible * Remove TestRandom impl for IndexedAttestation * Remove IndexedAttestation faulty Decode impl * Drop TestRandom impl * Add PendingAttestationInElectra * Indexed att on disk (#35) * indexed att on disk * fix lints * Update slasher/src/migrate.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * add electra fork enabled fn to ForkName impl (#36) * add electra fork enabled fn to ForkName impl * remove inadvertent file * Update common/eth2/src/types.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * Dedup attestation constructor logic in attester cache * Use if let Ok for committee_bits * Dedup Attestation constructor code * Diff reduction in tests * Fix beacon_chain tests * Diff reduction * Use Ord for ForkName in pubsub * Resolve into_attestation_and_indices todo * Remove stale TODO * Fix beacon_chain tests * Test spec invariant * Use electra_enabled in pubsub * Remove get_indexed_attestation_from_signed_aggregate * Use ok_or instead of if let else * committees are sorted * remove dup method `get_indexed_attestation_from_committees` * Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview Electra attestations #5712 review * update default persisted op pool deserialization * ensure aggregate and proof uses serde untagged on ref * Fork aware ssz static attestation tests * Electra attestation changes from Lions review (#5971) * dedup/cleanup and remove unneeded hashset use * remove irrelevant TODOs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Fix Compilation Break * Merge pull request #5973 from ethDreamer/beacon-api-electra Fix Compilation Break * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Rename Constants, Structures, and Fields * Second Round * Third Round * Round Four * Electra attestation changes sean review (#5972) * instantiate empty bitlist in unreachable code * clean up error conversion * fork enabled bool cleanup * remove a couple todos * return bools instead of options in `aggregate` and use the result * delete commented out code * use map macros in simple transformations * remove signers_disjoint_from * get ef tests compiling * get ef tests compiling * update intentionally excluded files * Avoid changing slasher schema for Electra * Delete slasher schema v4 * Fix clippy * Fix compilation of beacon_chain tests * Update database.rs * Update per_block_processing.rs * Add electra lightclient types * Update slasher/src/database.rs * fix imports * Merge pull request #5980 from dapplion/electra-lightclient Add electra lightclient types * Merge pull request #5975 from michaelsproul/electra-slasher-no-migration Avoid changing slasher schema for Electra * Update beacon_node/beacon_chain/src/attestation_verification.rs * Update beacon_node/beacon_chain/src/attestation_verification.rs * Merge pull request #5974 from ethDreamer/electra-devnet-1 Electra: `devnet-1` Rename and Move Stuff * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * clean up enabled and disabled fork logic in ef tests * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Electra devnet 1 process operations (#5994) * Implement `process_consolidation_requests()` * Finish Changes to `process_operations()` * Fix Lint * Fix test * use electra_enabled() * add light client ssz tests * Update `engine-api` for `devnet-1` * disable failing light client ssz tests * correctly exclude light client struct from accessed files * Fix Failing Tests * Fix Calculations Relying on MAX_EB (#5983) * Fix Calculations Relying on MAX_EB * Use spec fn for max_effective_balance * The great renaming receipt -> request * Address some more review comments * Electra: Get `devnet-1` ef-tests Working * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api * Update beacon_node/beacon_chain/src/electra_readiness.rs * Update consensus/types/src/chain_spec.rs * Update pending balance deposit processing (#6005) * Update pending balance deposit processing * Update single_pass.rs --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> * Merge pull request #6001 from ethDreamer/electra-devnet-1-engine-api Update `engine-api` for `devnet-1` * Merge pull request #6019 from ethDreamer/electra-devnet-1-ef-tests Electra: Get `devnet-1` ef-tests Working * Fix Bug in `engine_api` by Renaming Field * Merge pull request #6047 from ethDreamer/electra-devnet-1-engine-api-fix Fix Bug in `engine_api` by Renaming Field * one more stupid rename * update GET requests * update POST requests * add client updates and test updates * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * compile after merge * unwrap -> unwrap_err * self review * fix tests * convert op pool messages to electra in electra * remove methods to post without content header * Add BeaconBlocksByRange v3 * Revert "Add BeaconBlocksByRange v3" This reverts commit |
||
|
Eitan Seri-Levi
|
3913ea44c6 |
Persist light client updates (#5545)
* persist light client updates * update beacon chain to serve light client updates * resolve todos * cache best update * extend cache parts * is better light client update * resolve merge conflict * initial api changes * add lc update db column * fmt * added tests * add sim * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * fix some weird issues with the simulator * tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * test changes * merge conflict * testing * started work on ef tests and some code clean up * update tests * linting * noop pre altair, were still failing on electra though * allow for zeroed light client header * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * merge unstable * remove unwraps * remove unwraps * Update light_client_update.rs * merge unstable * move functionality to helper methods * refactor is best update fn * refactor is best update fn * improve organization of light client server cache logic * fork diget calc, and only spawn as many blcoks as we need for the lc update test * fetch lc update from the cache if it exists * fmt * Fix beacon_chain tests * Add debug code to update ranking_order ef test * Fix compare code * merge conflicts * fix test * Merge branch 'persist-light-client-updates' of https://github.com/eserilev/lighthouse into persist-light-client-updates * Use blinded blocks for light client proofs * fix ef test * merge conflicts * fix lc update check * Lint * resolve merge conflict * Merge branch 'persist-light-client-updates' of https://github.com/eserilev/lighthouse into persist-light-client-updates * revert basic sim * small fix * revert sim * Review PR * resolve merge conflicts * Merge branch 'unstable' into persist-light-client-updates |
||
|
realbigsean
|
8a32df756d |
Ef tests electra (#5758)
* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge pull request #5749 from sigp/electra_op_pool Optimise Electra op pool aggregation * don't fail on empty consolidations * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * update committee offset * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * update committee offset * update committee offset * update committee offset * update committee offset * update committee offset * only increment deposit index on state for old deposit flow * Revert "only increment deposit index on state for old deposit flow" This reverts commit |
||
|
ethDreamer
|
c52c598f69 |
Electra: Remaining Consensus Data Structures (#5712)
* Attestation superstruct changes for EIP 7549 (#5644)
* update
* experiment
* superstruct changes
* revert
* superstruct changes
* fix tests
* indexed attestation
* indexed attestation superstruct
* updated TODOs
* `superstruct` the `AttesterSlashing` (#5636)
* `superstruct` Attester Fork Variants
* Push a little further
* Deal with Encode / Decode of AttesterSlashing
* not so sure about this..
* Stop Encode/Decode Bounds from Propagating Out
* Tons of Changes..
* More Conversions to AttestationRef
* Add AsReference trait (#15)
* Add AsReference trait
* Fix some snafus
* Got it Compiling! :D
* Got Tests Building
* Get beacon chain tests compiling
---------
Co-authored-by: Michael Sproul <micsproul@gmail.com>
* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes
* Make EF Tests Fork-Agnostic (#5713)
* Finish EF Test Fork Agnostic (#5714)
* Superstruct `AggregateAndProof` (#5715)
* Upgrade `superstruct` to `0.8.0`
* superstruct `AggregateAndProof`
* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes
* cargo fmt
* Merge pull request #5726 from realbigsean/electra_attestation_changes
Merge unstable into Electra attestation changes
* EIP7549 `get_attestation_indices` (#5657)
* get attesting indices electra impl
* fmt
* get tests to pass
* fmt
* fix some beacon chain tests
* fmt
* fix slasher test
* fmt got me again
* fix more tests
* fix tests
* Some small changes (#5739)
* cargo fmt (#5740)
* Sketch op pool changes
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
* Get `electra_op_pool` up to date (#5756)
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
---------
Co-authored-by: realbigsean <sean@sigmaprime.io>
* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)
This reverts commit
|
||
|
Eitan Seri-Levi
|
e4d4e439cb |
Add Capella & Deneb light client support (#4946)
* rebase and add comment * conditional test * test * optimistic chould be working now * finality should be working now * try again * try again * clippy fix * add lc bootstrap beacon api * add lc optimistic/finality update to events * fmt * That error isn't occuring on my computer but I think this should fix it * Merge branch 'unstable' into light_client_beacon_api_1 # Conflicts: # beacon_node/beacon_chain/src/events.rs # beacon_node/http_api/src/lib.rs # beacon_node/http_api/src/test_utils.rs # beacon_node/http_api/tests/main.rs # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/lighthouse_network/src/service/api_types.rs # beacon_node/network/src/beacon_processor/worker/rpc_methods.rs # beacon_node/tests/test.rs # common/eth2/src/types.rs # lighthouse/src/main.rs * Add missing test file * Update light client types to comply with Altair light client spec. * Fix test compilation * Merge branch 'unstable' into light_client_beacon_api_1 * Support deserializing light client structures for the Bellatrix fork * Move `get_light_client_bootstrap` logic to `BeaconChain`. `LightClientBootstrap` API to return `ForkVersionedResponse`. * Misc fixes. - log cleanup - move http_api config mutation to `config::get_config` for consistency - fix light client API responses * Add light client bootstrap API test and fix existing ones. * Merge branch 'unstable' into light_client_beacon_api_1 * Fix test for `light-client-server` http api config. * Appease clippy * Add Altair light client SSZ tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light_client_beacon_api_1 * updates to light client header * light client header from signed beacon block * using options * implement helper functions * placeholder conversion from vec hash256 to exec branch * add deneb * using fixed vector * remove unwraps * by epoch * compute merkle proof * merkle proof * update comments * resolve merge conflicts * linting * Merge branch 'unstable' into light-client-ssz-tests # Conflicts: # beacon_node/beacon_chain/src/beacon_chain.rs # consensus/types/src/light_client_bootstrap.rs # consensus/types/src/light_client_header.rs * superstruct attempt * superstruct changes * lint * altair * update * update * changes to light_client_optimistic_ and finality * merge unstable * refactor * resolved merge conflicts * Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types * block_to_light_client_header fork aware * fmt * comment fix * comment fix * include merge fork, update deserialize_by_fork, refactor * fmt * pass by ref to prevent clone * rename merkle proof fn * add FIXME * LightClientHeader TestRandom * fix comments * fork version deserialize * merge unstable * move fn arguments, fork name calc * use task executor * remove unneeded fns * remove dead code * add manual ssz decoding/encoding and add ssz_tests_by_fork macro * merge deneb types with tests * merge ssz tests, revert code deletion, cleanup * move chainspec * update ssz tests * fmt * light client ssz tests * change to superstruct * changes from feedback * linting * Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types * test fix * cleanup * Remove unused `derive`. * Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types * beta compiler fix * merge |
||
|
realbigsean
|
4172d9f75c |
Update to consensus spec v1.4.0-beta.6 (#5094)
* get latest ef tests passing
* fix tests
* Fix invalid payload recovery tests
* Merge branch 'unstable' into update-to-spec-v1.4.0-beta.6
* Revert "fix tests"
This reverts commit
|
||
|
realbigsean
|
7d468cb487 |
More deneb cleanup (#4640)
* remove protoc and token from network tests github action
* delete unused beacon chain methods
* downgrade writing blobs to store log
* reduce diff in block import logic
* remove some todo's and deneb built in network
* remove unnecessary error, actually use some added metrics
* remove some metrics, fix missing components on publish funcitonality
* fix status tests
* rename sidecar by root to blobs by root
* clean up some metrics
* remove unnecessary feature gate from attestation subnet tests, clean up blobs by range response code
* pawan's suggestion in `protocol_info`, peer score in matching up batch sync block and blobs
* fix range tests for deneb
* pub block and blob db cache behind the same mutex
* remove unused errs and an empty file
* move sidecar trait to new file
* move types from payload to eth2 crate
* update comment and add flag value name
* make function private again, remove allow unused
* use reth rlp for tx decoding
* fix compile after merge
* rename kzg commitments
* cargo fmt
* remove unused dep
* Update beacon_node/execution_layer/src/lib.rs
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
* Update beacon_node/beacon_processor/src/lib.rs
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
* pawan's suggestiong for vec capacity
* cargo fmt
* Revert "use reth rlp for tx decoding"
This reverts commit
|
||
|
realbigsean
|
adbb62f7f3 |
Devnet6 (#4404)
* some blob reprocessing work
* remove ForceBlockLookup
* reorder enum match arms in sync manager
* a lot more reprocessing work
* impl logic for triggerng blob lookups along with block lookups
* deal with rpc blobs in groups per block in the da checker. don't cache missing blob ids in the da checker.
* make single block lookup generic
* more work
* add delayed processing logic and combine some requests
* start fixing some compile errors
* fix compilation in main block lookup mod
* much work
* get things compiling
* parent blob lookups
* fix compile
* revert red/stevie changes
* fix up sync manager delay message logic
* add peer usefulness enum
* should remove lookup refactor
* consolidate retry error handling
* improve peer scoring during certain failures in parent lookups
* improve retry code
* drop parent lookup if either req has a peer disconnect during download
* refactor single block processed method
* processing peer refactor
* smol bugfix
* fix some todos
* fix lints
* fix lints
* fix compile in lookup tests
* fix lints
* fix lints
* fix existing block lookup tests
* renamings
* fix after merge
* cargo fmt
* compilation fix in beacon chain tests
* fix
* refactor lookup tests to work with multiple forks and response types
* make tests into macros
* wrap availability check error
* fix compile after merge
* add random blobs
* start fixing up lookup verify error handling
* some bug fixes and the start of deneb only tests
* make tests work for all forks
* track information about peer source
* error refactoring
* improve peer scoring
* fix test compilation
* make sure blobs are sent for processing after stream termination, delete copied tests
* add some tests and fix a bug
* smol bugfixes and moar tests
* add tests and fix some things
* compile after merge
* lots of refactoring
* retry on invalid block/blob
* merge unknown parent messages before current slot lookup
* get tests compiling
* penalize blob peer on invalid blobs
* Check disk on in-memory cache miss
* Update beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
* Update beacon_node/network/src/sync/network_context.rs
Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>
* fix bug in matching blocks and blobs in range sync
* pr feedback
* fix conflicts
* upgrade logs from warn to crit when we receive incorrect response in range
* synced_and_connected_within_tolerance -> should_search_for_block
* remove todo
* add data gas used and update excess data gas to u64
* Fix Broken Overflow Tests
* payload verification with commitments
* fix merge conflicts
* restore payload file
* Restore payload file
* remove todo
* add max blob commitments per block
* c-kzg lib update
* Fix ef tests
* Abstract over minimal/mainnet spec in kzg crate
* Start integrating new KZG
* checkpoint sync without alignment
* checkpoint sync without alignment
* add import
* add import
* query for checkpoint state by slot rather than state root (teku doesn't serve by state root)
* query for checkpoint state by slot rather than state root (teku doesn't serve by state root)
* loosen check
* get state first and query by most recent block root
* Revert "loosen check"
This reverts commit
|
||
|
realbigsean
|
da7fab5188 |
Ef test version update (#4142)
* Revert "revert change to ef_tests" This reverts commit |
||
|
realbigsean
|
a5addf661c |
Rename eip4844 to deneb (#4129)
* rename 4844 to deneb * rename 4844 to deneb * move excess data gas field * get EF tests working * fix ef tests lint * fix the blob identifier ef test * fix accessed files ef test script * get beacon chain tests passing |
||
|
Diva M
|
20be7024e1 |
Revert "Use consensus-spec-tests v1.3.0-rc.3 (#4021)"
This reverts commit
|
||
|
Diva M
|
d93753cc88 | Merge branch 'unstable' into off-4844 | ||
|
Paul Hauner
|
caa6190d4a |
Use consensus-spec-tests v1.3.0-rc.3 (#4021)
## Issue Addressed NA ## Proposed Changes Updates our `ef_tests` to use: https://github.com/ethereum/consensus-specs/releases/tag/v1.3.0-rc.3 This required: - Skipping a `merkle_proof_validity` test (see #4022) - Account for the `eip4844` tests changing name to `deneb` - My IDE did some Python linting during this change. It seemed simple and nice so I left it there. ## Additional Info NA |
||
|
realbigsean
|
ddcd10b194 | merge latest capella changes | ||
|
realbigsean
|
302eaca344 |
intentionally skip LightClientHeader ssz static tests
|
||
|
realbigsean
|
63da81de62 | check for eip4844 unaccessed files | ||
|
realbigsean
|
98b11bbd3f |
add historical summaries (#3865)
* add historical summaries * fix tree hash caching, disable the sanity slots test with fake crypto * add ssz static HistoricalSummary * only store historical summaries after capella * Teach `UpdatePattern` about Capella * Tidy EF tests * Clippy Co-authored-by: Michael Sproul <michael@sigmaprime.io> |
||
|
Mark Mackey
|
8a04c3428e |
Merged with unstable
|
||
|
Michael Sproul
|
3be41006a6 |
Add --light-client-server flag and state cache utils (#3714)
## Issue Addressed Part of https://github.com/sigp/lighthouse/issues/3651. ## Proposed Changes Add a flag for enabling the light client server, which should be checked before gossip/RPC traffic is processed (e.g. https://github.com/sigp/lighthouse/pull/3693, https://github.com/sigp/lighthouse/pull/3711). The flag is available at runtime from `beacon_chain.config.enable_light_client_server`. Additionally, a new method `BeaconChain::with_mutable_state_for_block` is added which I envisage being used for computing light client updates. Unfortunately its performance will be quite poor on average because it will only run quickly with access to the tree hash cache. Each slot the tree hash cache is only available for a brief window of time between the head block being processed and the state advance at 9s in the slot. When the state advance happens the cache is moved and mutated to get ready for the next slot, which makes it no longer useful for merkle proofs related to the head block. Rather than spend more time trying to optimise this I think we should continue prototyping with this code, and I'll make sure `tree-states` is ready to ship before we enable the light client server in prod (cf. https://github.com/sigp/lighthouse/pull/3206). ## Additional Info I also fixed a bug in the implementation of `BeaconState::compute_merkle_proof` whereby the tree hash cache was moved with `.take()` but never put back with `.restore()`. |
||
|
realbigsean
|
8656d23327 | merge with unstable | ||
|
will
|
9f242137b0 |
Add a new bls test (#3235)
## Issue Addressed Which issue # does this PR address? #2629 ## Proposed Changes Please list or describe the changes introduced by this PR. 1. ci would dowload the bls test cases from https://github.com/ethereum/bls12-381-tests/ 2. all the bls test cases(except eth ones) would use cases in the archive from step one 3. The bls test cases from https://github.com/ethereum/consensus-spec-tests would stay there and no use . For the future , these bls test cases would be remove suggested from https://github.com/ethereum/consensus-spec-tests/issues/25 . So it would do no harm and compatible for future cases. ## Additional Info Please provide any additional information. For example, future considerations or information useful for reviewers. Question: I am not sure if I should implement tests about `deserialization_G1`, `deserialization_G2` and `hash_to_G2` for the issue. |
||
|
realbigsean
|
88006735c4 | compile | ||
|
Paul Hauner
|
a0605c4ee6 |
Bump EF tests to v1.2.0 rc.3 (#3483)
## Issue Addressed NA ## Proposed Changes Bumps test vectors and ignores another weird MacOS file. ## Additional Info NA |
||
|
Paul Hauner
|
d9d1288156 |
Add mainnet merge values 🐼 (#3462)
## Issue Addressed NA ## Proposed Changes Adds **tentative** values for the merge TTD and Bellatrix as per https://github.com/ethereum/consensus-specs/pull/2969 ## Additional Info - ~~Blocked on https://github.com/ethereum/consensus-specs/pull/2969~~ |
||
|
Michael Sproul
|
d04fde3ba9 |
Remove equivocating validators from fork choice (#3371)
## Issue Addressed Closes https://github.com/sigp/lighthouse/issues/3241 Closes https://github.com/sigp/lighthouse/issues/3242 ## Proposed Changes * [x] Implement logic to remove equivocating validators from fork choice per https://github.com/ethereum/consensus-specs/pull/2845 * [x] Update tests to v1.2.0-rc.1. The new test which exercises `equivocating_indices` is passing. * [x] Pull in some SSZ abstractions from the `tree-states` branch that make implementing Vec-compatible encoding for types like `BTreeSet` and `BTreeMap`. * [x] Implement schema upgrades and downgrades for the database (new schema version is V11). * [x] Apply attester slashings from blocks to fork choice ## Additional Info * This PR doesn't need the `BTreeMap` impl, but `tree-states` does, and I don't think there's any harm in keeping it. But I could also be convinced to drop it. Blocked on #3322. |
||
|
Michael Sproul
|
ef7351ddfe |
Update to spec v1.1.8 (#2893)
## Proposed Changes Change the canonical fork name for the merge to Bellatrix. Keep other merge naming the same to avoid churn. I've also fixed and enabled the `fork` and `transition` tests for Bellatrix, and the v1.1.7 fork choice tests. Additionally, the `BellatrixPreset` has been added with tests. It gets served via the `/config/spec` API endpoint along with the other presets. |
||
|
realbigsean
|
b22ac95d7f |
v1.1.6 Fork Choice changes (#2822)
## Issue Addressed Resolves: https://github.com/sigp/lighthouse/issues/2741 Includes: https://github.com/sigp/lighthouse/pull/2853 so that we can get ssz static tests passing here on v1.1.6. If we want to merge that first, we can make this diff slightly smaller ## Proposed Changes - Changes the `justified_epoch` and `finalized_epoch` in the `ProtoArrayNode` each to an `Option<Checkpoint>`. The `Option` is necessary only for the migration, so not ideal. But does allow us to add a default logic to `None` on these fields during the database migration. - Adds a database migration from a legacy fork choice struct to the new one, search for all necessary block roots in fork choice by iterating through blocks in the db. - updates related to https://github.com/ethereum/consensus-specs/pull/2727 - We will have to update the persisted forkchoice to make sure the justified checkpoint stored is correct according to the updated fork choice logic. This boils down to setting the forkchoice store's justified checkpoint to the justified checkpoint of the block that advanced the finalized checkpoint to the current one. - AFAICT there's no migration steps necessary for the update to allow applying attestations from prior blocks, but would appreciate confirmation on that - I updated the consensus spec tests to v1.1.6 here, but they will fail until we also implement the proposer score boost updates. I confirmed that the previously failing scenario `new_finalized_slot_is_justified_checkpoint_ancestor` will now pass after the boost updates, but haven't confirmed _all_ tests will pass because I just quickly stubbed out the proposer boost test scenario formatting. - This PR now also includes proposer boosting https://github.com/ethereum/consensus-specs/pull/2730 ## Additional Info I realized checking justified and finalized roots in fork choice makes it more likely that we trigger this bug: https://github.com/ethereum/consensus-specs/pull/2727 It's possible the combination of justified checkpoint and finalized checkpoint in the forkchoice store is different from in any block in fork choice. So when trying to startup our store's justified checkpoint seems invalid to the rest of fork choice (but it should be valid). When this happens we get an `InvalidBestNode` error and fail to start up. So I'm including that bugfix in this branch. Todo: - [x] Fix fork choice tests - [x] Self review - [x] Add fix for https://github.com/ethereum/consensus-specs/pull/2727 - [x] Rebase onto Kintusgi - [x] Fix `num_active_validators` calculation as @michaelsproul pointed out - [x] Clean up db migrations Co-authored-by: realbigsean <seananderson33@gmail.com> |
||
|
Pawan Dhananjay
|
e391b32858 |
Merge devnet 3 (#2859)
## Issue Addressed N/A ## Proposed Changes Changes required for the `merge-devnet-3`. Added some more non substantive renames on top of @realbigsean 's commit. Note: this doesn't include the proposer boosting changes in kintsugi v3. This devnet isn't running with the proposer boosting fork choice changes so if we are looking to merge https://github.com/sigp/lighthouse/pull/2822 into `unstable`, then I think we should just maintain this branch for the devnet temporarily. Co-authored-by: realbigsean <seananderson33@gmail.com> Co-authored-by: Paul Hauner <paul@paulhauner.com> |
||
|
Paul Hauner
|
5f0fef2d1e |
Kintsugi on_merge_block tests (#2811)
* Start v1.1.5 updates * Implement new payload creation logic * Tidy, add comments * Remove unused error enums * Add validate payload for gossip * Refactor validate_merge_block * Split payload verification in per block processing * Add execute_payload * Tidy * Tidy * Start working on new fork choice tests * Fix failing merge block test * Skip block_lookup_failed test * Fix failing terminal block test * Fixes from self-review * Address review comments |