mirror of
https://github.com/sigp/lighthouse.git
synced 2026-03-02 16:21:42 +00:00
stable
66 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Eitan Seri-Levi
|
9bec8df37a |
Add Gloas data column support (#8682)
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu> Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com> |
||
|
Mac L
|
58b153cac5 |
Remove remaining facade module re-exports from consensus/types (#8672)
Removes the remaining facade re-exports from `consensus/types`. I have left `graffiti` as I think it has some utility so am leaning towards keeping it in the final API design. Co-Authored-By: Mac L <mjladson@pm.me> |
||
|
Mac L
|
f3fd1f210b |
Remove consensus/types re-exports (#8540)
There are certain crates which we re-export within `types` which creates a fragmented DevEx, where there are various ways to import the same crates.
```rust
// consensus/types/src/lib.rs
pub use bls::{
AggregatePublicKey, AggregateSignature, Error as BlsError, Keypair, PUBLIC_KEY_BYTES_LEN,
PublicKey, PublicKeyBytes, SIGNATURE_BYTES_LEN, SecretKey, Signature, SignatureBytes,
get_withdrawal_credentials,
};
pub use context_deserialize::{ContextDeserialize, context_deserialize};
pub use fixed_bytes::FixedBytesExtended;
pub use milhouse::{self, List, Vector};
pub use ssz_types::{BitList, BitVector, FixedVector, VariableList, typenum, typenum::Unsigned};
pub use superstruct::superstruct;
```
This PR removes these re-exports and makes it explicit that these types are imported from a non-`consensus/types` crate.
Co-Authored-By: Mac L <mjladson@pm.me>
|
||
|
Jimmy Chen
|
b4704eab4a |
Fulu update to spec v1.6.0-alpha.4 (#7890)
Fulu update to spec [v1.6.0-alpha.4](https://github.com/ethereum/consensus-specs/releases/tag/v1.6.0-alpha.4). - Make `number_of_columns` a preset - Optimise `get_custody_groups` to avoid computing if cgc = 128 - Add support for additional typenum values in type_dispatch macro |
||
|
ethDreamer
|
b08d49c4cb |
Changes for fusaka-devnet-1 (#7559)
Changes for [fusaka-devnet-1](https://notes.ethereum.org/@ethpandaops/fusaka-devnet-1) [Consensus Specs v1.6.0-alpha.1](https://github.com/ethereum/consensus-specs/pull/4346) * [EIP-7917: Deterministic Proposer Lookahead](https://eips.ethereum.org/EIPS/eip-7917) * [EIP-7892: Blob Parameter Only Hardforks](https://eips.ethereum.org/EIPS/eip-7892) |
||
|
ethDreamer
|
2d9fc34d43 |
Fulu EF tests v1.6.0-alpha.0 (#7540)
Update to EF tests v1.6.0-alpha.0 |
||
|
SunnysidedJ
|
593390162f |
peerdas-devnet-7: update DataColumnSidecarsByRoot request to use DataColumnsByRootIdentifier (#7399)
Update DataColumnSidecarsByRoot request to use DataColumnsByRootIdentifier #7377 As described in https://github.com/ethereum/consensus-specs/pull/4284 |
||
|
Michael Sproul
|
ff739d56be |
Fix light client merkle proofs (#7007)
Fix a regression introduced in this PR: - https://github.com/sigp/lighthouse/pull/6361 We were indexing into the `MerkleTree` with raw generalized indices, which was incorrect and triggering `debug_assert` failures, as described here: - https://github.com/sigp/lighthouse/issues/7005 - Convert `generalized_index` to the correct leaf index prior to proof generation. - Add sanity checks on indices used in `BeaconState::generate_proof`. - Remove debug asserts from `MerkleTree::generate_proof` in favour of actual errors. This would have caught the bug earlier. - Refactor the EF tests so that the merkle validity tests are actually run. They were misconfigured in a way that resulted in them running silently with 0 test cases, and the `check_all_files_accessed.py` script still had an ignore that covered the test files, so this omission wasn't detected. |
||
|
Michael Sproul
|
ceb5ecf349 |
Update EF tests to spec v1.5.0-beta.2 (#6958)
Update spec tests for recent v1.5.0-beta.2 release. There are no substantial changes for Electra and earlier, and the Fulu test updates to be made are tracked here: - https://github.com/sigp/lighthouse/issues/6957 - Add `SingleAttestation` SSZ tests - Add new `deposit_with_reorg` fork choice tests - Update tag to v1.5.0-beta.2 - Ignore Fulu tests |
||
|
Jimmy Chen
|
70194dfc6a |
Implement PeerDAS Fulu fork activation (#6795)
Addresses #6706 This PR activates PeerDAS at the Fulu fork epoch instead of `EIP_7594_FORK_EPOCH`. This means we no longer support testing PeerDAS with Deneb / Electrs, as it's now part of a hard fork. |
||
|
Jimmy Chen
|
e98209d118 |
Implement PeerDAS subnet decoupling (aka custody groups) (#6736)
* Implement PeerDAS subnet decoupling (aka custody groups). * Merge branch 'unstable' into decouple-subnets * Refactor feature testing for spec tests (#6737) Squashed commit of the following: commit |
||
|
Michael Sproul
|
06e4d22d49 |
Electra spec changes for v1.5.0-beta.0 (#6731)
* First pass * Add restrictions to RuntimeVariableList api * Use empty_uninitialized and fix warnings * Fix some todos * Merge branch 'unstable' into max-blobs-preset * Fix take impl on RuntimeFixedList * cleanup * Fix test compilations * Fix some more tests * Fix test from unstable * Merge branch 'unstable' into max-blobs-preset * Implement "Bugfix and more withdrawal tests" * Implement "Add missed exit checks to consolidation processing" * Implement "Update initial earliest_exit_epoch calculation" * Implement "Limit consolidating balance by validator.effective_balance" * Implement "Use 16-bit random value in validator filter" * Implement "Do not change creds type on consolidation" * Rename PendingPartialWithdraw index field to validator_index * Skip slots to get test to pass and add TODO * Implement "Synchronously check all transactions to have non-zero length" * Merge remote-tracking branch 'origin/unstable' into max-blobs-preset * Remove footgun function * Minor simplifications * Move from preset to config * Fix typo * Revert "Remove footgun function" This reverts commit |
||
|
Jimmy Chen
|
c9747fb77f |
Refactor feature testing for spec tests (#6737)
* Refactor spec testing for features and simplify usage. * Fix `SszStatic` tests for PeerDAS: exclude eip7594 test vectors when testing Electra types. * Merge branch 'unstable' into refactor-ef-tests-features |
||
|
Pawan Dhananjay
|
d74b2d96f5 |
Electra alpha8 spec updates (#6496)
* Fix partial withdrawals count * Remove get_active_balance * Remove queue_entire_balance_and_reset_validator * Switch to compounding when consolidating with source==target * Queue deposit requests and apply them during epoch processing * Fix ef tests * Clear todos * Fix engine api formatting issues * Merge branch 'unstable' into electra-alpha7 * Make add_validator_to_registry more in line with the spec * Address some review comments * Cleanup * Update initialize_beacon_state_from_eth1 * Merge branch 'unstable' into electra-alpha7 * Fix rpc decoding for blobs by range/root * Fix block hash computation * Fix process_deposits bug * Merge branch 'unstable' into electra-alpha7 * Fix topup deposit processing bug * Update builder api for electra * Refactor mock builder to separate functionality * Merge branch 'unstable' into electra-alpha7 * Address review comments * Use copied for reference rather than cloned * Optimise and simplify PendingDepositsContext::new * Merge remote-tracking branch 'origin/unstable' into electra-alpha7 * Fix processing of deposits with invalid signatures * Remove redundant code in genesis init * Revert "Refactor mock builder to separate functionality" This reverts commit |
||
|
Eitan Seri-Levi
|
9d069a9588 |
Fix electra light client types (#6361)
* persist light client updates * update beacon chain to serve light client updates * resolve todos * cache best update * extend cache parts * is better light client update * resolve merge conflict * initial api changes * add lc update db column * fmt * added tests * add sim * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * fix some weird issues with the simulator * tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * test changes * merge conflict * testing * started work on ef tests and some code clean up * update tests * linting * noop pre altair, were still failing on electra though * allow for zeroed light client header * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * merge unstable * remove unwraps * remove unwraps * fetch bootstrap without always querying for state * storing bootstrap parts in db * mroe code cleanup * test * prune sync committee branches from dropped chains * Update light_client_update.rs * merge unstable * move functionality to helper methods * refactor is best update fn * refactor is best update fn * improve organization of light client server cache logic * fork diget calc, and only spawn as many blcoks as we need for the lc update test * resovle merge conflict * add electra bootstrap logic, add logic to cache current sync committee * add latest sync committe branch cache * fetch lc update from the cache if it exists * fmt * Fix beacon_chain tests * Add debug code to update ranking_order ef test * Fix compare code * merge conflicts * merge conflict * add better error messaging * resolve merge conflicts * remove lc update from basicsim * rename sync comittte variable and fix persist condition * refactor get_light_client_update logic * add better comments, return helpful error messages over http and rpc * pruning canonical non checkpoint slots * fix test * rerun test * update pruning logic, add tests * fix tests * fix imports * fmt * refactor db code * Refactor db method * Refactor db method * lc electra changes * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * add additional comments * testing lc merkle changes * lc electra * update struct defs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * fix merge * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-bootstrap * fix merge * linting * merge conflict * prevent overflow * enable lc server for http api tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * get tests working: * remove related TODOs * fix test lint * Merge branch 'persist-light-client-bootstrap' of https://github.com/eserilev/lighthouse into light-client-electra * fix tests * fix conflicts * remove prints * Merge branch 'persist-light-client-bootstrap' of https://github.com/eserilev/lighthouse into light-client-electra * remove warning * resolve conflicts * merge conflicts * linting * remove comments * cleanup * linting * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * pre/post electra light client cached data * add proof type alias * move is_empty_branch method out of impl * add ssz tests for all forks * refactor beacon state proof codepaths * rename method * fmt * clean up proof logic * refactor merkle proof api * fmt * Merge branch 'unstable' into light-client-electra * Use superstruct mapping macros * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra * rename proof to merkleproof * fmt * Resolve merge conflicts * merge conflicts |
||
|
Michael Sproul
|
a732a87846 |
Remove TTD flags and safe-slots-to-import-* (#6489)
* Delete SAFE_SLOTS_TO_IMPORT_OPTIMISTICALLY * Update fork choice tests * Remove TTD related flags * Add deprecation warning * Remove more dead code * Delete EF on_merge_block tests * Remove even more dead code * Address Mac's review comments |
||
|
Pawan Dhananjay
|
83d5c521d7 |
Electra updates for v1.5.0-alpha.6 (#6445)
* Update process_slashing * Update test vectors version * Delete Domain::Consolidation * Rename to get_max_effective_balance * Fix unused; lint * Add the pre-electra slashing processing * lint * Change request json types * Move requests from payload to beacon block body * Refactor engine api * Fix warnings * Update engine api to latest * engine api changed..again * yet again * Merge branch 'engine-requests' into electra-updates * Fix tests * Store reference instead of bytes in NewPayloadRequest * Merge branch 'unstable' into electra-updates * Update beacon_node/execution_layer/src/engine_api/json_structures.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> * Update beacon_node/execution_layer/src/lib.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> * Update beacon_node/execution_layer/src/test_utils/handle_rpc.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> |
||
|
kevaundray
|
653126f42e |
chore: update and enable PeerDAS tests to latest spec test release (1.5.0-alpha-5) (#6312)
* enable DAS tests * update spec testing code * Update PeerDAS kzg tests input structures to latest spec. * Update `ef_tests` ignore files. |
||
|
Lion - dapplion
|
f75a2cf65b |
PeerDAS implementation (#5683)
* 1D PeerDAS prototype: Data format and Distribution (#5050) * Build and publish column sidecars. Add stubs for gossip. * Add blob column subnets * Add `BlobColumnSubnetId` and initial compute subnet logic. * Subscribe to blob column subnets. * Introduce `BLOB_COLUMN_SUBNET_COUNT` based on DAS configuration parameter changes. * Fix column sidecar type to use `VariableList` for data. * Fix lint errors. * Update types and naming to latest consensus-spec #3574. * Fix test and some cleanups. * Merge branch 'unstable' into das * Merge branch 'unstable' into das * Merge branch 'unstable' into das # Conflicts: # consensus/types/src/chain_spec.rs * Add `DataColumnSidecarsByRoot ` req/resp protocol (#5196) * Add stub for `DataColumnsByRoot` * Add basic implementation of serving RPC data column from DA checker. * Store data columns in early attester cache and blobs db. * Apply suggestions from code review Co-authored-by: Eitan Seri-Levi <eserilev@gmail.com> Co-authored-by: Jacob Kaufmann <jacobkaufmann18@gmail.com> * Fix build. * Store `DataColumnInfo` in database and various cleanups. * Update `DataColumnSidecar` ssz max size and remove panic code. --------- Co-authored-by: Eitan Seri-Levi <eserilev@gmail.com> Co-authored-by: Jacob Kaufmann <jacobkaufmann18@gmail.com> * feat: add DAS KZG in data col construction (#5210) * feat: add DAS KZG in data col construction * refactor data col sidecar construction * refactor: add data cols to GossipVerifiedBlockContents * Disable windows tests for `das` branch. (c-kzg doesn't build on windows) * Formatting and lint changes only. * refactor: remove iters in construction of data cols * Update vec capacity and error handling. * Add `data_column_sidecar_computation_seconds` metric. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Merge branch 'unstable' into das # Conflicts: # .github/workflows/test-suite.yml # beacon_node/lighthouse_network/src/types/topics.rs * fix: update data col subnet count from 64 to 32 (#5413) * feat: add peerdas custody field to ENR (#5409) * feat: add peerdas custody field to ENR * add hash prefix step in subnet computation * refactor test and fix possible u64 overflow * default to min custody value if not present in ENR * Merge branch 'unstable' into das * Merge branch 'unstable' into das-unstable-merge-0415 # Conflicts: # Cargo.lock # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/data_availability_checker/availability_view.rs # beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs # beacon_node/beacon_chain/src/data_availability_checker/processing_cache.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/network/src/network_beacon_processor/mod.rs # beacon_node/network/src/sync/block_lookups/tests.rs # crypto/kzg/Cargo.toml * Merge remote-tracking branch 'sigp/unstable' into das * Merge remote-tracking branch 'sigp/unstable' into das * Fix merge conflicts. * Send custody data column to `DataAvailabilityChecker` for determining block importability (#5570) * Only import custody data columns after publishing a block. * Add `subscribe-all-data-column-subnets` and pass custody column count to `availability_cache`. * Add custody requirement checks to `availability_cache`. * Fix config not being passed to DAChecker and add more logging. * Introduce `peer_das_epoch` and make blobs and columns mutually exclusive. * Add DA filter for PeerDAS. * Fix data availability check and use test_logger in tests. * Fix subscribe to all data column subnets not working correctly. * Fix tests. * Only publish column sidecars if PeerDAS is activated. Add `PEER_DAS_EPOCH` chain spec serialization. * Remove unused data column index in `OverflowKey`. * Fix column sidecars incorrectly produced when there are no blobs. * Re-instate index to `OverflowKey::DataColumn` and downgrade noisy debug log to `trace`. * DAS sampling on sync (#5616) * Data availability sampling on sync * Address @jimmygchen review * Trigger sampling * Address some review comments and only send `SamplingBlock` sync message after PEER_DAS_EPOCH. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Merge branch 'unstable' into das # Conflicts: # Cargo.lock # Cargo.toml # beacon_node/beacon_chain/src/block_verification.rs # beacon_node/http_api/src/publish_blocks.rs # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/protocol.rs # beacon_node/lighthouse_network/src/types/pubsub.rs # beacon_node/network/src/sync/block_lookups/single_block_lookup.rs # beacon_node/store/src/hot_cold_store.rs # consensus/types/src/beacon_state.rs # consensus/types/src/chain_spec.rs # consensus/types/src/eth_spec.rs * Merge branch 'unstable' into das * Re-process early sampling requests (#5569) * Re-process early sampling requests # Conflicts: # beacon_node/beacon_processor/src/work_reprocessing_queue.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/network/src/network_beacon_processor/rpc_methods.rs * Update beacon_node/beacon_processor/src/work_reprocessing_queue.rs Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Add missing var * Beta compiler fixes and small typo fixes. * Remove duplicate method. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Merge remote-tracking branch 'sigp/unstable' into das * Fix merge conflict. * Add data columns by root to currently supported protocol list (#5678) * Add data columns by root to currently supported protocol list. * Add missing data column by roots handling. * Merge branch 'unstable' into das # Conflicts: # Cargo.lock # Cargo.toml # beacon_node/network/src/sync/block_lookups/tests.rs # beacon_node/network/src/sync/manager.rs * Fix simulator tests on `das` branch (#5731) * Bump genesis delay in sim tests as KZG setup takes longer for DAS. * Fix incorrect YAML spacing. * DataColumnByRange boilerplate (#5353) * add boilerplate * fmt * PeerDAS custody lookup sync (#5684) * Implement custody sync * Lint * Fix tests * Fix rebase issue * Add data column kzg verification and update `c-kzg`. (#5701) * Add data column kzg verification and update `c-kzg`. * Fix incorrect `Cell` size. * Add kzg verification on rpc blocks. * Add kzg verification on rpc data columns. * Rename `PEER_DAS_EPOCH` to `EIP7594_FORK_EPOCH` for client interop. (#5750) * Fetch custody columns in range sync (#5747) * Fetch custody columns in range sync * Clean up todos * Remove `BlobSidecar` construction and publish after PeerDAS activated (#5759) * Avoid building and publishing blob sidecars after PeerDAS. * Ignore gossip blobs with a slot greater than peer das activation epoch. * Only attempt to verify blob count and import blobs before PeerDAS. * #5684 review comments (#5748) * #5684 review comments. * Doc and message update only. * Fix incorrect condition when constructing `RpcBlock` with `DataColumn`s * Make sampling tests deterministic (#5775) * PeerDAS spec tests (#5772) * Add get_custody_columns spec tests. * Add kzg merkle proof spec tests. * Add SSZ spec tests. * Add remaining KZG tests * Load KZG only once per process, exclude electra tests and add missing SSZ tests. * Fix lint and missing changes. * Ignore macOS generated file. * Merge remote branch 'sigp/unstable' into das * Merge remote tracking branch 'origin/unstable' into das * Implement unconditional reconstruction for supernodes (#5781) * Implement unconditional reconstruction for supernodes * Move code into KzgVerifiedCustodyDataColumn * Remove expect * Add test * Thanks justin * Add withhold attack mode for interop (#5788) * Add withhold attack mode * Update readme * Drop added readmes * Undo styling changes * Add column gossip verification and handle unknown parent block (#5783) * Add column gossip verification and handle missing parent for columns. * Review PR * Fix rebase issue * more lint issues :) --------- Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com> * Trigger sampling on sync events (#5776) * Trigger sampling on sync events * Update beacon_chain.rs * Fix tests * Fix tests * PeerDAS parameter changes for devnet-0 (#5779) * Update PeerDAS parameters to latest values. * Lint fix * Fix lint. * Update hardcoded subnet count to 64 (#5791) * Fix incorrect columns per subnet and config cleanup (#5792) * Tidy up PeerDAS preset and config values. * Fix broken config * Fix DAS branch CI (#5793) * Fix invalid syntax. * Update cli doc. Ignore get_custody_columns test temporarily. * Fix failing test and add verify inclusion test. * Undo accidentally removed code. * Only attempt reconstruct columns once. (#5794) * Re-enable precompute table for peerdas kzg (#5795) * Merge branch 'unstable' into das * Update subscription filter. (#5797) * Remove penalty for duplicate columns (expected due to reconstruction) (#5798) * Revert DAS config for interop testing. Optimise get_custody_columns function. (#5799) * Don't perform reconstruction for proposer node as it already has all the columns. (#5806) * Multithread compute_cells_and_proofs (#5805) * Multi-thread reconstruct data columns * Multi-thread path for block production * Merge branch 'unstable' into das # Conflicts: # .github/workflows/test-suite.yml # beacon_node/network/src/sync/block_lookups/mod.rs # beacon_node/network/src/sync/block_lookups/single_block_lookup.rs # beacon_node/network/src/sync/network_context.rs * Fix CI errors. * Move PeerDAS type-level config to configurable `ChainSpec` (#5828) * Move PeerDAS type level config to `ChainSpec`. * Fix tests * Misc custody lookup improvements (#5821) * Improve custody requests * Type DataColumnsByRootRequestId * Prioritize peers and load balance * Update tests * Address PR review * Merge branch 'unstable' into das * Rename deploy_block in network config (`das` branch) (#5852) * Rename deploy_block.txt to deposit_contract_block.txt * fmt --------- Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com> * Merge branch 'unstable' into das * Fix CI and merge issues. * Merge branch 'unstable' into das # Conflicts: # beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs # lcli/src/main.rs * Store data columns individually in store and caches (#5890) * Store data columns individually in store and caches * Implement data column pruning * Merge branch 'unstable' into das # Conflicts: # Cargo.lock * Update reconstruction benches to newer criterion version. (#5949) * Merge branch 'unstable' into das # Conflicts: # .github/workflows/test-suite.yml * chore: add `recover_cells_and_compute_proofs` method (#5938) * chore: add recover_cells_and_compute_proofs method * Introduce type alias `CellsAndKzgProofs` to address type complexity. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Update `csc` format in ENR and spec tests for devnet-1 (#5966) * Update `csc` format in ENR. * Add spec tests for `recover_cells_and_kzg_proofs`. * Add tests for ENR. * Fix failing tests. * Add protection against invalid csc value in ENR. * Fix lint * Fix csc encoding and decoding (#5997) * Fix data column rpc request not being sent due to incorrect limits set. (#6000) * Fix incorrect inbound request count causing rate limiting. (#6025) * Merge branch 'stable' into das # Conflicts: # beacon_node/network/src/sync/block_lookups/tests.rs # beacon_node/network/src/sync/block_sidecar_coupling.rs # beacon_node/network/src/sync/manager.rs # beacon_node/network/src/sync/network_context.rs # beacon_node/network/src/sync/network_context/requests.rs * Merge remote-tracking branch 'unstable' into das * Add kurtosis config for DAS testing (#5968) * Add kurtosis config for DAS testing. * Fix invalid yaml file * Update network parameter files. * chore: add rust PeerdasKZG crypto library for peerdas functionality and rollback c-kzg dependency to 4844 version (#5941) * chore: add recover_cells_and_compute_proofs method * chore: add rust peerdas crypto library * chore: integrate peerdaskzg rust library into kzg crate * chore(multi): - update `ssz_cell_to_crypto_cell` - update conversion from the crypto cell type to a Vec<u8>. Since the Rust library defines them as references to an array, the conversion is simply `to_vec` * chore(multi): - update rest of code to handle the new crypto `Cell` type - update test case code to no longer use the Box type * chore: cleanup of superfluous conversions * chore: revert c-kzg dependency back to v1 * chore: move dependency into correct order * chore: update rust dependency - This version includes a new method `PeerDasContext::with_num_threads` * chore: remove Default initialization of PeerDasContext and explicitly set the parameters in `new_from_trusted_setup` * chore: cleanup exports * chore: commit updated cargo.lock * Update Cargo.toml Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * chore: rename dependency * chore: update peerdas lib - sets the blst version to 0.3 so that it matches whatever lighthouse is using. Although 0.3.12 is latest, lighthouse is pinned to 0.3.3 * chore: fix clippy lifetime - Rust doesn't allow you to elide the lifetime on type aliases * chore: cargo clippy fix * chore: cargo fmt * chore: update lib to add redundant checks (these will be removed in consensus-specs PR 3819) * chore: update dependency to ignore proofs * chore: update peerdas lib to latest * update lib * chore: remove empty proof parameter --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Update PeerDAS interop testnet config (#6069) * Update interop testnet config. * Fix typo and remove target peers * Avoid retrying same sampling peer that previously failed. (#6084) * Various fixes to custody range sync (#6004) * Only start requesting batches when there are good peers across all custody columns to avoid spaming block requests. * Add custody peer check before mutating `BatchInfo` to avoid inconsistent state. * Add check to cover a case where batch is not processed while waiting for custody peers to become available. * Fix lint and logic error * Fix `good_peers_on_subnet` always returning false for `DataColumnSubnet`. * Add test for `get_custody_peers_for_column` * Revert epoch parameter refactor. * Fall back to default custody requiremnt if peer ENR is not present. * Add metrics and update code comment. * Add more debug logs. * Use subscribed peers on subnet before MetaDataV3 is implemented. Remove peer_id matching when injecting error because multiple peers are used for range requests. Use randomized custodial peer to avoid repeatedly sending requests to failing peers. Batch by range request where possible. * Remove unused code and update docs. * Add comment * chore: update peerdas-kzg library (#6118) * chore: update peerDAS lib * chore: update library * chore: update library to version that include "init context" benchmarks and optional validation checks * chore: (can remove) -- Add benchmarks for init context * Prevent continuous searchers for low-peer networks (#6162) * Merge branch 'unstable' into das * Fix merge conflicts * Add cli flag to enable sampling and disable by default. (#6209) * chore: Use reference to an array representing a blob instead of an owned KzgBlob (#6179) * add KzgBlobRef type * modify code to use KzgBlobRef * clippy * Remove Deneb blob related changes to maintain compatibility with `c-kzg-4844`. --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Store computed custody subnets in PeerDB and fix custody lookup test (#6218) * Fix failing custody lookup tests. * Store custody subnets in PeerDB, fix custody lookup test and refactor some methods. * Merge branch 'unstable' into das # Conflicts: # beacon_node/beacon_chain/src/beacon_chain.rs # beacon_node/beacon_chain/src/block_verification_types.rs # beacon_node/beacon_chain/src/builder.rs # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs # beacon_node/beacon_chain/src/data_column_verification.rs # beacon_node/beacon_chain/src/early_attester_cache.rs # beacon_node/beacon_chain/src/historical_blocks.rs # beacon_node/beacon_chain/tests/store_tests.rs # beacon_node/lighthouse_network/src/discovery/enr.rs # beacon_node/network/src/service.rs # beacon_node/src/cli.rs # beacon_node/store/src/hot_cold_store.rs # beacon_node/store/src/lib.rs # lcli/src/generate_bootnode_enr.rs * Fix CI failures after merge. * Batch sampling requests by peer (#6256) * Batch sampling requests by peer * Fix clippy errors * Fix tests * Add column_index to error message for ease of tracing * Remove outdated comment * Fix range sync never evaluating request as finished, causing it to get stuck. (#6276) * Merge branch 'unstable' into das-0821-merge # Conflicts: # Cargo.lock # Cargo.toml # beacon_node/beacon_chain/src/beacon_chain.rs # beacon_node/beacon_chain/src/data_availability_checker.rs # beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs # beacon_node/beacon_chain/src/data_column_verification.rs # beacon_node/beacon_chain/src/kzg_utils.rs # beacon_node/beacon_chain/src/metrics.rs # beacon_node/beacon_processor/src/lib.rs # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/config.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/lighthouse_network/src/rpc/outbound.rs # beacon_node/lighthouse_network/src/rpc/rate_limiter.rs # beacon_node/lighthouse_network/src/service/api_types.rs # beacon_node/lighthouse_network/src/types/globals.rs # beacon_node/network/src/network_beacon_processor/mod.rs # beacon_node/network/src/network_beacon_processor/rpc_methods.rs # beacon_node/network/src/network_beacon_processor/sync_methods.rs # beacon_node/network/src/sync/block_lookups/common.rs # beacon_node/network/src/sync/block_lookups/mod.rs # beacon_node/network/src/sync/block_lookups/single_block_lookup.rs # beacon_node/network/src/sync/block_lookups/tests.rs # beacon_node/network/src/sync/manager.rs # beacon_node/network/src/sync/network_context.rs # consensus/types/src/data_column_sidecar.rs # crypto/kzg/Cargo.toml # crypto/kzg/benches/benchmark.rs # crypto/kzg/src/lib.rs * Fix custody tests and load PeerDAS KZG instead. * Fix ef tests and bench compilation. * Fix failing sampling test. * Merge pull request #6287 from jimmygchen/das-0821-merge Merge `unstable` into `das` 20240821 * Remove get_block_import_status * Merge branch 'unstable' into das * Re-enable Windows release tests. * Address some review comments. * Address more review comments and cleanups. * Comment out peer DAS KZG EF tests for now * Address more review comments and fix build. * Merge branch 'das' of github.com:sigp/lighthouse into das * Unignore Electra tests * Fix metric name * Address some of Pawan's review comments * Merge remote-tracking branch 'origin/unstable' into das * Update PeerDAS network parameters for peerdas-devnet-2 (#6290) * update subnet count & custody req * das network params * update ef tests --------- Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> |
||
|
ethDreamer
|
9a295d09b4 |
Implement Electra devnet-1 changes (#5937)
* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix genesis from eth1 for electra * Fix Electra Fork Choice Tests (#5764) * add new operation tests * Fix Electra Fork Choice Tests (#5764) * Fix Electra Fork Choice Tests (#5764) * Fix Electra Fork Choice Tests (#5764) * add epoch processing new tests * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix ef test compilation * disable and enable ef test for epoch processing appropriately * add consolidations to merkle calc for inclusion proof * fix should_override_fc tests * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * get ssz tests compiling * ssz merkle validity for only deneb * im dumb af * Merge remote-tracking branch 'upstream/ef-tests-electra' into ef-tests-electra * enable tests * Fix Consolidation Sigs & Withdrawals * Merge pull request #5766 from ethDreamer/two_fixes Fix Consolidation Sigs & Withdrawals * Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * comment out consolidation operation ef tests until they are added * add signed consolidation ssz tests * add as_iter to all List and Vector fields in the beacon state * remove dbg * Send unagg attestation based on fork * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Fix Epoch Processing Consolidations * Fix Final Spec Tests! * Fix ser/de * Merge branch 'electra-engine-api' into beacon-api-electra * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * update electra readiness with new endpoints * don't block mev boost till genesis finalization * Publish all aggregates * just one more check bro plz.. * Fix Bug In Block Processing with 0x02 Credentials * fix slashing handling * Fix Bug In Block Processing with 0x02 Credentials * Merge remote-tracking branch 'upstream/unstable' * Send unagg attestation based on fork * Publish all aggregates * just one more check bro plz.. * Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable Merge `unstable` into `electra_attestation_changes` * Merge pull request #5835 from realbigsean/fix-validator-logic Fix validator logic * Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling Electra slashing handling * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Rename deploy block electra (#5853) * Rename deploy_block.txt to deposit_contract_block.txt * fmt * Change for prater too * fix: serde rename camle case for execution payload body (#5846) * Merge branch 'electra-engine-api' into beacon-api-electra * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Electra attestation changes rm decode impl (#5856) * Remove Crappy Decode impl for Attestation * Remove Inefficient Attestation Decode impl * Implement Schema Upgrade / Downgrade * Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Fix failing attestation tests and misc electra attestation cleanup (#5810) * - get attestation related beacon chain tests to pass - observed attestations are now keyed off of data + committee index - rename op pool attestationref to compactattestationref - remove unwraps in agg pool and use options instead - cherry pick some changes from ef-tests-electra * cargo fmt * fix failing test * Revert dockerfile changes * make committee_index return option * function args shouldnt be a ref to attestation ref * fmt * fix dup imports --------- Co-authored-by: realbigsean <seananderson33@GMAIL.com> * fix some todos (#5817) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * add consolidations to merkle calc for inclusion proof * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Remove Duplicate KZG Commitment Merkle Proof Code (#5874) * Remove Duplicate KZG Commitment Merkle Proof Code * s/tree_lists/fields/ * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix compile * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Fix slasher tests (#5906) * Fix electra tests * Add electra attestations to double vote tests * Update superstruct to 0.8 * Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes * Small cleanup in slasher tests * Clean up Electra observed aggregates (#5929) * Use consistent key in observed_attestations * Remove unwraps from observed aggregates * move committee bits to the last index in attestation (#5924) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * De-dup attestation constructor logic * Remove unwraps in Attestation construction * Dedup match_attestation_data * Remove outdated TODO * Use ForkName Ord in fork-choice tests * Use ForkName Ord in BeaconBlockBody * Make to_electra not fallible * Remove TestRandom impl for IndexedAttestation * Remove IndexedAttestation faulty Decode impl * Drop TestRandom impl * Add PendingAttestationInElectra * Indexed att on disk (#35) * indexed att on disk * fix lints * Update slasher/src/migrate.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * add electra fork enabled fn to ForkName impl (#36) * add electra fork enabled fn to ForkName impl * remove inadvertent file * Update common/eth2/src/types.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * Dedup attestation constructor logic in attester cache * Use if let Ok for committee_bits * Dedup Attestation constructor code * Diff reduction in tests * Fix beacon_chain tests * Diff reduction * Use Ord for ForkName in pubsub * Resolve into_attestation_and_indices todo * Remove stale TODO * Fix beacon_chain tests * Test spec invariant * Use electra_enabled in pubsub * Remove get_indexed_attestation_from_signed_aggregate * Use ok_or instead of if let else * committees are sorted * remove dup method `get_indexed_attestation_from_committees` * Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview Electra attestations #5712 review * update default persisted op pool deserialization * ensure aggregate and proof uses serde untagged on ref * Fork aware ssz static attestation tests * Electra attestation changes from Lions review (#5971) * dedup/cleanup and remove unneeded hashset use * remove irrelevant TODOs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Fix Compilation Break * Merge pull request #5973 from ethDreamer/beacon-api-electra Fix Compilation Break * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Rename Constants, Structures, and Fields * Second Round * Third Round * Round Four * Electra attestation changes sean review (#5972) * instantiate empty bitlist in unreachable code * clean up error conversion * fork enabled bool cleanup * remove a couple todos * return bools instead of options in `aggregate` and use the result * delete commented out code * use map macros in simple transformations * remove signers_disjoint_from * get ef tests compiling * get ef tests compiling * update intentionally excluded files * Avoid changing slasher schema for Electra * Delete slasher schema v4 * Fix clippy * Fix compilation of beacon_chain tests * Update database.rs * Update per_block_processing.rs * Add electra lightclient types * Update slasher/src/database.rs * fix imports * Merge pull request #5980 from dapplion/electra-lightclient Add electra lightclient types * Merge pull request #5975 from michaelsproul/electra-slasher-no-migration Avoid changing slasher schema for Electra * Update beacon_node/beacon_chain/src/attestation_verification.rs * Update beacon_node/beacon_chain/src/attestation_verification.rs * Merge pull request #5974 from ethDreamer/electra-devnet-1 Electra: `devnet-1` Rename and Move Stuff * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * clean up enabled and disabled fork logic in ef tests * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Electra devnet 1 process operations (#5994) * Implement `process_consolidation_requests()` * Finish Changes to `process_operations()` * Fix Lint * Fix test * use electra_enabled() * add light client ssz tests * Update `engine-api` for `devnet-1` * disable failing light client ssz tests * correctly exclude light client struct from accessed files * Fix Failing Tests * Fix Calculations Relying on MAX_EB (#5983) * Fix Calculations Relying on MAX_EB * Use spec fn for max_effective_balance * The great renaming receipt -> request * Address some more review comments * Electra: Get `devnet-1` ef-tests Working * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api * Update beacon_node/beacon_chain/src/electra_readiness.rs * Update consensus/types/src/chain_spec.rs * Update pending balance deposit processing (#6005) * Update pending balance deposit processing * Update single_pass.rs --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> * Merge pull request #6001 from ethDreamer/electra-devnet-1-engine-api Update `engine-api` for `devnet-1` * Merge pull request #6019 from ethDreamer/electra-devnet-1-ef-tests Electra: Get `devnet-1` ef-tests Working * Fix Bug in `engine_api` by Renaming Field * Merge pull request #6047 from ethDreamer/electra-devnet-1-engine-api-fix Fix Bug in `engine_api` by Renaming Field * one more stupid rename * update GET requests * update POST requests * add client updates and test updates * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * compile after merge * unwrap -> unwrap_err * self review * fix tests * convert op pool messages to electra in electra * remove methods to post without content header * Add BeaconBlocksByRange v3 * Revert "Add BeaconBlocksByRange v3" This reverts commit |
||
|
Eitan Seri-Levi
|
3913ea44c6 |
Persist light client updates (#5545)
* persist light client updates * update beacon chain to serve light client updates * resolve todos * cache best update * extend cache parts * is better light client update * resolve merge conflict * initial api changes * add lc update db column * fmt * added tests * add sim * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * fix some weird issues with the simulator * tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * test changes * merge conflict * testing * started work on ef tests and some code clean up * update tests * linting * noop pre altair, were still failing on electra though * allow for zeroed light client header * Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates * merge unstable * remove unwraps * remove unwraps * Update light_client_update.rs * merge unstable * move functionality to helper methods * refactor is best update fn * refactor is best update fn * improve organization of light client server cache logic * fork diget calc, and only spawn as many blcoks as we need for the lc update test * fetch lc update from the cache if it exists * fmt * Fix beacon_chain tests * Add debug code to update ranking_order ef test * Fix compare code * merge conflicts * fix test * Merge branch 'persist-light-client-updates' of https://github.com/eserilev/lighthouse into persist-light-client-updates * Use blinded blocks for light client proofs * fix ef test * merge conflicts * fix lc update check * Lint * resolve merge conflict * Merge branch 'persist-light-client-updates' of https://github.com/eserilev/lighthouse into persist-light-client-updates * revert basic sim * small fix * revert sim * Review PR * resolve merge conflicts * Merge branch 'unstable' into persist-light-client-updates |
||
|
realbigsean
|
8a32df756d |
Ef tests electra (#5758)
* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge pull request #5749 from sigp/electra_op_pool Optimise Electra op pool aggregation * don't fail on empty consolidations * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * update committee offset * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * update committee offset * update committee offset * update committee offset * update committee offset * update committee offset * only increment deposit index on state for old deposit flow * Revert "only increment deposit index on state for old deposit flow" This reverts commit |
||
|
ethDreamer
|
c52c598f69 |
Electra: Remaining Consensus Data Structures (#5712)
* Attestation superstruct changes for EIP 7549 (#5644)
* update
* experiment
* superstruct changes
* revert
* superstruct changes
* fix tests
* indexed attestation
* indexed attestation superstruct
* updated TODOs
* `superstruct` the `AttesterSlashing` (#5636)
* `superstruct` Attester Fork Variants
* Push a little further
* Deal with Encode / Decode of AttesterSlashing
* not so sure about this..
* Stop Encode/Decode Bounds from Propagating Out
* Tons of Changes..
* More Conversions to AttestationRef
* Add AsReference trait (#15)
* Add AsReference trait
* Fix some snafus
* Got it Compiling! :D
* Got Tests Building
* Get beacon chain tests compiling
---------
Co-authored-by: Michael Sproul <micsproul@gmail.com>
* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes
* Make EF Tests Fork-Agnostic (#5713)
* Finish EF Test Fork Agnostic (#5714)
* Superstruct `AggregateAndProof` (#5715)
* Upgrade `superstruct` to `0.8.0`
* superstruct `AggregateAndProof`
* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes
* cargo fmt
* Merge pull request #5726 from realbigsean/electra_attestation_changes
Merge unstable into Electra attestation changes
* EIP7549 `get_attestation_indices` (#5657)
* get attesting indices electra impl
* fmt
* get tests to pass
* fmt
* fix some beacon chain tests
* fmt
* fix slasher test
* fmt got me again
* fix more tests
* fix tests
* Some small changes (#5739)
* cargo fmt (#5740)
* Sketch op pool changes
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
* Get `electra_op_pool` up to date (#5756)
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
---------
Co-authored-by: realbigsean <sean@sigmaprime.io>
* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)
This reverts commit
|
||
|
Mac L
|
13f94ef0f3 |
Rename Merge to Bellatrix (#5601)
* Rename Merge to Bellatrix * Remove tree-hash-cache which got readded from the rebase |
||
|
Eitan Seri-Levi
|
e4d4e439cb |
Add Capella & Deneb light client support (#4946)
* rebase and add comment * conditional test * test * optimistic chould be working now * finality should be working now * try again * try again * clippy fix * add lc bootstrap beacon api * add lc optimistic/finality update to events * fmt * That error isn't occuring on my computer but I think this should fix it * Merge branch 'unstable' into light_client_beacon_api_1 # Conflicts: # beacon_node/beacon_chain/src/events.rs # beacon_node/http_api/src/lib.rs # beacon_node/http_api/src/test_utils.rs # beacon_node/http_api/tests/main.rs # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/lighthouse_network/src/service/api_types.rs # beacon_node/network/src/beacon_processor/worker/rpc_methods.rs # beacon_node/tests/test.rs # common/eth2/src/types.rs # lighthouse/src/main.rs * Add missing test file * Update light client types to comply with Altair light client spec. * Fix test compilation * Merge branch 'unstable' into light_client_beacon_api_1 * Support deserializing light client structures for the Bellatrix fork * Move `get_light_client_bootstrap` logic to `BeaconChain`. `LightClientBootstrap` API to return `ForkVersionedResponse`. * Misc fixes. - log cleanup - move http_api config mutation to `config::get_config` for consistency - fix light client API responses * Add light client bootstrap API test and fix existing ones. * Merge branch 'unstable' into light_client_beacon_api_1 * Fix test for `light-client-server` http api config. * Appease clippy * Add Altair light client SSZ tests * Merge branch 'unstable' of https://github.com/sigp/lighthouse into light_client_beacon_api_1 * updates to light client header * light client header from signed beacon block * using options * implement helper functions * placeholder conversion from vec hash256 to exec branch * add deneb * using fixed vector * remove unwraps * by epoch * compute merkle proof * merkle proof * update comments * resolve merge conflicts * linting * Merge branch 'unstable' into light-client-ssz-tests # Conflicts: # beacon_node/beacon_chain/src/beacon_chain.rs # consensus/types/src/light_client_bootstrap.rs # consensus/types/src/light_client_header.rs * superstruct attempt * superstruct changes * lint * altair * update * update * changes to light_client_optimistic_ and finality * merge unstable * refactor * resolved merge conflicts * Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types * block_to_light_client_header fork aware * fmt * comment fix * comment fix * include merge fork, update deserialize_by_fork, refactor * fmt * pass by ref to prevent clone * rename merkle proof fn * add FIXME * LightClientHeader TestRandom * fix comments * fork version deserialize * merge unstable * move fn arguments, fork name calc * use task executor * remove unneeded fns * remove dead code * add manual ssz decoding/encoding and add ssz_tests_by_fork macro * merge deneb types with tests * merge ssz tests, revert code deletion, cleanup * move chainspec * update ssz tests * fmt * light client ssz tests * change to superstruct * changes from feedback * linting * Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types * test fix * cleanup * Remove unused `derive`. * Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types * beta compiler fix * merge |
||
|
realbigsean
|
4172d9f75c |
Update to consensus spec v1.4.0-beta.6 (#5094)
* get latest ef tests passing
* fix tests
* Fix invalid payload recovery tests
* Merge branch 'unstable' into update-to-spec-v1.4.0-beta.6
* Revert "fix tests"
This reverts commit
|
||
|
Pawan Dhananjay
|
31044402ee |
Sidecar inclusion proof (#4900)
* Refactor BlobSidecar to new type * Fix some compile errors * Gossip verification compiles * Fix http api types take 1 * Fix another round of compile errors * Beacon node crate compiles * EF tests compile * Remove all blob signing from VC * fmt * Tests compile * Fix some tests * Fix more http tests * get compiling * Fix gossip conditions and tests * Add basic proof generation and verification * remove unnecessary ssz decode * add back build_sidecar * remove default at fork for blobs * fix beacon chain tests * get relase tests compiling * fix lints * fix existing spec tests * add new ef tests * fix gossip duplicate rule * lints * add back sidecar signature check in gossip * add finalized descendant check to blob sidecar gossip * fix error conversion * fix release tests * sidecar inclusion self review cleanup * Add proof verification and computation metrics * Remove accidentally committed file * Unify some block and blob errors; add slashing conditions for sidecars * Address review comment * Clean up re-org tests (#4957) * Address more review comments * Add Comments & Eliminate Unnecessary Clones * update names * Update beacon_node/beacon_chain/src/metrics.rs Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Update beacon_node/network/src/network_beacon_processor/tests.rs Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * pr feedback * fix test compile * Sidecar Inclusion proof small refactor and updates (#4967) * Update some comments, variables and small cosmetic fixes. * Couple blobs and proofs into a tuple in `PayloadAndBlobs` for simplicity and safety. * Update function comment. * Update testing/ef_tests/src/cases/merkle_proof_validity.rs Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> * Rename the block and blob wrapper types used in the beacon API interfaces. * make sure gossip invalid blobs are passed to the slasher (#4970) * Add blob headers to slasher before adding to DA checker * Replace Vec with HashSet in BlockQueue * fmt * Rename gindex -> index * Simplify gossip condition --------- Co-authored-by: realbigsean <seananderson33@gmail.com> Co-authored-by: realbigsean <sean@sigmaprime.io> Co-authored-by: Michael Sproul <michael@sigmaprime.io> Co-authored-by: Mark Mackey <mark@sigmaprime.io> Co-authored-by: Jimmy Chen <jchen.tc@gmail.com> |
||
|
realbigsean
|
adbb62f7f3 |
Devnet6 (#4404)
* some blob reprocessing work
* remove ForceBlockLookup
* reorder enum match arms in sync manager
* a lot more reprocessing work
* impl logic for triggerng blob lookups along with block lookups
* deal with rpc blobs in groups per block in the da checker. don't cache missing blob ids in the da checker.
* make single block lookup generic
* more work
* add delayed processing logic and combine some requests
* start fixing some compile errors
* fix compilation in main block lookup mod
* much work
* get things compiling
* parent blob lookups
* fix compile
* revert red/stevie changes
* fix up sync manager delay message logic
* add peer usefulness enum
* should remove lookup refactor
* consolidate retry error handling
* improve peer scoring during certain failures in parent lookups
* improve retry code
* drop parent lookup if either req has a peer disconnect during download
* refactor single block processed method
* processing peer refactor
* smol bugfix
* fix some todos
* fix lints
* fix lints
* fix compile in lookup tests
* fix lints
* fix lints
* fix existing block lookup tests
* renamings
* fix after merge
* cargo fmt
* compilation fix in beacon chain tests
* fix
* refactor lookup tests to work with multiple forks and response types
* make tests into macros
* wrap availability check error
* fix compile after merge
* add random blobs
* start fixing up lookup verify error handling
* some bug fixes and the start of deneb only tests
* make tests work for all forks
* track information about peer source
* error refactoring
* improve peer scoring
* fix test compilation
* make sure blobs are sent for processing after stream termination, delete copied tests
* add some tests and fix a bug
* smol bugfixes and moar tests
* add tests and fix some things
* compile after merge
* lots of refactoring
* retry on invalid block/blob
* merge unknown parent messages before current slot lookup
* get tests compiling
* penalize blob peer on invalid blobs
* Check disk on in-memory cache miss
* Update beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
* Update beacon_node/network/src/sync/network_context.rs
Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>
* fix bug in matching blocks and blobs in range sync
* pr feedback
* fix conflicts
* upgrade logs from warn to crit when we receive incorrect response in range
* synced_and_connected_within_tolerance -> should_search_for_block
* remove todo
* add data gas used and update excess data gas to u64
* Fix Broken Overflow Tests
* payload verification with commitments
* fix merge conflicts
* restore payload file
* Restore payload file
* remove todo
* add max blob commitments per block
* c-kzg lib update
* Fix ef tests
* Abstract over minimal/mainnet spec in kzg crate
* Start integrating new KZG
* checkpoint sync without alignment
* checkpoint sync without alignment
* add import
* add import
* query for checkpoint state by slot rather than state root (teku doesn't serve by state root)
* query for checkpoint state by slot rather than state root (teku doesn't serve by state root)
* loosen check
* get state first and query by most recent block root
* Revert "loosen check"
This reverts commit
|
||
|
ethDreamer
|
a22e4bf636 | Implement KZG EF Tests (#4274) | ||
|
realbigsean
|
a5addf661c |
Rename eip4844 to deneb (#4129)
* rename 4844 to deneb * rename 4844 to deneb * move excess data gas field * get EF tests working * fix ef tests lint * fix the blob identifier ef test * fix accessed files ef test script * get beacon chain tests passing |
||
|
Pawan Dhananjay
|
b276af98b7 |
Rework block processing (#4092)
* introduce availability pending block * add intoavailableblock trait * small fixes * add 'gossip blob cache' and start to clean up processing and transition types * shard memory blob cache * Initial commit * Fix after rebase * Add gossip verification conditions * cache cleanup * general chaos * extended chaos * cargo fmt * more progress * more progress * tons of changes, just tryna compile * everything, everywhere, all at once * Reprocess an ExecutedBlock on unavailable blobs * Add sus gossip verification for blobs * Merge stuff * Remove reprocessing cache stuff * lint * Add a wrapper to allow construction of only valid `AvailableBlock`s * rename blob arc list to blob list * merge cleanuo * Revert "merge cleanuo" This reverts commit |
||
|
Diva M
|
1b9cfcc11b | Merge branch 'unstable' into eip4844 | ||
|
Paul Hauner
|
1f8c17b530 |
Fork choice modifications and cleanup (#3962)
## Issue Addressed NA ## Proposed Changes - Implements https://github.com/ethereum/consensus-specs/pull/3290/ - Bumps `ef-tests` to [v1.3.0-rc.4](https://github.com/ethereum/consensus-spec-tests/releases/tag/v1.3.0-rc.4). The `CountRealizedFull` concept has been removed and the `--count-unrealized-full` and `--count-unrealized` BN flags now do nothing but log a `WARN` when used. ## Database Migration Debt This PR removes the `best_justified_checkpoint` from fork choice. This field is persisted on-disk and the correct way to go about this would be to make a DB migration to remove the field. However, in this PR I've simply stubbed out the value with a junk value. I've taken this approach because if we're going to do a DB migration I'd love to remove the `Option`s around the justified and finalized checkpoints on `ProtoNode` whilst we're at it. Those options were added in #2822 which was included in Lighthouse v2.1.0. The options were only put there to handle the migration and they've been set to `Some` ever since v2.1.0. There's no reason to keep them as options anymore. I started adding the DB migration to this branch but I started to feel like I was bloating this rather critical PR with nice-to-haves. I've kept the partially-complete migration [over in my repo](https://github.com/paulhauner/lighthouse/tree/fc-pr-18-migration) so we can pick it up after this PR is merged. |
||
|
realbigsean
|
c705be202b |
run historical_summary ef tests for 4844
|
||
|
realbigsean
|
438126f19a | merge upstream, fix compile errors | ||
|
realbigsean
|
98b11bbd3f |
add historical summaries (#3865)
* add historical summaries * fix tree hash caching, disable the sanity slots test with fake crypto * add ssz static HistoricalSummary * only store historical summaries after capella * Teach `UpdatePattern` about Capella * Tidy EF tests * Clippy Co-authored-by: Michael Sproul <michael@sigmaprime.io> |
||
|
ethDreamer
|
cb94f639b0 | Isolate withdrawals-processing Feature (#3854) | ||
|
realbigsean
|
f45d117e73 | merge with capella | ||
|
Mark Mackey
|
b75ca74222 |
Removed withdrawals feature flag
|
||
|
realbigsean
|
d893706e0e | merge with capella | ||
|
ethDreamer
|
b1c33361ea |
Fixed Clippy Complaints & Some Failing Tests (#3791)
* Fixed Clippy Complaints & Some Failing Tests * Update Dockerfile to Rust-1.65 * EF test file renamed * Touch up comments based on feedback |
||
|
realbigsean
|
8102a01085 | merge with upstream | ||
|
Mark Mackey
|
8a04c3428e |
Merged with unstable
|
||
|
realbigsean
|
beddcfaac2 | get spec tests working and fix json serialization | ||
|
Michael Sproul
|
0cdd049da9 |
Fixes to make EF Capella tests pass (#3719)
* Fixes to make EF Capella tests pass * Clippy for state_processing |
||
|
Giulio rebuffo
|
9d6209725f |
Added Merkle Proof Generation for Beacon State (#3674)
## Issue Addressed This PR addresses partially #3651 ## Proposed Changes This PR adds the following methods: * a new method to trait `TreeHash`, `hash_tree_leaves` which returns all the Merkle leaves of the ssz object. * a new method to `BeaconState`: `compute_merkle_proof` which generates a specific merkle proof for given depth and index by using the `hash_tree_leaves` as leaves function. ## Additional Info Now here is some rationale on why I decided to go down this route: adding a new function to commonly used trait is a pain but was necessary to make sure we have all merkle leaves for every object, that is why I just added `hash_tree_leaves` in the trait and not `compute_merkle_proof` as well. although it would make sense it gives us code duplication/harder review time and we just need it from one specific object in one specific usecase so not worth the effort YET. In my humble opinion. Co-authored-by: Michael Sproul <micsproul@gmail.com> |
||
|
Michael Sproul
|
e4cbdc1c77 |
Optimistic sync spec tests (v1.2.0) (#3564)
## Issue Addressed Implements new optimistic sync test format from https://github.com/ethereum/consensus-specs/pull/2982. ## Proposed Changes - Add parsing and runner support for the new test format. - Extend the mock EL with a set of canned responses keyed by block hash. Although this doubles up on some of the existing functionality I think it's really nice to use compared to the `preloaded_responses` or static responses. I think we could write novel new opt sync tests using these primtives much more easily than the previous ones. Forks are natively supported, and different responses to `forkchoiceUpdated` and `newPayload` are also straight-forward. ## Additional Info Blocked on merge of the spec PR and release of new test vectors. |
||
|
will
|
9f242137b0 |
Add a new bls test (#3235)
## Issue Addressed Which issue # does this PR address? #2629 ## Proposed Changes Please list or describe the changes introduced by this PR. 1. ci would dowload the bls test cases from https://github.com/ethereum/bls12-381-tests/ 2. all the bls test cases(except eth ones) would use cases in the archive from step one 3. The bls test cases from https://github.com/ethereum/consensus-spec-tests would stay there and no use . For the future , these bls test cases would be remove suggested from https://github.com/ethereum/consensus-spec-tests/issues/25 . So it would do no harm and compatible for future cases. ## Additional Info Please provide any additional information. For example, future considerations or information useful for reviewers. Question: I am not sure if I should implement tests about `deserialization_G1`, `deserialization_G2` and `hash_to_G2` for the issue. |
||
|
realbigsean
|
6c2d8b2262 |
Builder Specs v0.2.0 (#3134)
## Issue Addressed https://github.com/sigp/lighthouse/issues/3091 Extends https://github.com/sigp/lighthouse/pull/3062, adding pre-bellatrix block support on blinded endpoints and allowing the normal proposal flow (local payload construction) on blinded endpoints. This resulted in better fallback logic because the VC will not have to switch endpoints on failure in the BN <> Builder API, the BN can just fallback immediately and without repeating block processing that it shouldn't need to. We can also keep VC fallback from the VC<>BN API's blinded endpoint to full endpoint. ## Proposed Changes - Pre-bellatrix blocks on blinded endpoints - Add a new `PayloadCache` to the execution layer - Better fallback-from-builder logic ## Todos - [x] Remove VC transition logic - [x] Add logic to only enable builder flow after Merge transition finalization - [x] Tests - [x] Fix metrics - [x] Rustdocs Co-authored-by: Mac L <mjladson@pm.me> Co-authored-by: realbigsean <sean@sigmaprime.io> |
||
|
Michael Sproul
|
d04fde3ba9 |
Remove equivocating validators from fork choice (#3371)
## Issue Addressed Closes https://github.com/sigp/lighthouse/issues/3241 Closes https://github.com/sigp/lighthouse/issues/3242 ## Proposed Changes * [x] Implement logic to remove equivocating validators from fork choice per https://github.com/ethereum/consensus-specs/pull/2845 * [x] Update tests to v1.2.0-rc.1. The new test which exercises `equivocating_indices` is passing. * [x] Pull in some SSZ abstractions from the `tree-states` branch that make implementing Vec-compatible encoding for types like `BTreeSet` and `BTreeMap`. * [x] Implement schema upgrades and downgrades for the database (new schema version is V11). * [x] Apply attester slashings from blocks to fork choice ## Additional Info * This PR doesn't need the `BTreeMap` impl, but `tree-states` does, and I don't think there's any harm in keeping it. But I could also be convinced to drop it. Blocked on #3322. |