mirror of
https://github.com/sigp/lighthouse.git
synced 2026-03-11 04:31:51 +00:00
5ebb44e22275ecd9972e8b3c54fc38c35e9d9d62
63 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
chonghe
|
522bd9e9c6 |
Update Rust Edition to 2024 (#7766)
* #7749 Thanks @dknopik and @michaelsproul for your help! |
||
|
Jimmy Chen
|
40c2fd5ff4 |
Instrument tracing spans for block processing and import (#7816)
#7815 - removes all existing spans, so some span fields that appear in logs like `service_name` may be lost. - instruments a few key code paths in the beacon node, starting from **root spans** named below: * Gossip block and blobs * `process_gossip_data_column_sidecar` * `process_gossip_blob` * `process_gossip_block` * Rpc block and blobs * `process_rpc_block` * `process_rpc_blobs` * `process_rpc_custody_columns` * Rpc blocks (range and backfill) * `process_chain_segment` * `PendingComponents` lifecycle * `pending_components` To test locally: * Run Grafana and Tempo with https://github.com/sigp/lighthouse-metrics/pull/57 * Run Lighthouse BN with `--telemetry-collector-url http://localhost:4317` Some captured traces can be found here: https://hackmd.io/@jimmygchen/r1sLOxPPeg Removing the old spans seem to have reduced the memory usage quite a lot - i think we were using them on long running tasks and too excessively: <img width="910" height="495" alt="image" src="https://github.com/user-attachments/assets/5208bbe4-53b2-4ead-bc71-0b782c788669" /> |
||
|
ethDreamer
|
b08d49c4cb |
Changes for fusaka-devnet-1 (#7559)
Changes for [fusaka-devnet-1](https://notes.ethereum.org/@ethpandaops/fusaka-devnet-1) [Consensus Specs v1.6.0-alpha.1](https://github.com/ethereum/consensus-specs/pull/4346) * [EIP-7917: Deterministic Proposer Lookahead](https://eips.ethereum.org/EIPS/eip-7917) * [EIP-7892: Blob Parameter Only Hardforks](https://eips.ethereum.org/EIPS/eip-7892) |
||
|
Eitan Seri-Levi
|
268809a530 |
Rust clippy 1.87 lint fixes (#7471)
Fix clippy lints for `rustc` 1.87 clippy complains about `BeaconChainError` being too large. I went on a bit of a boxing spree because of this. We may instead want to `Box` some of the `BeaconChainError` variants? |
||
|
Michael Sproul
|
bde0f1ef0b | Merge remote-tracking branch 'origin/release-v7.0.0' into unstable | ||
|
Michael Sproul
|
0875326cb6 | Prevent duplicate effective balance processing (#7209) | ||
|
Paul Hauner
|
8d1abce26e |
Bump SSZ version for larger bitfield SmallVec (#6915)
NA Bumps the `ethereum_ssz` version, along with other crates that share the dep. Primarily, this give us bitfields which can store 128 bytes on the stack before allocating, rather than 32 bytes (https://github.com/sigp/ethereum_ssz/pull/38). The validator count has increase massively since we set it at 32 bytes, so aggregation bitfields (et al) now require a heap allocation. This new value of 128 should get us to ~2m active validators. |
||
|
Pawan Dhananjay
|
266b241123 |
Electra minor refactorings (#6839)
N/A
Fix some typos and other minor refactorings in the electra code. Thanks @jtraglia for bringing them up.
Note to reviewiers:
|
||
|
Michael Sproul
|
06e4d22d49 |
Electra spec changes for v1.5.0-beta.0 (#6731)
* First pass * Add restrictions to RuntimeVariableList api * Use empty_uninitialized and fix warnings * Fix some todos * Merge branch 'unstable' into max-blobs-preset * Fix take impl on RuntimeFixedList * cleanup * Fix test compilations * Fix some more tests * Fix test from unstable * Merge branch 'unstable' into max-blobs-preset * Implement "Bugfix and more withdrawal tests" * Implement "Add missed exit checks to consolidation processing" * Implement "Update initial earliest_exit_epoch calculation" * Implement "Limit consolidating balance by validator.effective_balance" * Implement "Use 16-bit random value in validator filter" * Implement "Do not change creds type on consolidation" * Rename PendingPartialWithdraw index field to validator_index * Skip slots to get test to pass and add TODO * Implement "Synchronously check all transactions to have non-zero length" * Merge remote-tracking branch 'origin/unstable' into max-blobs-preset * Remove footgun function * Minor simplifications * Move from preset to config * Fix typo * Revert "Remove footgun function" This reverts commit |
||
|
Pawan Dhananjay
|
1f6850fae2 |
Rust 1.84 lints (#6781)
* Fix few lints * Fix remaining lints * Use fully qualified syntax |
||
|
Pawan Dhananjay
|
d74b2d96f5 |
Electra alpha8 spec updates (#6496)
* Fix partial withdrawals count * Remove get_active_balance * Remove queue_entire_balance_and_reset_validator * Switch to compounding when consolidating with source==target * Queue deposit requests and apply them during epoch processing * Fix ef tests * Clear todos * Fix engine api formatting issues * Merge branch 'unstable' into electra-alpha7 * Make add_validator_to_registry more in line with the spec * Address some review comments * Cleanup * Update initialize_beacon_state_from_eth1 * Merge branch 'unstable' into electra-alpha7 * Fix rpc decoding for blobs by range/root * Fix block hash computation * Fix process_deposits bug * Merge branch 'unstable' into electra-alpha7 * Fix topup deposit processing bug * Update builder api for electra * Refactor mock builder to separate functionality * Merge branch 'unstable' into electra-alpha7 * Address review comments * Use copied for reference rather than cloned * Optimise and simplify PendingDepositsContext::new * Merge remote-tracking branch 'origin/unstable' into electra-alpha7 * Fix processing of deposits with invalid signatures * Remove redundant code in genesis init * Revert "Refactor mock builder to separate functionality" This reverts commit |
||
|
Pawan Dhananjay
|
83d5c521d7 |
Electra updates for v1.5.0-alpha.6 (#6445)
* Update process_slashing * Update test vectors version * Delete Domain::Consolidation * Rename to get_max_effective_balance * Fix unused; lint * Add the pre-electra slashing processing * lint * Change request json types * Move requests from payload to beacon block body * Refactor engine api * Fix warnings * Update engine api to latest * engine api changed..again * yet again * Merge branch 'engine-requests' into electra-updates * Fix tests * Store reference instead of bytes in NewPayloadRequest * Merge branch 'unstable' into electra-updates * Update beacon_node/execution_layer/src/engine_api/json_structures.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> * Update beacon_node/execution_layer/src/lib.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> * Update beacon_node/execution_layer/src/test_utils/handle_rpc.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> |
||
|
Michael Sproul
|
1447eeb40b |
Improve single-node testnet support and Arc NetworkConfig/ChainSpec (#6396)
* Arc ChainSpec and NetworkConfig * Fix release tests * Fix lint * Merge remote-tracking branch 'origin/unstable' into single-node-testnet |
||
|
Daniel Knopik
|
8b085dd167 |
Fix phase0 block reward in rewards API (#5101)
* Added Block Rewards * added new type * added enum * Fix phase0 block reward in rewards API (#4929) * Merge 'guav00a/proposer-rewards-api' * Merge unstable * Revamp phase0 reward API tests - Add test_rewards_base_slashings (testing #5101) - Improve fix to not include proposer reward in attestation reward API calculation (#4856) - Adjust test approach for phase0 tests: Pad with empty epochs to include all rewards in calculation - Simplify and unify code across all reward tests * Merge branch 'unstable' into fix-4929 * Merge branch 'unstable' into fix-4929 * Merge remote-tracking branch 'origin/unstable' into fix-4929 * Fix merge fallout * Remove junk revived in merge * Address review - check for attestations with lower inclusion delay - check for double attestations in block - add test * Merge branch 'unstable' into fix-4929 * Merge branch 'unstable' into fix-4929 |
||
|
Eitan Seri-Levi
|
99e53b88c3 |
Migrate from ethereum-types to alloy-primitives (#6078)
* Remove use of ethers_core::RlpStream
* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core
* Remove old code
* Simplify keccak call
* Remove unused package
* Merge branch 'unstable' of https://github.com/ethDreamer/lighthouse into remove_use_of_ethers_core
* Merge branch 'unstable' into remove_use_of_ethers_core
* Run clippy
* Merge branch 'remove_use_of_ethers_core' of https://github.com/dospore/lighthouse into remove_use_of_ethers_core
* Check all cargo fmt
* migrate to alloy primitives init
* fix deps
* integrate alloy-primitives
* resolve dep issues
* more changes based on dep changes
* add TODOs
* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core
* Revert lock
* Add BeaconBlocksByRange v3
* continue migration
* Revert "Add BeaconBlocksByRange v3"
This reverts commit
|
||
|
Michael Sproul
|
e09fe5a372 |
Implement Electra spec v1.5.0-alpha.5 (#6305)
* Implement spec v1.5.0-alpha.5 * Merge remote-tracking branch 'origin/unstable' into spec-1.5.0-alpha.5 * Ignore broken Electra light client tests |
||
|
ethDreamer
|
9a295d09b4 |
Implement Electra devnet-1 changes (#5937)
* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix genesis from eth1 for electra * Fix Electra Fork Choice Tests (#5764) * add new operation tests * Fix Electra Fork Choice Tests (#5764) * Fix Electra Fork Choice Tests (#5764) * Fix Electra Fork Choice Tests (#5764) * add epoch processing new tests * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix ef test compilation * disable and enable ef test for epoch processing appropriately * add consolidations to merkle calc for inclusion proof * fix should_override_fc tests * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * get ssz tests compiling * ssz merkle validity for only deneb * im dumb af * Merge remote-tracking branch 'upstream/ef-tests-electra' into ef-tests-electra * enable tests * Fix Consolidation Sigs & Withdrawals * Merge pull request #5766 from ethDreamer/two_fixes Fix Consolidation Sigs & Withdrawals * Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * comment out consolidation operation ef tests until they are added * add signed consolidation ssz tests * add as_iter to all List and Vector fields in the beacon state * remove dbg * Send unagg attestation based on fork * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Fix Epoch Processing Consolidations * Fix Final Spec Tests! * Fix ser/de * Merge branch 'electra-engine-api' into beacon-api-electra * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * update electra readiness with new endpoints * don't block mev boost till genesis finalization * Publish all aggregates * just one more check bro plz.. * Fix Bug In Block Processing with 0x02 Credentials * fix slashing handling * Fix Bug In Block Processing with 0x02 Credentials * Merge remote-tracking branch 'upstream/unstable' * Send unagg attestation based on fork * Publish all aggregates * just one more check bro plz.. * Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable Merge `unstable` into `electra_attestation_changes` * Merge pull request #5835 from realbigsean/fix-validator-logic Fix validator logic * Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling Electra slashing handling * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Rename deploy block electra (#5853) * Rename deploy_block.txt to deposit_contract_block.txt * fmt * Change for prater too * fix: serde rename camle case for execution payload body (#5846) * Merge branch 'electra-engine-api' into beacon-api-electra * Merge branch 'beacon-api-electra' into p2p-electra * Merge branch 'p2p-electra' into ef-tests-electra * Electra attestation changes rm decode impl (#5856) * Remove Crappy Decode impl for Attestation * Remove Inefficient Attestation Decode impl * Implement Schema Upgrade / Downgrade * Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Fix failing attestation tests and misc electra attestation cleanup (#5810) * - get attestation related beacon chain tests to pass - observed attestations are now keyed off of data + committee index - rename op pool attestationref to compactattestationref - remove unwraps in agg pool and use options instead - cherry pick some changes from ef-tests-electra * cargo fmt * fix failing test * Revert dockerfile changes * make committee_index return option * function args shouldnt be a ref to attestation ref * fmt * fix dup imports --------- Co-authored-by: realbigsean <seananderson33@GMAIL.com> * fix some todos (#5817) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * add consolidations to merkle calc for inclusion proof * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Remove Duplicate KZG Commitment Merkle Proof Code (#5874) * Remove Duplicate KZG Commitment Merkle Proof Code * s/tree_lists/fields/ * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * fix compile * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Fix slasher tests (#5906) * Fix electra tests * Add electra attestations to double vote tests * Update superstruct to 0.8 * Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes * Small cleanup in slasher tests * Clean up Electra observed aggregates (#5929) * Use consistent key in observed_attestations * Remove unwraps from observed aggregates * move committee bits to the last index in attestation (#5924) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * De-dup attestation constructor logic * Remove unwraps in Attestation construction * Dedup match_attestation_data * Remove outdated TODO * Use ForkName Ord in fork-choice tests * Use ForkName Ord in BeaconBlockBody * Make to_electra not fallible * Remove TestRandom impl for IndexedAttestation * Remove IndexedAttestation faulty Decode impl * Drop TestRandom impl * Add PendingAttestationInElectra * Indexed att on disk (#35) * indexed att on disk * fix lints * Update slasher/src/migrate.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * add electra fork enabled fn to ForkName impl (#36) * add electra fork enabled fn to ForkName impl * remove inadvertent file * Update common/eth2/src/types.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * Dedup attestation constructor logic in attester cache * Use if let Ok for committee_bits * Dedup Attestation constructor code * Diff reduction in tests * Fix beacon_chain tests * Diff reduction * Use Ord for ForkName in pubsub * Resolve into_attestation_and_indices todo * Remove stale TODO * Fix beacon_chain tests * Test spec invariant * Use electra_enabled in pubsub * Remove get_indexed_attestation_from_signed_aggregate * Use ok_or instead of if let else * committees are sorted * remove dup method `get_indexed_attestation_from_committees` * Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview Electra attestations #5712 review * update default persisted op pool deserialization * ensure aggregate and proof uses serde untagged on ref * Fork aware ssz static attestation tests * Electra attestation changes from Lions review (#5971) * dedup/cleanup and remove unneeded hashset use * remove irrelevant TODOs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Fix Compilation Break * Merge pull request #5973 from ethDreamer/beacon-api-electra Fix Compilation Break * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Rename Constants, Structures, and Fields * Second Round * Third Round * Round Four * Electra attestation changes sean review (#5972) * instantiate empty bitlist in unreachable code * clean up error conversion * fork enabled bool cleanup * remove a couple todos * return bools instead of options in `aggregate` and use the result * delete commented out code * use map macros in simple transformations * remove signers_disjoint_from * get ef tests compiling * get ef tests compiling * update intentionally excluded files * Avoid changing slasher schema for Electra * Delete slasher schema v4 * Fix clippy * Fix compilation of beacon_chain tests * Update database.rs * Update per_block_processing.rs * Add electra lightclient types * Update slasher/src/database.rs * fix imports * Merge pull request #5980 from dapplion/electra-lightclient Add electra lightclient types * Merge pull request #5975 from michaelsproul/electra-slasher-no-migration Avoid changing slasher schema for Electra * Update beacon_node/beacon_chain/src/attestation_verification.rs * Update beacon_node/beacon_chain/src/attestation_verification.rs * Merge pull request #5974 from ethDreamer/electra-devnet-1 Electra: `devnet-1` Rename and Move Stuff * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * clean up enabled and disabled fork logic in ef tests * Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1 * Electra devnet 1 process operations (#5994) * Implement `process_consolidation_requests()` * Finish Changes to `process_operations()` * Fix Lint * Fix test * use electra_enabled() * add light client ssz tests * Update `engine-api` for `devnet-1` * disable failing light client ssz tests * correctly exclude light client struct from accessed files * Fix Failing Tests * Fix Calculations Relying on MAX_EB (#5983) * Fix Calculations Relying on MAX_EB * Use spec fn for max_effective_balance * The great renaming receipt -> request * Address some more review comments * Electra: Get `devnet-1` ef-tests Working * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api * Update beacon_node/beacon_chain/src/electra_readiness.rs * Update consensus/types/src/chain_spec.rs * Update pending balance deposit processing (#6005) * Update pending balance deposit processing * Update single_pass.rs --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> * Merge pull request #6001 from ethDreamer/electra-devnet-1-engine-api Update `engine-api` for `devnet-1` * Merge pull request #6019 from ethDreamer/electra-devnet-1-ef-tests Electra: Get `devnet-1` ef-tests Working * Fix Bug in `engine_api` by Renaming Field * Merge pull request #6047 from ethDreamer/electra-devnet-1-engine-api-fix Fix Bug in `engine_api` by Renaming Field * one more stupid rename * update GET requests * update POST requests * add client updates and test updates * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * compile after merge * unwrap -> unwrap_err * self review * fix tests * convert op pool messages to electra in electra * remove methods to post without content header * Add BeaconBlocksByRange v3 * Revert "Add BeaconBlocksByRange v3" This reverts commit |
||
|
realbigsean
|
8a32df756d |
Ef tests electra (#5758)
* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * Merge pull request #5749 from sigp/electra_op_pool Optimise Electra op pool aggregation * don't fail on empty consolidations * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc * Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api * Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra * Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra * Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra * update committee offset * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra * update committee offset * update committee offset * update committee offset * update committee offset * update committee offset * only increment deposit index on state for old deposit flow * Revert "only increment deposit index on state for old deposit flow" This reverts commit |
||
|
Pawan Dhananjay
|
2a13b4f3fa |
Electra engine api (#5743)
* Attestation superstruct changes for EIP 7549 (#5644)
* update
* experiment
* superstruct changes
* revert
* superstruct changes
* fix tests
* indexed attestation
* indexed attestation superstruct
* updated TODOs
* `superstruct` the `AttesterSlashing` (#5636)
* `superstruct` Attester Fork Variants
* Push a little further
* Deal with Encode / Decode of AttesterSlashing
* not so sure about this..
* Stop Encode/Decode Bounds from Propagating Out
* Tons of Changes..
* More Conversions to AttestationRef
* Add AsReference trait (#15)
* Add AsReference trait
* Fix some snafus
* Got it Compiling! :D
* Got Tests Building
* Get beacon chain tests compiling
---------
Co-authored-by: Michael Sproul <micsproul@gmail.com>
* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes
* Make EF Tests Fork-Agnostic (#5713)
* Finish EF Test Fork Agnostic (#5714)
* Superstruct `AggregateAndProof` (#5715)
* Upgrade `superstruct` to `0.8.0`
* superstruct `AggregateAndProof`
* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes
* cargo fmt
* Merge pull request #5726 from realbigsean/electra_attestation_changes
Merge unstable into Electra attestation changes
* process withdrawals updates
* cleanup withdrawals processing
* update `process_operations` deposit length check
* add apply_deposit changes
* add execution layer withdrawal request processing
* process deposit receipts
* add consolidation processing
* update process operations function
* exit updates
* clean up
* update slash_validator
* EIP7549 `get_attestation_indices` (#5657)
* get attesting indices electra impl
* fmt
* get tests to pass
* fmt
* fix some beacon chain tests
* fmt
* fix slasher test
* fmt got me again
* fix more tests
* fix tests
* Some small changes (#5739)
* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra
* cargo fmt (#5740)
* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra
* fix attestation verification
* Add new engine api methods
* Fix the versioning of v4 requests
* Handle new engine api methods in mock EL
* Note todo
* Fix todos
* Add support for electra fields in getPayloadBodies
* Add comments for potential versioning confusion
* Sketch op pool changes
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra
* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
* Get `electra_op_pool` up to date (#5756)
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
---------
Co-authored-by: realbigsean <sean@sigmaprime.io>
* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)
This reverts commit
|
||
|
Michael Sproul
|
16b81132ca |
Electra epoch processing (#5761)
* Attestation superstruct changes for EIP 7549 (#5644)
* update
* experiment
* superstruct changes
* revert
* superstruct changes
* fix tests
* indexed attestation
* indexed attestation superstruct
* updated TODOs
* `superstruct` the `AttesterSlashing` (#5636)
* `superstruct` Attester Fork Variants
* Push a little further
* Deal with Encode / Decode of AttesterSlashing
* not so sure about this..
* Stop Encode/Decode Bounds from Propagating Out
* Tons of Changes..
* More Conversions to AttestationRef
* Add AsReference trait (#15)
* Add AsReference trait
* Fix some snafus
* Got it Compiling! :D
* Got Tests Building
* Get beacon chain tests compiling
---------
Co-authored-by: Michael Sproul <micsproul@gmail.com>
* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes
* Make EF Tests Fork-Agnostic (#5713)
* Finish EF Test Fork Agnostic (#5714)
* Superstruct `AggregateAndProof` (#5715)
* Upgrade `superstruct` to `0.8.0`
* superstruct `AggregateAndProof`
* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes
* cargo fmt
* Merge pull request #5726 from realbigsean/electra_attestation_changes
Merge unstable into Electra attestation changes
* process withdrawals updates
* cleanup withdrawals processing
* update `process_operations` deposit length check
* add apply_deposit changes
* add execution layer withdrawal request processing
* process deposit receipts
* add consolidation processing
* update process operations function
* exit updates
* clean up
* update slash_validator
* EIP7549 `get_attestation_indices` (#5657)
* get attesting indices electra impl
* fmt
* get tests to pass
* fmt
* fix some beacon chain tests
* fmt
* fix slasher test
* fmt got me again
* fix more tests
* fix tests
* Some small changes (#5739)
* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra
* cargo fmt (#5740)
* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra
* fix attestation verification
* Sketch op pool changes
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
* Get `electra_op_pool` up to date (#5756)
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
---------
Co-authored-by: realbigsean <sean@sigmaprime.io>
* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)
This reverts commit
|
||
|
ethDreamer
|
c52c598f69 |
Electra: Remaining Consensus Data Structures (#5712)
* Attestation superstruct changes for EIP 7549 (#5644)
* update
* experiment
* superstruct changes
* revert
* superstruct changes
* fix tests
* indexed attestation
* indexed attestation superstruct
* updated TODOs
* `superstruct` the `AttesterSlashing` (#5636)
* `superstruct` Attester Fork Variants
* Push a little further
* Deal with Encode / Decode of AttesterSlashing
* not so sure about this..
* Stop Encode/Decode Bounds from Propagating Out
* Tons of Changes..
* More Conversions to AttestationRef
* Add AsReference trait (#15)
* Add AsReference trait
* Fix some snafus
* Got it Compiling! :D
* Got Tests Building
* Get beacon chain tests compiling
---------
Co-authored-by: Michael Sproul <micsproul@gmail.com>
* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes
* Make EF Tests Fork-Agnostic (#5713)
* Finish EF Test Fork Agnostic (#5714)
* Superstruct `AggregateAndProof` (#5715)
* Upgrade `superstruct` to `0.8.0`
* superstruct `AggregateAndProof`
* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes
* cargo fmt
* Merge pull request #5726 from realbigsean/electra_attestation_changes
Merge unstable into Electra attestation changes
* EIP7549 `get_attestation_indices` (#5657)
* get attesting indices electra impl
* fmt
* get tests to pass
* fmt
* fix some beacon chain tests
* fmt
* fix slasher test
* fmt got me again
* fix more tests
* fix tests
* Some small changes (#5739)
* cargo fmt (#5740)
* Sketch op pool changes
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
* Get `electra_op_pool` up to date (#5756)
* fix get attesting indices (#5742)
* fix get attesting indices
* better errors
* fix compile
* only get committee index once
* Ef test fixes (#5753)
* attestation related ef test fixes
* delete commented out stuff
* Fix Aggregation Pool for Electra (#5754)
* Fix Aggregation Pool for Electra
* Remove Outdated Interface
* fix ssz (#5755)
---------
Co-authored-by: realbigsean <sean@sigmaprime.io>
* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)
This reverts commit
|
||
|
realbigsean
|
a74098044a |
Rust 1.79 lints (#5927)
* max_value -> MAX
* remove unnecesary closures
* a couple more max_value -> MAX
* a couple more max_value -> MAX
* Revert "a couple more max_value -> MAX"
This reverts commit
|
||
|
Pawan Dhananjay
|
1af3f0f9d8 |
Make modified helpers in electra fork aware (#5698)
* Make modified helpers in electra fork aware * Make more functions fork aware * formatting fixes only. |
||
|
Michael Sproul
|
ee974db0ba |
Add metric for current epoch total balance (#5688)
* Add metric for current epoch total balance |
||
|
Michael Sproul
|
61962898e2 |
In-memory tree states (#5533)
* Consensus changes
* EF tests
* lcli
* common and watch
* account manager
* cargo
* fork choice
* promise cache
* beacon chain
* interop genesis
* http api
* lighthouse
* op pool
* beacon chain misc
* parallel state cache
* store
* fix issues in store
* IT COMPILES
* Remove some unnecessary module qualification
* Revert Arced pubkey optimization (#5536)
* Merge remote-tracking branch 'origin/unstable' into tree-states-memory
* Fix caching, rebasing and some tests
* Remove unused deps
* Merge remote-tracking branch 'origin/unstable' into tree-states-memory
* Small cleanups
* Revert shuffling cache/promise cache changes
* Fix state advance bugs
* Fix shuffling tests
* Remove some resolved FIXMEs
* Remove StateProcessingStrategy
* Optimise withdrawals calculation
* Don't reorg if state cache is missed
* Remove inconsistent state func
* Fix beta compiler
* Rebase early, rebase often
* Fix state caching behaviour
* Update to milhouse release
* Fix on-disk consensus context format
* Merge remote-tracking branch 'origin/unstable' into tree-states-memory
* Squashed commit of the following:
commit
|
||
|
ethDreamer
|
cda926ce1b |
Rename Functions to More Closely Match Spec (#5591)
* Rename Functions to More Closely Match Spec |
||
|
GeemoCandama
|
32be063f0f |
Support LightClientFinalityUpdate and LightClientOptimisticUpdate rpcs (#3849)
* add light client optimistic and finality update rpc * Arc the updates in the response * add conditional advertisement for both LightClientOptimisticUpdate and LightClientFinalityUpdate * alter display for inboundrequest light client optimistic and finality updates * remove LightClientOptimistic/FinalityReuest struct and some minor fixes * rebase * failing rpc_test for LightClientBootstrap and beginning of MockLib2pLightClient * minor change * added MockRPCHandler by importing everything except OutboundRequest. Need to implement the ConnectionHandler trait now should be copy pastable * almost there but ran into issue where needed to implement BaseOutboundRequest. * failing but running with a light client service of sorts * small test change * changed Protocol::LightClientBootstrap response limit * deleted some stuff from ConnectionHandler Implementation for the mock light client if you need to make something with multiple requests work maybe check here * deleted purging expired inbound/outbound streams code * deleted drive inbound streams that need to be processed * removed unused imports * made things private again * deleted inject_fully_negotiated_inbound * made more things private again * more * turned the logger off in the test * added failing test for new rpc * add rate limit for new rpcs * change InboundUpgrade function to use new rpcs. fmt. add test for LightClientFinalityUpdate * rebasing fix * add LightClientUpdate to handle_rpc functions * added context bytes * fmt * use correct unsed_tcp4_port function * fix for recent config changes and adding context_bytes for the light client protocols * fix clippy complaint * Merge branch 'unstable' into lc-reqresp # Conflicts: # beacon_node/beacon_processor/src/lib.rs # beacon_node/lighthouse_network/src/peer_manager/mod.rs # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/config.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/lighthouse_network/src/rpc/mod.rs # beacon_node/lighthouse_network/src/rpc/outbound.rs # beacon_node/lighthouse_network/src/rpc/protocol.rs # beacon_node/lighthouse_network/src/rpc/rate_limiter.rs # beacon_node/lighthouse_network/src/rpc/self_limiter.rs # beacon_node/lighthouse_network/src/service/api_types.rs # beacon_node/lighthouse_network/tests/common/mod.rs # beacon_node/lighthouse_network/tests/rpc_tests.rs # beacon_node/network/src/network_beacon_processor/rpc_methods.rs # beacon_node/network/src/router.rs * Error handling updates and various cleanups. * Moar minor clean ups. * Do not ban peer for rate limiting light client requests * Merge branch 'unstable' into lc-reqresp. Also removed the mock light client tests to make it compile (See #4940). # Conflicts: # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/methods.rs # beacon_node/lighthouse_network/src/rpc/mod.rs # beacon_node/lighthouse_network/src/rpc/protocol.rs # beacon_node/lighthouse_network/src/service/api_types.rs # beacon_node/lighthouse_network/tests/common/mod.rs # beacon_node/network/src/network_beacon_processor/rpc_methods.rs # beacon_node/network/src/router.rs # consensus/types/src/light_client_bootstrap.rs # consensus/types/src/light_client_finality_update.rs # consensus/types/src/light_client_optimistic_update.rs * Remove unnecessary changes * Add missing light client queue handling. * Merge branch 'unstable' into lc-reqresp * Merge branch 'unstable' into lc-reqresp # Conflicts: # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/service/api_types.rs # consensus/types/src/light_client_finality_update.rs # consensus/types/src/light_client_optimistic_update.rs * Add context bytes for light client RPC responses. * Add RPC limits for light client object. * Fix lint * Fix incorrect light client max size computation. * Merge branch 'unstable' into lc-reqresp # Conflicts: # beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs # beacon_node/lighthouse_network/src/rpc/protocol.rs # beacon_node/lighthouse_network/src/service/api_types.rs * Remove unwanted local changes. * Merge branch 'unstable' into lc-reqresp * Replace `unimplemented` electra code path with deneb values. |
||
|
Michael Sproul
|
06eb181b42 |
Delete ParticipationCache (#5525)
* Delete `ParticipationCache` |
||
|
Michael Sproul
|
feb531f85b |
Single-pass epoch processing and optimised block processing (#5279)
* Single-pass epoch processing (#4483, #4573) Co-authored-by: Michael Sproul <michael@sigmaprime.io> * Delete unused epoch processing code (#5170) * Delete unused epoch processing code * Compare total deltas * Remove unnecessary apply_pending * cargo fmt * Remove newline * Use epoch cache in block packing (#5223) * Remove progressive balances mode (#5224) * inline inactivity_penalty_quotient_for_state * drop previous_epoch_total_active_balance * fc lint * spec compliant process_sync_aggregate (#15) * spec compliant process_sync_aggregate * Update consensus/state_processing/src/per_block_processing/altair/sync_committee.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Delete the participation cache (#16) * update help * Fix op_pool tests * Fix fork choice tests * Merge remote-tracking branch 'sigp/unstable' into epoch-single-pass * Simplify exit cache (#5280) * Fix clippy on exit cache * Clean up single-pass a bit (#5282) * Address Mark's review of single-pass (#5386) * Merge remote-tracking branch 'origin/unstable' into epoch-single-pass * Address Sean's review comments (#5414) * Address most of Sean's review comments * Simplify total balance cache building * Clean up unused junk * Merge remote-tracking branch 'origin/unstable' into epoch-single-pass * More self-review * Merge remote-tracking branch 'origin/unstable' into epoch-single-pass * Merge branch 'unstable' into epoch-single-pass * Fix imports for beta compiler * Fix tests, probably |
||
|
Mac L
|
969d12dc6f |
Use E for EthSpec globally (#5264)
* Use `E` for `EthSpec` globally * Fix tests * Merge branch 'unstable' into e-ethspec * Merge branch 'unstable' into e-ethspec # Conflicts: # beacon_node/execution_layer/src/engine_api.rs # beacon_node/execution_layer/src/engine_api/http.rs # beacon_node/execution_layer/src/engine_api/json_structures.rs # beacon_node/execution_layer/src/test_utils/handle_rpc.rs # beacon_node/store/src/partial_beacon_state.rs # consensus/types/src/beacon_block.rs # consensus/types/src/beacon_block_body.rs # consensus/types/src/beacon_state.rs # consensus/types/src/config_and_preset.rs # consensus/types/src/execution_payload.rs # consensus/types/src/execution_payload_header.rs # consensus/types/src/light_client_optimistic_update.rs # consensus/types/src/payload.rs # lcli/src/parse_ssz.rs |
||
|
Eitan Seri-Levi
|
01ec42e75a |
Fix Rust beta compiler errors 1.78.0-beta.1 (#5439)
* remove redundant imports * fix test * contains key * fmt * Merge branch 'unstable' into fix-beta-compiler |
||
|
Jimmy Chen
|
c0b6b92f27 |
Merge unstable 20230925 into deneb-free-blobs.
|
||
|
Lion - dapplion
|
5c5afafc0d |
Update Deneb to 1.4.0-beta.2 (devnet-9) (#4735)
* Add MAX_PER_EPOCH_ACTIVATION_CHURN_LIMIT * Update tests to 1.4.0-beta.2 * Implement equivocation check for proposer boost * Use hotfix tests and fix minimal config * Start updating fork choice tests for Deneb * Finish implementing fork choice blob handling --------- Co-authored-by: Michael Sproul <michael@sigmaprime.io> |
||
|
João Oliveira
|
dcd69dfc62 |
Move dependencies to workspace (#4650)
## Issue Addressed Synchronize dependencies and edition on the workspace `Cargo.toml` ## Proposed Changes with https://github.com/rust-lang/cargo/issues/8415 merged it's now possible to synchronize details on the workspace `Cargo.toml` like the metadata and dependencies. By only having dependencies that are shared between multiple crates aligned on the workspace `Cargo.toml` it's easier to not miss duplicate versions of the same dependency and therefore ease on the compile times. ## Additional Info this PR also removes the no longer required direct dependency of the `serde_derive` crate. should be reviewed after https://github.com/sigp/lighthouse/pull/4639 get's merged. closes https://github.com/sigp/lighthouse/issues/4651 Co-authored-by: Michael Sproul <michael@sigmaprime.io> Co-authored-by: Michael Sproul <micsproul@gmail.com> |
||
|
zhiqiangxu
|
501ce62d7c |
minor optimize process_active_validator: avoid a call to state.get_validator (#4608)
|
||
|
Jimmy Chen
|
fc7f1ba6b9 |
Phase 0 attestation rewards via Beacon API (#4474)
## Issue Addressed Addresses #4026. Beacon-API spec [here](https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Beacon/getAttestationsRewards). Endpoint: `POST /eth/v1/beacon/rewards/attestations/{epoch}` This endpoint already supports post-Altair epochs. This PR adds support for phase 0 rewards calculation. ## Proposed Changes - [x] Attestation rewards API to support phase 0 rewards calculation, re-using logic from `state_processing`. Refactored `get_attestation_deltas` slightly to support computing deltas for a subset of validators. - [x] Add `inclusion_delay` to `ideal_rewards` (`beacon-API` spec update to follow) - [x] Add `inactivity` penalties to both `ideal_rewards` and `total_rewards` (`beacon-API` spec update to follow) - [x] Add tests to compute attestation rewards and compare results with beacon states ## Additional Notes - The extra penalty for missing attestations or being slashed during an inactivity leak is currently not included in the API response (for both phase 0 and Altair) in the spec. - I went with adding `inactivity` as a separate component rather than combining them with the 4 rewards, because this is how it was grouped in [the phase 0 spec](https://github.com/ethereum/consensus-specs/blob/dev/specs/phase0/beacon-chain.md#get_attestation_deltas). During inactivity leak, all rewards include the optimal reward, and inactivity penalties are calculated separately (see below code snippet from the spec), so it would be quite confusing if we merge them. This would also work better with Altair, because there's no "cancelling" of rewards and inactivity penalties are more separate. - Altair calculation logic (to include inactivity penalties) to be updated in a follow-up PR. ```python def get_attestation_deltas(state: BeaconState) -> Tuple[Sequence[Gwei], Sequence[Gwei]]: """ Return attestation reward/penalty deltas for each validator. """ source_rewards, source_penalties = get_source_deltas(state) target_rewards, target_penalties = get_target_deltas(state) head_rewards, head_penalties = get_head_deltas(state) inclusion_delay_rewards, _ = get_inclusion_delay_deltas(state) _, inactivity_penalties = get_inactivity_penalty_deltas(state) rewards = [ source_rewards[i] + target_rewards[i] + head_rewards[i] + inclusion_delay_rewards[i] for i in range(len(state.validators)) ] penalties = [ source_penalties[i] + target_penalties[i] + head_penalties[i] + inactivity_penalties[i] for i in range(len(state.validators)) ] return rewards, penalties ``` ## Example API Response <details> <summary>Click me</summary> ```json { "ideal_rewards": [ { "effective_balance": "1000000000", "head": "6638", "target": "6638", "source": "6638", "inclusion_delay": "9783", "inactivity": "0" }, { "effective_balance": "2000000000", "head": "13276", "target": "13276", "source": "13276", "inclusion_delay": "19565", "inactivity": "0" }, { "effective_balance": "3000000000", "head": "19914", "target": "19914", "source": "19914", "inclusion_delay": "29349", "inactivity": "0" }, { "effective_balance": "4000000000", "head": "26553", "target": "26553", "source": "26553", "inclusion_delay": "39131", "inactivity": "0" }, { "effective_balance": "5000000000", "head": "33191", "target": "33191", "source": "33191", "inclusion_delay": "48914", "inactivity": "0" }, { "effective_balance": "6000000000", "head": "39829", "target": "39829", "source": "39829", "inclusion_delay": "58697", "inactivity": "0" }, { "effective_balance": "7000000000", "head": "46468", "target": "46468", "source": "46468", "inclusion_delay": "68480", "inactivity": "0" }, { "effective_balance": "8000000000", "head": "53106", "target": "53106", "source": "53106", "inclusion_delay": "78262", "inactivity": "0" }, { "effective_balance": "9000000000", "head": "59744", "target": "59744", "source": "59744", "inclusion_delay": "88046", "inactivity": "0" }, { "effective_balance": "10000000000", "head": "66383", "target": "66383", "source": "66383", "inclusion_delay": "97828", "inactivity": "0" }, { "effective_balance": "11000000000", "head": "73021", "target": "73021", "source": "73021", "inclusion_delay": "107611", "inactivity": "0" }, { "effective_balance": "12000000000", "head": "79659", "target": "79659", "source": "79659", "inclusion_delay": "117394", "inactivity": "0" }, { "effective_balance": "13000000000", "head": "86298", "target": "86298", "source": "86298", "inclusion_delay": "127176", "inactivity": "0" }, { "effective_balance": "14000000000", "head": "92936", "target": "92936", "source": "92936", "inclusion_delay": "136959", "inactivity": "0" }, { "effective_balance": "15000000000", "head": "99574", "target": "99574", "source": "99574", "inclusion_delay": "146742", "inactivity": "0" }, { "effective_balance": "16000000000", "head": "106212", "target": "106212", "source": "106212", "inclusion_delay": "156525", "inactivity": "0" }, { "effective_balance": "17000000000", "head": "112851", "target": "112851", "source": "112851", "inclusion_delay": "166307", "inactivity": "0" }, { "effective_balance": "18000000000", "head": "119489", "target": "119489", "source": "119489", "inclusion_delay": "176091", "inactivity": "0" }, { "effective_balance": "19000000000", "head": "126127", "target": "126127", "source": "126127", "inclusion_delay": "185873", "inactivity": "0" }, { "effective_balance": "20000000000", "head": "132766", "target": "132766", "source": "132766", "inclusion_delay": "195656", "inactivity": "0" }, { "effective_balance": "21000000000", "head": "139404", "target": "139404", "source": "139404", "inclusion_delay": "205439", "inactivity": "0" }, { "effective_balance": "22000000000", "head": "146042", "target": "146042", "source": "146042", "inclusion_delay": "215222", "inactivity": "0" }, { "effective_balance": "23000000000", "head": "152681", "target": "152681", "source": "152681", "inclusion_delay": "225004", "inactivity": "0" }, { "effective_balance": "24000000000", "head": "159319", "target": "159319", "source": "159319", "inclusion_delay": "234787", "inactivity": "0" }, { "effective_balance": "25000000000", "head": "165957", "target": "165957", "source": "165957", "inclusion_delay": "244570", "inactivity": "0" }, { "effective_balance": "26000000000", "head": "172596", "target": "172596", "source": "172596", "inclusion_delay": "254352", "inactivity": "0" }, { "effective_balance": "27000000000", "head": "179234", "target": "179234", "source": "179234", "inclusion_delay": "264136", "inactivity": "0" }, { "effective_balance": "28000000000", "head": "185872", "target": "185872", "source": "185872", "inclusion_delay": "273918", "inactivity": "0" }, { "effective_balance": "29000000000", "head": "192510", "target": "192510", "source": "192510", "inclusion_delay": "283701", "inactivity": "0" }, { "effective_balance": "30000000000", "head": "199149", "target": "199149", "source": "199149", "inclusion_delay": "293484", "inactivity": "0" }, { "effective_balance": "31000000000", "head": "205787", "target": "205787", "source": "205787", "inclusion_delay": "303267", "inactivity": "0" }, { "effective_balance": "32000000000", "head": "212426", "target": "212426", "source": "212426", "inclusion_delay": "313050", "inactivity": "0" } ], "total_rewards": [ { "validator_index": "0", "head": "212426", "target": "212426", "source": "212426", "inclusion_delay": "313050", "inactivity": "0" }, { "validator_index": "32", "head": "212426", "target": "212426", "source": "212426", "inclusion_delay": "313050", "inactivity": "0" }, { "validator_index": "63", "head": "-357771", "target": "-357771", "source": "-357771", "inclusion_delay": "0", "inactivity": "0" } ] } ``` </details> |
||
|
Pawan Dhananjay
|
f2223feb21 |
Rust 1.71 lints (#4503)
## Issue Addressed N/A ## Proposed Changes Add lints for rust 1.71 [ |
||
|
Jimmy Chen
|
46be05f728 |
Cache target attester balances for unrealized FFG progression calculation (#4362)
## Issue Addressed #4118 ## Proposed Changes This PR introduces a "progressive balances" cache on the `BeaconState`, which keeps track of the accumulated target attestation balance for the current & previous epochs. The cached values are utilised by fork choice to calculate unrealized justification and finalization (instead of converting epoch participation arrays to balances for each block we receive). This optimization will be rolled out gradually to allow for more testing. A new `--progressive-balances disabled|checked|strict|fast` flag is introduced to support this: - `checked`: enabled with checks against participation cache, and falls back to the existing epoch processing calculation if there is a total target attester balance mismatch. There is no performance gain from this as the participation cache still needs to be computed. **This is the default mode for now.** - `strict`: enabled with checks against participation cache, returns error if there is a mismatch. **Used for testing only**. - `fast`: enabled with no comparative checks and without computing the participation cache. This mode gives us the performance gains from the optimization. This is still experimental and not currently recommended for production usage, but will become the default mode in a future release. - `disabled`: disable the usage of progressive cache, and use the existing method for FFG progression calculation. This mode may be useful if we find a bug and want to stop the frequent error logs. ### Tasks - [x] Initial cache implementation in `BeaconState` - [x] Perform checks in fork choice to compare the progressive balances cache against results from `ParticipationCache` - [x] Add CLI flag, and disable the optimization by default - [x] Testing on Goerli & Benchmarking - [x] Move caching logic from state processing to the `ProgressiveBalancesCache` (see [this comment](https://github.com/sigp/lighthouse/pull/4362#discussion_r1230877001)) - [x] Add attesting balance metrics Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io> |
||
|
realbigsean
|
98b11bbd3f |
add historical summaries (#3865)
* add historical summaries * fix tree hash caching, disable the sanity slots test with fake crypto * add ssz static HistoricalSummary * only store historical summaries after capella * Teach `UpdatePattern` about Capella * Tidy EF tests * Clippy Co-authored-by: Michael Sproul <michael@sigmaprime.io> |
||
|
Divma
|
ffbf70e2d9 |
Clippy lints for rust 1.66 (#3810)
## Issue Addressed Fixes the new clippy lints for rust 1.66 ## Proposed Changes Most of the changes come from: - [unnecessary_cast](https://rust-lang.github.io/rust-clippy/master/index.html#unnecessary_cast) - [iter_kv_map](https://rust-lang.github.io/rust-clippy/master/index.html#iter_kv_map) - [needless_borrow](https://rust-lang.github.io/rust-clippy/master/index.html#needless_borrow) ## Additional Info na |
||
|
Mac L
|
80359d8ddb |
Fix attestation performance API InvalidValidatorIndex error (#3503)
## Issue Addressed When requesting an index which is not active during `start_epoch`, Lighthouse returns: ``` curl "http://localhost:5052/lighthouse/analysis/attestation_performance/999999999?start_epoch=100000&end_epoch=100000" ``` ```json { "code": 500, "message": "INTERNAL_SERVER_ERROR: ParticipationCache(InvalidValidatorIndex(999999999))", "stacktraces": [] } ``` This error occurs even when the index in question becomes active before `end_epoch` which is undesirable as it can prevent larger queries from completing. ## Proposed Changes In the event the index is out-of-bounds (has not yet been activated), simply return all fields as `false`: ``` -> curl "http://localhost:5052/lighthouse/analysis/attestation_performance/999999999?start_epoch=100000&end_epoch=100000" ``` ```json [ { "index": 999999999, "epochs": { "100000": { "active": false, "head": false, "target": false, "source": false } } } ] ``` By doing this, we cover the case where a validator becomes active sometime between `start_epoch` and `end_epoch`. ## Additional Info Note that this error only occurs for epochs after the Altair hard fork. |
||
|
Michael Sproul
|
66eca1a882 |
Refactor op pool for speed and correctness (#3312)
## Proposed Changes
This PR has two aims: to speed up attestation packing in the op pool, and to fix bugs in the verification of attester slashings, proposer slashings and voluntary exits. The changes are bundled into a single database schema upgrade (v12).
Attestation packing is sped up by removing several inefficiencies:
- No more recalculation of `attesting_indices` during packing.
- No (unnecessary) examination of the `ParticipationFlags`: a bitfield suffices. See `RewardCache`.
- No re-checking of attestation validity during packing: the `AttestationMap` provides attestations which are "correct by construction" (I have checked this using Hydra).
- No SSZ re-serialization for the clunky `AttestationId` type (it can be removed in a future release).
So far the speed-up seems to be roughly 2-10x, from 500ms down to 50-100ms.
Verification of attester slashings, proposer slashings and voluntary exits is fixed by:
- Tracking the `ForkVersion`s that were used to verify each message inside the `SigVerifiedOp`. This allows us to quickly re-verify that they match the head state's opinion of what the `ForkVersion` should be at the epoch(s) relevant to the message.
- Storing the `SigVerifiedOp` on disk rather than the raw operation. This allows us to continue track the fork versions after a reboot.
This is mostly contained in this commit
|
||
|
realbigsean
|
20ebf1f3c1 |
Realized unrealized experimentation (#3322)
## Issue Addressed Add a flag that optionally enables unrealized vote tracking. Would like to test out on testnets and benchmark differences in methods of vote tracking. This PR includes a DB schema upgrade to enable to new vote tracking style. Co-authored-by: realbigsean <sean@sigmaprime.io> Co-authored-by: Paul Hauner <paul@paulhauner.com> Co-authored-by: sean <seananderson33@gmail.com> Co-authored-by: Mac L <mjladson@pm.me> |
||
|
Paul Hauner
|
be4e261e74 |
Use async code when interacting with EL (#3244)
## Overview
This rather extensive PR achieves two primary goals:
1. Uses the finalized/justified checkpoints of fork choice (FC), rather than that of the head state.
2. Refactors fork choice, block production and block processing to `async` functions.
Additionally, it achieves:
- Concurrent forkchoice updates to the EL and cache pruning after a new head is selected.
- Concurrent "block packing" (attestations, etc) and execution payload retrieval during block production.
- Concurrent per-block-processing and execution payload verification during block processing.
- The `Arc`-ification of `SignedBeaconBlock` during block processing (it's never mutated, so why not?):
- I had to do this to deal with sending blocks into spawned tasks.
- Previously we were cloning the beacon block at least 2 times during each block processing, these clones are either removed or turned into cheaper `Arc` clones.
- We were also `Box`-ing and un-`Box`-ing beacon blocks as they moved throughout the networking crate. This is not a big deal, but it's nice to avoid shifting things between the stack and heap.
- Avoids cloning *all the blocks* in *every chain segment* during sync.
- It also has the potential to clean up our code where we need to pass an *owned* block around so we can send it back in the case of an error (I didn't do much of this, my PR is already big enough 😅)
- The `BeaconChain::HeadSafetyStatus` struct was removed. It was an old relic from prior merge specs.
For motivation for this change, see https://github.com/sigp/lighthouse/pull/3244#issuecomment-1160963273
## Changes to `canonical_head` and `fork_choice`
Previously, the `BeaconChain` had two separate fields:
```
canonical_head: RwLock<Snapshot>,
fork_choice: RwLock<BeaconForkChoice>
```
Now, we have grouped these values under a single struct:
```
canonical_head: CanonicalHead {
cached_head: RwLock<Arc<Snapshot>>,
fork_choice: RwLock<BeaconForkChoice>
}
```
Apart from ergonomics, the only *actual* change here is wrapping the canonical head snapshot in an `Arc`. This means that we no longer need to hold the `cached_head` (`canonical_head`, in old terms) lock when we want to pull some values from it. This was done to avoid deadlock risks by preventing functions from acquiring (and holding) the `cached_head` and `fork_choice` locks simultaneously.
## Breaking Changes
### The `state` (root) field in the `finalized_checkpoint` SSE event
Consider the scenario where epoch `n` is just finalized, but `start_slot(n)` is skipped. There are two state roots we might in the `finalized_checkpoint` SSE event:
1. The state root of the finalized block, which is `get_block(finalized_checkpoint.root).state_root`.
4. The state root at slot of `start_slot(n)`, which would be the state from (1), but "skipped forward" through any skip slots.
Previously, Lighthouse would choose (2). However, we can see that when [Teku generates that event](
|
||
|
Paul Hauner
|
11d80a6a38 |
Optimise per_epoch_processing low-hanging-fruit (#3254)
## Issue Addressed NA ## Proposed Changes - Uses a `Vec` in `SingleEpochParticipationCache` rather than `HashMap` to speed up processing times at the cost of memory usage. - Cache the result of `integer_sqrt` rather than recomputing for each validator. - Cache `state.previous_epoch` rather than recomputing it for each validator. ### Benchmarks Benchmarks on a recent mainnet state using #3252 to get timing. #### Without this PR ``` lcli skip-slots --state-path /tmp/state-0x3cdc.ssz --partial-state-advance --slots 32 --state-root 0x3cdc33cd02713d8d6cc33a6dbe2d3a5bf9af1d357de0d175a403496486ff845e --runs 10 [2022-06-09T08:21:02Z INFO lcli::skip_slots] Using mainnet spec [2022-06-09T08:21:02Z INFO lcli::skip_slots] Advancing 32 slots [2022-06-09T08:21:02Z INFO lcli::skip_slots] Doing 10 runs [2022-06-09T08:21:02Z INFO lcli::skip_slots] State path: "/tmp/state-0x3cdc.ssz" SSZ decoding /tmp/state-0x3cdc.ssz: 43ms [2022-06-09T08:21:03Z INFO lcli::skip_slots] Run 0: 245.718794ms [2022-06-09T08:21:03Z INFO lcli::skip_slots] Run 1: 245.364782ms [2022-06-09T08:21:03Z INFO lcli::skip_slots] Run 2: 255.866179ms [2022-06-09T08:21:04Z INFO lcli::skip_slots] Run 3: 243.838909ms [2022-06-09T08:21:04Z INFO lcli::skip_slots] Run 4: 250.431425ms [2022-06-09T08:21:04Z INFO lcli::skip_slots] Run 5: 248.68765ms [2022-06-09T08:21:04Z INFO lcli::skip_slots] Run 6: 262.051113ms [2022-06-09T08:21:05Z INFO lcli::skip_slots] Run 7: 264.293967ms [2022-06-09T08:21:05Z INFO lcli::skip_slots] Run 8: 293.202007ms [2022-06-09T08:21:05Z INFO lcli::skip_slots] Run 9: 264.552017ms ``` #### With this PR: ``` lcli skip-slots --state-path /tmp/state-0x3cdc.ssz --partial-state-advance --slots 32 --state-root 0x3cdc33cd02713d8d6cc33a6dbe2d3a5bf9af1d357de0d175a403496486ff845e --runs 10 [2022-06-09T08:57:59Z INFO lcli::skip_slots] Run 0: 73.898678ms [2022-06-09T08:57:59Z INFO lcli::skip_slots] Run 1: 75.536978ms [2022-06-09T08:57:59Z INFO lcli::skip_slots] Run 2: 75.176104ms [2022-06-09T08:57:59Z INFO lcli::skip_slots] Run 3: 76.460828ms [2022-06-09T08:57:59Z INFO lcli::skip_slots] Run 4: 75.904195ms [2022-06-09T08:58:00Z INFO lcli::skip_slots] Run 5: 75.53077ms [2022-06-09T08:58:00Z INFO lcli::skip_slots] Run 6: 74.745572ms [2022-06-09T08:58:00Z INFO lcli::skip_slots] Run 7: 75.823489ms [2022-06-09T08:58:00Z INFO lcli::skip_slots] Run 8: 74.892055ms [2022-06-09T08:58:00Z INFO lcli::skip_slots] Run 9: 76.333569ms ``` ## Additional Info NA |
||
|
Michael Sproul
|
bac7c3fa54 |
v2.2.0 (#3139)
## Proposed Changes Cut release v2.2.0 including proposer boost. ## Additional Info I also updated the clippy lints for the imminent release of Rust 1.60, although LH v2.2.0 will continue to compile using Rust 1.58 (our MSRV). |
||
|
Michael Sproul
|
5e1f8a8480 |
Update to Rust 1.59 and 2021 edition (#3038)
## Proposed Changes Lots of lint updates related to `flat_map`, `unwrap_or_else` and string patterns. I did a little more creative refactoring in the op pool, but otherwise followed Clippy's suggestions. ## Additional Info We need this PR to unblock CI. |
||
|
realbigsean
|
de49c7ddaa |
1.1.5 merge spec tests (#2781)
* Fix arbitrary check kintsugi * Add merge chain spec fields, and a function to determine which constant to use based on the state variant * increment spec test version * Remove `Transaction` enum wrapper * Remove Transaction new-type * Remove gas validations * Add `--terminal-block-hash-epoch-override` flag * Increment spec tests version to 1.1.5 * Remove extraneous gossip verification https://github.com/ethereum/consensus-specs/pull/2687 * - Remove unused Error variants - Require both "terminal-block-hash-epoch-override" and "terminal-block-hash-override" when either flag is used * - Remove a couple more unused Error variants Co-authored-by: Paul Hauner <paul@paulhauner.com> |
||
|
Paul Hauner
|
e2d09bb8ac |
Add BeaconChainHarness::builder (#2707)
## Issue Addressed NA ## Proposed Changes This PR is near-identical to https://github.com/sigp/lighthouse/pull/2652, however it is to be merged into `unstable` instead of `merge-f2f`. Please see that PR for reasoning. I'm making this duplicate PR to merge to `unstable` in an effort to shrink the diff between `unstable` and `merge-f2f` by doing smaller, lead-up PRs. ## Additional Info NA |
||
|
Michael Sproul
|
9c785a9b33 |
Optimize process_attestation with active balance cache (#2560)
## Proposed Changes Cache the total active balance for the current epoch in the `BeaconState`. Computing this value takes around 1ms, and this was negatively impacting block processing times on Prater, particularly when reconstructing states. With a large number of attestations in each block, I saw the `process_attestations` function taking 150ms, which means that reconstructing hot states can take up to 4.65s (31 * 150ms), and reconstructing freezer states can take up to 307s (2047 * 150ms). I opted to add the cache to the beacon state rather than computing the total active balance at the start of state processing and threading it through. Although this would be simpler in a way, it would waste time, particularly during block replay, as the total active balance doesn't change for the duration of an epoch. So we save ~32ms for hot states, and up to 8.1s for freezer states (using `--slots-per-restore-point 8192`). |