Implement Electra devnet-1 changes (#5937)

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* fix genesis from eth1 for electra

* Fix Electra Fork Choice Tests (#5764)

* add new operation tests

* Fix Electra Fork Choice Tests (#5764)

* Fix Electra Fork Choice Tests (#5764)

* Fix Electra Fork Choice Tests (#5764)

* add epoch processing new tests

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* fix ef test compilation

* disable and enable ef test for epoch processing appropriately

* add consolidations to merkle calc for inclusion proof

* fix should_override_fc tests

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* get ssz tests compiling

* ssz merkle validity for only deneb

* im dumb af

* Merge remote-tracking branch 'upstream/ef-tests-electra' into ef-tests-electra

* enable tests

* Fix Consolidation Sigs & Withdrawals

* Merge pull request #5766 from ethDreamer/two_fixes

Fix Consolidation Sigs & Withdrawals

* Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* comment out consolidation operation ef tests until they are added

* add signed consolidation ssz tests

* add as_iter to all List and Vector fields in the beacon state

* remove dbg

* Send unagg attestation based on fork

* Merge branch 'beacon-api-electra' into p2p-electra

* Merge branch 'p2p-electra' into ef-tests-electra

* Fix Epoch Processing Consolidations

* Fix Final Spec Tests!

* Fix ser/de

* Merge branch 'electra-engine-api' into beacon-api-electra

* Merge branch 'beacon-api-electra' into p2p-electra

* Merge branch 'p2p-electra' into ef-tests-electra

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* update electra readiness with new endpoints

* don't block mev boost till genesis finalization

* Publish all aggregates

* just one more check bro plz..

* Fix Bug In Block Processing with 0x02 Credentials

* fix slashing handling

* Fix Bug In Block Processing with 0x02 Credentials

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request #5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Rename deploy block electra (#5853)

* Rename deploy_block.txt to deposit_contract_block.txt

* fmt

* Change for prater too

* fix: serde rename camle case for execution payload body (#5846)

* Merge branch 'electra-engine-api' into beacon-api-electra

* Merge branch 'beacon-api-electra' into p2p-electra

* Merge branch 'p2p-electra' into ef-tests-electra

* Electra attestation changes rm decode impl (#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into p2p-electra

* Merge branch 'p2p-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Remove Duplicate KZG Commitment Merkle Proof Code (#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* fix compile

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Fix slasher tests (#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* move committee bits to the last index in attestation (#5924)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations #5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1

* Fix Compilation Break

* Merge pull request #5973 from ethDreamer/beacon-api-electra

Fix Compilation Break

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1

* Rename Constants, Structures, and Fields

* Second Round

* Third Round

* Round Four

* Electra attestation changes sean review (#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Update per_block_processing.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request #5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request #5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Merge pull request #5974 from ethDreamer/electra-devnet-1

Electra: `devnet-1` Rename and Move Stuff

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* clean up enabled and disabled fork logic in ef tests

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1

* Electra devnet 1 process operations (#5994)

* Implement `process_consolidation_requests()`

* Finish Changes to `process_operations()`

* Fix Lint

* Fix test

* use electra_enabled()

* add light client ssz tests

* Update `engine-api` for `devnet-1`

* disable failing light client ssz tests

* correctly exclude light client struct from accessed files

* Fix Failing Tests

* Fix Calculations Relying on MAX_EB (#5983)

* Fix Calculations Relying on MAX_EB

* Use spec fn for max_effective_balance

* The great renaming receipt -> request

* Address some more review comments

* Electra: Get `devnet-1` ef-tests Working

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Update beacon_node/beacon_chain/src/electra_readiness.rs

* Update consensus/types/src/chain_spec.rs

* Update pending balance deposit processing (#6005)

* Update pending balance deposit processing

* Update single_pass.rs

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Merge pull request #6001 from ethDreamer/electra-devnet-1-engine-api

Update `engine-api` for `devnet-1`

* Merge pull request #6019 from ethDreamer/electra-devnet-1-ef-tests

Electra: Get `devnet-1` ef-tests Working

* Fix Bug in `engine_api` by Renaming Field

* Merge pull request #6047 from ethDreamer/electra-devnet-1-engine-api-fix

Fix Bug in `engine_api` by Renaming Field

* one more stupid rename

* update GET requests

* update POST requests

* add client updates and test updates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* compile after merge

* unwrap -> unwrap_err

* self review

* fix tests

* convert op pool messages to electra in electra

* remove methods to post without content header

* Add BeaconBlocksByRange v3

* Revert "Add BeaconBlocksByRange v3"

This reverts commit e3ce7fc5ea.

* filter instead of convert

* Add range sync metrics to track efficiency (#6095)

* Add more range sync metrics to track efficiency

* Add ignored blocks metrics

* Enable the outbound rate limiter by default, and update blobs method quotas (#6093)

* Enable the outbound rate limiter by default, and update blobs method quotas.

* Lint and book updates.

* Beacon api + validator electra (#5744)

* Attestation superstruct changes for EIP 7549 (#5644)

* update

* experiment

* superstruct changes

* revert

* superstruct changes

* fix tests

* indexed attestation

* indexed attestation superstruct

* updated TODOs

* `superstruct` the `AttesterSlashing` (#5636)

* `superstruct` Attester Fork Variants

* Push a little further

* Deal with Encode / Decode of AttesterSlashing

* not so sure about this..

* Stop Encode/Decode Bounds from Propagating Out

* Tons of Changes..

* More Conversions to AttestationRef

* Add AsReference trait (#15)

* Add AsReference trait

* Fix some snafus

* Got it Compiling! :D

* Got Tests Building

* Get beacon chain tests compiling

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes

* Make EF Tests Fork-Agnostic (#5713)

* Finish EF Test Fork Agnostic (#5714)

* Superstruct `AggregateAndProof` (#5715)

* Upgrade `superstruct` to `0.8.0`

* superstruct `AggregateAndProof`

* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes

* cargo fmt

* Merge pull request #5726 from realbigsean/electra_attestation_changes

Merge unstable into Electra attestation changes

* process withdrawals updates

* cleanup withdrawals processing

* update `process_operations` deposit length check

* add apply_deposit changes

* add execution layer withdrawal request processing

* process deposit receipts

* add consolidation processing

* update process operations function

* exit updates

* clean up

* update slash_validator

* EIP7549 `get_attestation_indices` (#5657)

* get attesting indices electra impl

* fmt

* get tests to pass

* fmt

* fix some beacon chain tests

* fmt

* fix slasher test

* fmt got me again

* fix more tests

* fix tests

* Some small changes (#5739)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* cargo fmt (#5740)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* fix attestation verification

* Add new engine api methods

* Fix the versioning of v4 requests

* Handle new engine api methods in mock EL

* Note todo

* Fix todos

* Add support for electra fields in getPayloadBodies

* Add comments for potential versioning confusion

* udpates for aggregate attestation endpoint

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Sketch op pool changes

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

* Get `electra_op_pool` up to date (#5756)

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>

* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)

This reverts commit ab9e58aa3d.

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Compute on chain aggregate impl (#5752)

* add compute_on_chain_agg impl to op pool changes

* fmt

* get op pool tests to pass

* update beacon api aggregate attestationendpoint

* update the naive agg pool interface (#5760)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* updates after merge

* Fix bugs in cross-committee aggregation

* Add comment to max cover optimisation

* Fix assert

* Electra epoch processing

* add deposit limit for old deposit queue

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge pull request #5749 from sigp/electra_op_pool

Optimise Electra op pool aggregation

* don't fail on empty consolidations

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* update committee offset

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* update committee offset

* update committee offset

* update committee offset

* only increment the state deposit index on old deposit flow

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* use correct max eb in epoch cache initialization

* drop initiate validator ordering optimization

* fix initiate exit for single pass

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* accept new payload v4 in mock el

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Electra Fork Choice Tests (#5764)

* Fix Electra Fork Choice Tests (#5764)

* Fix Electra Fork Choice Tests (#5764)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Consolidation Sigs & Withdrawals

* Merge pull request #5766 from ethDreamer/two_fixes

Fix Consolidation Sigs & Withdrawals

* Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Send unagg attestation based on fork

* Fix ser/de

* Merge branch 'electra-engine-api' into beacon-api-electra

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* update electra readiness with new endpoints

* fix slashing handling

* Fix Bug In Block Processing with 0x02 Credentials

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request #5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* fix: serde rename camle case for execution payload body (#5846)

* Merge branch 'electra-engine-api' into beacon-api-electra

* Electra attestation changes rm decode impl (#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Remove Duplicate KZG Commitment Merkle Proof Code (#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* fix compile

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix slasher tests (#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations #5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Compilation Break

* Merge pull request #5973 from ethDreamer/beacon-api-electra

Fix Compilation Break

* Electra attestation changes sean review (#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Update per_block_processing.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request #5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request #5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* The great renaming receipt -> request

* Address some more review comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Update beacon_node/beacon_chain/src/electra_readiness.rs

* Update consensus/types/src/chain_spec.rs

* update GET requests

* update POST requests

* add client updates and test updates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* compile after merge

* unwrap -> unwrap_err

* self review

* fix tests

* convert op pool messages to electra in electra

* remove methods to post without content header

* filter instead of convert

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge branch 'beacon-api-electra' of https://github.com/sigp/lighthouse into ef-tests-electra

* fix ef test compile

* use `enabled` fork methods

* get tests passing

* last updates

* Fix bug on engine api with consolidations (#6101)

* Fix small bug in engine_api

* Merge pull request #6116 from ethDreamer/electra-devnet-1

Fix payload_bodies bug in engine_api

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into ef-tests-electra

* add deposit request file

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into ef-tests-electra

* Merge branch 'ef-tests-electra' of https://github.com/sigp/lighthouse into electra-devnet-1

* clean up after merge

* Fix Broken Test on `devnet-1`

* another small bug

* Merge pull request #6132 from ethDreamer/devnet-1-fix-test

Fix Broken Test on `devnet-1`

* Fix Deserialization Bug in `engine-api`

* Merge pull request #6235 from ethDreamer/fix_deserialization

Fix Deserialization Bug in `engine-api`

* Merge branch 'unstable' into electra-devnet-1

* remove import

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-devnet-1

* Remove false negative checks on `shuffling_is_compatible`

* Fix flaky validator monitor test
This commit is contained in:
ethDreamer
2024-08-21 23:51:43 -05:00
committed by GitHub
parent 677f96a333
commit 9a295d09b4
42 changed files with 880 additions and 630 deletions

View File

@@ -2,7 +2,8 @@ use crate::engines::ForkchoiceState;
use crate::http::{
ENGINE_FORKCHOICE_UPDATED_V1, ENGINE_FORKCHOICE_UPDATED_V2, ENGINE_FORKCHOICE_UPDATED_V3,
ENGINE_GET_CLIENT_VERSION_V1, ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V1,
ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V1, ENGINE_GET_PAYLOAD_V1, ENGINE_GET_PAYLOAD_V2,
ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V2, ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V1,
ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V2, ENGINE_GET_PAYLOAD_V1, ENGINE_GET_PAYLOAD_V2,
ENGINE_GET_PAYLOAD_V3, ENGINE_GET_PAYLOAD_V4, ENGINE_NEW_PAYLOAD_V1, ENGINE_NEW_PAYLOAD_V2,
ENGINE_NEW_PAYLOAD_V3, ENGINE_NEW_PAYLOAD_V4,
};
@@ -20,11 +21,11 @@ use reqwest::StatusCode;
use serde::{Deserialize, Serialize};
use strum::IntoStaticStr;
use superstruct::superstruct;
use types::execution_payload::{DepositRequests, WithdrawalRequests};
use types::execution_payload::{ConsolidationRequests, DepositRequests, WithdrawalRequests};
pub use types::{
Address, BeaconBlockRef, EthSpec, ExecutionBlockHash, ExecutionPayload, ExecutionPayloadHeader,
ExecutionPayloadRef, FixedVector, ForkName, Hash256, Transactions, Uint256, VariableList,
Withdrawal, Withdrawals,
Address, BeaconBlockRef, ConsolidationRequest, EthSpec, ExecutionBlockHash, ExecutionPayload,
ExecutionPayloadHeader, ExecutionPayloadRef, FixedVector, ForkName, Hash256, Transactions,
Uint256, VariableList, Withdrawal, Withdrawals,
};
use types::{
ExecutionPayloadBellatrix, ExecutionPayloadCapella, ExecutionPayloadDeneb,
@@ -42,7 +43,7 @@ pub use new_payload_request::{
NewPayloadRequestDeneb, NewPayloadRequestElectra,
};
use self::json_structures::{JsonDepositRequest, JsonWithdrawalRequest};
use self::json_structures::{JsonConsolidationRequest, JsonDepositRequest, JsonWithdrawalRequest};
pub const LATEST_TAG: &str = "latest";
@@ -74,6 +75,7 @@ pub enum Error {
UnsupportedForkVariant(String),
InvalidClientVersion(String),
RlpDecoderError(rlp::DecoderError),
TooManyConsolidationRequests(usize),
}
impl From<reqwest::Error> for Error {
@@ -206,6 +208,8 @@ pub struct ExecutionBlockWithTransactions<E: EthSpec> {
pub deposit_requests: Vec<JsonDepositRequest>,
#[superstruct(only(Electra))]
pub withdrawal_requests: Vec<JsonWithdrawalRequest>,
#[superstruct(only(Electra))]
pub consolidation_requests: Vec<JsonConsolidationRequest>,
}
impl<E: EthSpec> TryFrom<ExecutionPayload<E>> for ExecutionBlockWithTransactions<E> {
@@ -323,6 +327,11 @@ impl<E: EthSpec> TryFrom<ExecutionPayload<E>> for ExecutionBlockWithTransactions
.into_iter()
.map(|withdrawal| withdrawal.into())
.collect(),
consolidation_requests: block
.consolidation_requests
.into_iter()
.map(Into::into)
.collect(),
})
}
};
@@ -541,27 +550,106 @@ impl<E: EthSpec> GetPayloadResponse<E> {
}
}
#[superstruct(
variants(V1, V2),
variant_attributes(derive(Clone, Debug),),
partial_getter_error(ty = "Error", expr = "Error::IncorrectStateVariant")
)]
#[derive(Clone, Debug)]
pub struct ExecutionPayloadBodyV1<E: EthSpec> {
pub struct ExecutionPayloadBody<E: EthSpec> {
pub transactions: Transactions<E>,
pub withdrawals: Option<Withdrawals<E>>,
#[superstruct(only(V2))]
pub deposit_requests: Option<DepositRequests<E>>,
#[superstruct(only(V2))]
pub withdrawal_requests: Option<WithdrawalRequests<E>>,
#[superstruct(only(V2))]
pub consolidation_requests: Option<ConsolidationRequests<E>>,
}
impl<E: EthSpec> ExecutionPayloadBodyV1<E> {
impl<E: EthSpec> ExecutionPayloadBody<E> {
#[allow(clippy::type_complexity)]
pub fn deconstruct(
self,
) -> (
Transactions<E>,
Option<Withdrawals<E>>,
Option<DepositRequests<E>>,
Option<WithdrawalRequests<E>>,
Option<ConsolidationRequests<E>>,
) {
match self {
ExecutionPayloadBody::V1(body) => {
(body.transactions, body.withdrawals, None, None, None)
}
ExecutionPayloadBody::V2(body) => (
body.transactions,
body.withdrawals,
body.deposit_requests,
body.withdrawal_requests,
body.consolidation_requests,
),
}
}
pub fn to_payload(
self,
header: ExecutionPayloadHeader<E>,
) -> Result<ExecutionPayload<E>, String> {
match header {
ExecutionPayloadHeader::Bellatrix(header) => {
if self.withdrawals.is_some() {
let header_fork = header.fork_name_unchecked();
match &self {
Self::V1(_) => {
if header_fork.electra_enabled() {
return Err(format!(
"block {} is merge but payload body has withdrawals",
header.block_hash
"block {} is {} but response is ExecutionPayloadBodyV1. Does the EL support {}?",
header.block_hash(),
header_fork,
ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V2,
));
}
}
Self::V2(_) => {}
}
let (
transactions,
withdrawals,
deposit_requests,
withdrawal_requests,
consolidation_requests,
) = self.deconstruct();
if !header_fork.capella_enabled() && withdrawals.is_some() {
return Err(format!(
"block {} is {} but payload body has withdrawals",
header.block_hash(),
header_fork
));
}
if !header_fork.electra_enabled() {
if deposit_requests.is_some() {
return Err(format!(
"block {} is {} but payload body has deposit_requests",
header.block_hash(),
header_fork
));
}
if withdrawal_requests.is_some() {
return Err(format!(
"block {} is {} but payload body has withdrawal_requests",
header.block_hash(),
header_fork
));
}
if consolidation_requests.is_some() {
return Err(format!(
"block {} is {} but payload body has consolidation_requests",
header.block_hash(),
header_fork
));
}
}
match header {
ExecutionPayloadHeader::Bellatrix(header) => {
Ok(ExecutionPayload::Bellatrix(ExecutionPayloadBellatrix {
parent_hash: header.parent_hash,
fee_recipient: header.fee_recipient,
@@ -576,100 +664,108 @@ impl<E: EthSpec> ExecutionPayloadBodyV1<E> {
extra_data: header.extra_data,
base_fee_per_gas: header.base_fee_per_gas,
block_hash: header.block_hash,
transactions: self.transactions,
transactions,
}))
}
ExecutionPayloadHeader::Capella(header) => {
if let Some(withdrawals) = self.withdrawals {
Ok(ExecutionPayload::Capella(ExecutionPayloadCapella {
parent_hash: header.parent_hash,
fee_recipient: header.fee_recipient,
state_root: header.state_root,
receipts_root: header.receipts_root,
logs_bloom: header.logs_bloom,
prev_randao: header.prev_randao,
block_number: header.block_number,
gas_limit: header.gas_limit,
gas_used: header.gas_used,
timestamp: header.timestamp,
extra_data: header.extra_data,
base_fee_per_gas: header.base_fee_per_gas,
block_hash: header.block_hash,
transactions: self.transactions,
withdrawals,
}))
} else {
Err(format!(
"block {} is capella but payload body doesn't have withdrawals",
header.block_hash
))
}
let withdrawals = withdrawals.ok_or_else(|| {
format!(
"block {} is {} but payload body has withdrawals set to null",
header.block_hash, header_fork
)
})?;
Ok(ExecutionPayload::Capella(ExecutionPayloadCapella {
parent_hash: header.parent_hash,
fee_recipient: header.fee_recipient,
state_root: header.state_root,
receipts_root: header.receipts_root,
logs_bloom: header.logs_bloom,
prev_randao: header.prev_randao,
block_number: header.block_number,
gas_limit: header.gas_limit,
gas_used: header.gas_used,
timestamp: header.timestamp,
extra_data: header.extra_data,
base_fee_per_gas: header.base_fee_per_gas,
block_hash: header.block_hash,
transactions,
withdrawals,
}))
}
ExecutionPayloadHeader::Deneb(header) => {
if let Some(withdrawals) = self.withdrawals {
Ok(ExecutionPayload::Deneb(ExecutionPayloadDeneb {
parent_hash: header.parent_hash,
fee_recipient: header.fee_recipient,
state_root: header.state_root,
receipts_root: header.receipts_root,
logs_bloom: header.logs_bloom,
prev_randao: header.prev_randao,
block_number: header.block_number,
gas_limit: header.gas_limit,
gas_used: header.gas_used,
timestamp: header.timestamp,
extra_data: header.extra_data,
base_fee_per_gas: header.base_fee_per_gas,
block_hash: header.block_hash,
transactions: self.transactions,
withdrawals,
blob_gas_used: header.blob_gas_used,
excess_blob_gas: header.excess_blob_gas,
}))
} else {
Err(format!(
"block {} is post-capella but payload body doesn't have withdrawals",
header.block_hash
))
}
let withdrawals = withdrawals.ok_or_else(|| {
format!(
"block {} is {} but payload body has withdrawals set to null",
header.block_hash, header_fork
)
})?;
Ok(ExecutionPayload::Deneb(ExecutionPayloadDeneb {
parent_hash: header.parent_hash,
fee_recipient: header.fee_recipient,
state_root: header.state_root,
receipts_root: header.receipts_root,
logs_bloom: header.logs_bloom,
prev_randao: header.prev_randao,
block_number: header.block_number,
gas_limit: header.gas_limit,
gas_used: header.gas_used,
timestamp: header.timestamp,
extra_data: header.extra_data,
base_fee_per_gas: header.base_fee_per_gas,
block_hash: header.block_hash,
transactions,
withdrawals,
blob_gas_used: header.blob_gas_used,
excess_blob_gas: header.excess_blob_gas,
}))
}
ExecutionPayloadHeader::Electra(header) => {
let withdrawals_exist = self.withdrawals.is_some();
let deposit_requests_exist = self.deposit_requests.is_some();
let withdrawal_requests_exist = self.withdrawal_requests.is_some();
if let (Some(withdrawals), Some(deposit_requests), Some(withdrawal_requests)) = (
self.withdrawals,
self.deposit_requests,
self.withdrawal_requests,
) {
Ok(ExecutionPayload::Electra(ExecutionPayloadElectra {
parent_hash: header.parent_hash,
fee_recipient: header.fee_recipient,
state_root: header.state_root,
receipts_root: header.receipts_root,
logs_bloom: header.logs_bloom,
prev_randao: header.prev_randao,
block_number: header.block_number,
gas_limit: header.gas_limit,
gas_used: header.gas_used,
timestamp: header.timestamp,
extra_data: header.extra_data,
base_fee_per_gas: header.base_fee_per_gas,
block_hash: header.block_hash,
transactions: self.transactions,
withdrawals,
blob_gas_used: header.blob_gas_used,
excess_blob_gas: header.excess_blob_gas,
deposit_requests,
withdrawal_requests,
}))
} else {
Err(format!(
"block {} is post-electra but payload body doesn't have withdrawals/deposit_requests/withdrawal_requests \
withdrawals: {}, deposit_requests: {}, withdrawal_requests: {}",
header.block_hash, withdrawals_exist, deposit_requests_exist, withdrawal_requests_exist
))
}
let withdrawals = withdrawals.ok_or_else(|| {
format!(
"block {} is {} but payload body has withdrawals set to null",
header.block_hash, header_fork
)
})?;
let deposit_requests = deposit_requests.ok_or_else(|| {
format!(
"block {} is {} but payload body has deposit_requests set to null",
header.block_hash, header_fork
)
})?;
let withdrawal_requests = withdrawal_requests.ok_or_else(|| {
format!(
"block {} is {} but payload body has withdrawal_requests set to null",
header.block_hash, header_fork
)
})?;
let consolidation_requests = consolidation_requests.ok_or_else(|| {
format!(
"block {} is {} but payload body has consolidation_requests set to null",
header.block_hash, header_fork
)
})?;
Ok(ExecutionPayload::Electra(ExecutionPayloadElectra {
parent_hash: header.parent_hash,
fee_recipient: header.fee_recipient,
state_root: header.state_root,
receipts_root: header.receipts_root,
logs_bloom: header.logs_bloom,
prev_randao: header.prev_randao,
block_number: header.block_number,
gas_limit: header.gas_limit,
gas_used: header.gas_used,
timestamp: header.timestamp,
extra_data: header.extra_data,
base_fee_per_gas: header.base_fee_per_gas,
block_hash: header.block_hash,
transactions,
withdrawals,
blob_gas_used: header.blob_gas_used,
excess_blob_gas: header.excess_blob_gas,
deposit_requests,
withdrawal_requests,
consolidation_requests,
}))
}
}
}
@@ -686,6 +782,8 @@ pub struct EngineCapabilities {
pub forkchoice_updated_v3: bool,
pub get_payload_bodies_by_hash_v1: bool,
pub get_payload_bodies_by_range_v1: bool,
pub get_payload_bodies_by_hash_v2: bool,
pub get_payload_bodies_by_range_v2: bool,
pub get_payload_v1: bool,
pub get_payload_v2: bool,
pub get_payload_v3: bool,
@@ -723,6 +821,12 @@ impl EngineCapabilities {
if self.get_payload_bodies_by_range_v1 {
response.push(ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V1);
}
if self.get_payload_bodies_by_hash_v2 {
response.push(ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V2);
}
if self.get_payload_bodies_by_range_v2 {
response.push(ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V2);
}
if self.get_payload_v1 {
response.push(ENGINE_GET_PAYLOAD_V1);
}

View File

@@ -50,6 +50,8 @@ pub const ENGINE_FORKCHOICE_UPDATED_TIMEOUT: Duration = Duration::from_secs(8);
pub const ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V1: &str = "engine_getPayloadBodiesByHashV1";
pub const ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V1: &str = "engine_getPayloadBodiesByRangeV1";
pub const ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V2: &str = "engine_getPayloadBodiesByHashV2";
pub const ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V2: &str = "engine_getPayloadBodiesByRangeV2";
pub const ENGINE_GET_PAYLOAD_BODIES_TIMEOUT: Duration = Duration::from_secs(10);
pub const ENGINE_EXCHANGE_CAPABILITIES: &str = "engine_exchangeCapabilities";
@@ -78,6 +80,8 @@ pub static LIGHTHOUSE_CAPABILITIES: &[&str] = &[
ENGINE_FORKCHOICE_UPDATED_V3,
ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V1,
ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V1,
ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V2,
ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V2,
ENGINE_GET_CLIENT_VERSION_V1,
];
@@ -1035,7 +1039,7 @@ impl HttpJsonRpc {
pub async fn get_payload_bodies_by_hash_v1<E: EthSpec>(
&self,
block_hashes: Vec<ExecutionBlockHash>,
) -> Result<Vec<Option<ExecutionPayloadBodyV1<E>>>, Error> {
) -> Result<Vec<Option<ExecutionPayloadBody<E>>>, Error> {
let params = json!([block_hashes]);
let response: Vec<Option<JsonExecutionPayloadBodyV1<E>>> = self
@@ -1048,7 +1052,27 @@ impl HttpJsonRpc {
Ok(response
.into_iter()
.map(|opt_json| opt_json.map(From::from))
.map(|opt_json| opt_json.map(|v1| JsonExecutionPayloadBody::V1(v1).into()))
.collect())
}
pub async fn get_payload_bodies_by_hash_v2<E: EthSpec>(
&self,
block_hashes: Vec<ExecutionBlockHash>,
) -> Result<Vec<Option<ExecutionPayloadBody<E>>>, Error> {
let params = json!([block_hashes]);
let response: Vec<Option<JsonExecutionPayloadBodyV2<E>>> = self
.rpc_request(
ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V2,
params,
ENGINE_GET_PAYLOAD_BODIES_TIMEOUT * self.execution_timeout_multiplier,
)
.await?;
Ok(response
.into_iter()
.map(|opt_json| opt_json.map(|v2| JsonExecutionPayloadBody::V2(v2).into()))
.collect())
}
@@ -1056,7 +1080,7 @@ impl HttpJsonRpc {
&self,
start: u64,
count: u64,
) -> Result<Vec<Option<ExecutionPayloadBodyV1<E>>>, Error> {
) -> Result<Vec<Option<ExecutionPayloadBody<E>>>, Error> {
#[derive(Serialize)]
#[serde(transparent)]
struct Quantity(#[serde(with = "serde_utils::u64_hex_be")] u64);
@@ -1072,7 +1096,31 @@ impl HttpJsonRpc {
Ok(response
.into_iter()
.map(|opt_json| opt_json.map(From::from))
.map(|opt_json| opt_json.map(|v1| JsonExecutionPayloadBody::V1(v1).into()))
.collect())
}
pub async fn get_payload_bodies_by_range_v2<E: EthSpec>(
&self,
start: u64,
count: u64,
) -> Result<Vec<Option<ExecutionPayloadBody<E>>>, Error> {
#[derive(Serialize)]
#[serde(transparent)]
struct Quantity(#[serde(with = "serde_utils::u64_hex_be")] u64);
let params = json!([Quantity(start), Quantity(count)]);
let response: Vec<Option<JsonExecutionPayloadBodyV2<E>>> = self
.rpc_request(
ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V2,
params,
ENGINE_GET_PAYLOAD_BODIES_TIMEOUT * self.execution_timeout_multiplier,
)
.await?;
Ok(response
.into_iter()
.map(|opt_json| opt_json.map(|v2| JsonExecutionPayloadBody::V2(v2).into()))
.collect())
}
@@ -1099,6 +1147,10 @@ impl HttpJsonRpc {
.contains(ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V1),
get_payload_bodies_by_range_v1: capabilities
.contains(ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V1),
get_payload_bodies_by_hash_v2: capabilities
.contains(ENGINE_GET_PAYLOAD_BODIES_BY_HASH_V2),
get_payload_bodies_by_range_v2: capabilities
.contains(ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V2),
get_payload_v1: capabilities.contains(ENGINE_GET_PAYLOAD_V1),
get_payload_v2: capabilities.contains(ENGINE_GET_PAYLOAD_V2),
get_payload_v3: capabilities.contains(ENGINE_GET_PAYLOAD_V3),
@@ -1274,6 +1326,39 @@ impl HttpJsonRpc {
}
}
pub async fn get_payload_bodies_by_hash<E: EthSpec>(
&self,
block_hashes: Vec<ExecutionBlockHash>,
) -> Result<Vec<Option<ExecutionPayloadBody<E>>>, Error> {
let engine_capabilities = self.get_engine_capabilities(None).await?;
if engine_capabilities.get_payload_bodies_by_hash_v2 {
self.get_payload_bodies_by_hash_v2(block_hashes).await
} else if engine_capabilities.get_payload_bodies_by_hash_v1 {
self.get_payload_bodies_by_hash_v1(block_hashes).await
} else {
Err(Error::RequiredMethodUnsupported(
"engine_getPayloadBodiesByHash",
))
}
}
pub async fn get_payload_bodies_by_range<E: EthSpec>(
&self,
start: u64,
count: u64,
) -> Result<Vec<Option<ExecutionPayloadBody<E>>>, Error> {
let engine_capabilities = self.get_engine_capabilities(None).await?;
if engine_capabilities.get_payload_bodies_by_range_v2 {
self.get_payload_bodies_by_range_v2(start, count).await
} else if engine_capabilities.get_payload_bodies_by_range_v1 {
self.get_payload_bodies_by_range_v1(start, count).await
} else {
Err(Error::RequiredMethodUnsupported(
"engine_getPayloadBodiesByRange",
))
}
}
// automatically selects the latest version of
// forkchoice_updated that the execution engine supports
pub async fn forkchoice_updated(

View File

@@ -5,10 +5,7 @@ use strum::EnumString;
use superstruct::superstruct;
use types::beacon_block_body::KzgCommitments;
use types::blob_sidecar::BlobsList;
use types::{
DepositRequest, ExecutionLayerWithdrawalRequest, FixedVector, PublicKeyBytes, Signature,
Unsigned,
};
use types::{DepositRequest, FixedVector, PublicKeyBytes, Signature, Unsigned, WithdrawalRequest};
#[derive(Debug, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
@@ -106,11 +103,13 @@ pub struct JsonExecutionPayload<E: EthSpec> {
#[serde(with = "serde_utils::u64_hex_be")]
pub excess_blob_gas: u64,
#[superstruct(only(V4))]
// TODO(electra): Field name should be changed post devnet-0. See https://github.com/ethereum/execution-apis/pull/544
pub deposit_requests: VariableList<JsonDepositRequest, E::MaxDepositRequestsPerPayload>,
#[superstruct(only(V4))]
pub withdrawal_requests:
VariableList<JsonWithdrawalRequest, E::MaxWithdrawalRequestsPerPayload>,
#[superstruct(only(V4))]
pub consolidation_requests:
VariableList<JsonConsolidationRequest, E::MaxConsolidationRequestsPerPayload>,
}
impl<E: EthSpec> From<ExecutionPayloadBellatrix<E>> for JsonExecutionPayloadV1<E> {
@@ -225,6 +224,12 @@ impl<E: EthSpec> From<ExecutionPayloadElectra<E>> for JsonExecutionPayloadV4<E>
.map(Into::into)
.collect::<Vec<_>>()
.into(),
consolidation_requests: payload
.consolidation_requests
.into_iter()
.map(Into::into)
.collect::<Vec<_>>()
.into(),
}
}
}
@@ -353,6 +358,12 @@ impl<E: EthSpec> From<JsonExecutionPayloadV4<E>> for ExecutionPayloadElectra<E>
.map(Into::into)
.collect::<Vec<_>>()
.into(),
consolidation_requests: payload
.consolidation_requests
.into_iter()
.map(Into::into)
.collect::<Vec<_>>()
.into(),
}
}
}
@@ -737,45 +748,71 @@ impl From<ForkchoiceUpdatedResponse> for JsonForkchoiceUpdatedV1Response {
}
}
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(bound = "E: EthSpec")]
#[serde(rename_all = "camelCase")]
pub struct JsonExecutionPayloadBodyV1<E: EthSpec> {
#[superstruct(
variants(V1, V2),
variant_attributes(
derive(Clone, Debug, Serialize, Deserialize),
serde(bound = "E: EthSpec", rename_all = "camelCase"),
),
partial_getter_error(ty = "Error", expr = "Error::IncorrectStateVariant")
)]
#[derive(Clone, Debug, Serialize)]
#[serde(bound = "E: EthSpec", rename_all = "camelCase", untagged)]
pub struct JsonExecutionPayloadBody<E: EthSpec> {
#[serde(with = "ssz_types::serde_utils::list_of_hex_var_list")]
pub transactions: Transactions<E>,
pub withdrawals: Option<VariableList<JsonWithdrawal, E::MaxWithdrawalsPerPayload>>,
#[superstruct(only(V2))]
pub deposit_requests: Option<VariableList<JsonDepositRequest, E::MaxDepositRequestsPerPayload>>,
#[superstruct(only(V2))]
pub withdrawal_requests:
Option<VariableList<JsonWithdrawalRequest, E::MaxWithdrawalRequestsPerPayload>>,
#[superstruct(only(V2))]
pub consolidation_requests:
Option<VariableList<ConsolidationRequest, E::MaxConsolidationRequestsPerPayload>>,
}
impl<E: EthSpec> From<JsonExecutionPayloadBodyV1<E>> for ExecutionPayloadBodyV1<E> {
fn from(value: JsonExecutionPayloadBodyV1<E>) -> Self {
Self {
transactions: value.transactions,
withdrawals: value.withdrawals.map(|json_withdrawals| {
Withdrawals::<E>::from(
json_withdrawals
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
impl<E: EthSpec> From<JsonExecutionPayloadBody<E>> for ExecutionPayloadBody<E> {
fn from(value: JsonExecutionPayloadBody<E>) -> Self {
match value {
JsonExecutionPayloadBody::V1(body_v1) => Self::V1(ExecutionPayloadBodyV1 {
transactions: body_v1.transactions,
withdrawals: body_v1.withdrawals.map(|json_withdrawals| {
Withdrawals::<E>::from(
json_withdrawals
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
}),
}),
deposit_requests: value.deposit_requests.map(|json_receipts| {
DepositRequests::<E>::from(
json_receipts
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
}),
withdrawal_requests: value.withdrawal_requests.map(|json_withdrawal_requests| {
WithdrawalRequests::<E>::from(
json_withdrawal_requests
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
JsonExecutionPayloadBody::V2(body_v2) => Self::V2(ExecutionPayloadBodyV2 {
transactions: body_v2.transactions,
withdrawals: body_v2.withdrawals.map(|json_withdrawals| {
Withdrawals::<E>::from(
json_withdrawals
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
}),
deposit_requests: body_v2.deposit_requests.map(|json_receipts| {
DepositRequests::<E>::from(
json_receipts
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
}),
withdrawal_requests: body_v2.withdrawal_requests.map(|json_withdrawal_requests| {
WithdrawalRequests::<E>::from(
json_withdrawal_requests
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
}),
consolidation_requests: body_v2.consolidation_requests,
}),
}
}
@@ -896,27 +933,55 @@ impl From<JsonDepositRequest> for DepositRequest {
#[serde(rename_all = "camelCase")]
pub struct JsonWithdrawalRequest {
pub source_address: Address,
pub validator_public_key: PublicKeyBytes,
pub validator_pubkey: PublicKeyBytes,
#[serde(with = "serde_utils::u64_hex_be")]
pub amount: u64,
}
impl From<ExecutionLayerWithdrawalRequest> for JsonWithdrawalRequest {
fn from(withdrawal_request: ExecutionLayerWithdrawalRequest) -> Self {
impl From<WithdrawalRequest> for JsonWithdrawalRequest {
fn from(withdrawal_request: WithdrawalRequest) -> Self {
Self {
source_address: withdrawal_request.source_address,
validator_public_key: withdrawal_request.validator_pubkey,
validator_pubkey: withdrawal_request.validator_pubkey,
amount: withdrawal_request.amount,
}
}
}
impl From<JsonWithdrawalRequest> for ExecutionLayerWithdrawalRequest {
impl From<JsonWithdrawalRequest> for WithdrawalRequest {
fn from(json_withdrawal_request: JsonWithdrawalRequest) -> Self {
Self {
source_address: json_withdrawal_request.source_address,
validator_pubkey: json_withdrawal_request.validator_public_key,
validator_pubkey: json_withdrawal_request.validator_pubkey,
amount: json_withdrawal_request.amount,
}
}
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(rename_all = "camelCase")]
pub struct JsonConsolidationRequest {
pub source_address: Address,
pub source_pubkey: PublicKeyBytes,
pub target_pubkey: PublicKeyBytes,
}
impl From<ConsolidationRequest> for JsonConsolidationRequest {
fn from(consolidation_request: ConsolidationRequest) -> Self {
Self {
source_address: consolidation_request.source_address,
source_pubkey: consolidation_request.source_pubkey,
target_pubkey: consolidation_request.target_pubkey,
}
}
}
impl From<JsonConsolidationRequest> for ConsolidationRequest {
fn from(json_consolidation_request: JsonConsolidationRequest) -> Self {
Self {
source_address: json_consolidation_request.source_address,
source_pubkey: json_consolidation_request.source_pubkey,
target_pubkey: json_consolidation_request.target_pubkey,
}
}
}

View File

@@ -1771,10 +1771,10 @@ impl<E: EthSpec> ExecutionLayer<E> {
pub async fn get_payload_bodies_by_hash(
&self,
hashes: Vec<ExecutionBlockHash>,
) -> Result<Vec<Option<ExecutionPayloadBodyV1<E>>>, Error> {
) -> Result<Vec<Option<ExecutionPayloadBody<E>>>, Error> {
self.engine()
.request(|engine: &Engine| async move {
engine.api.get_payload_bodies_by_hash_v1(hashes).await
engine.api.get_payload_bodies_by_hash(hashes).await
})
.await
.map_err(Box::new)
@@ -1785,14 +1785,11 @@ impl<E: EthSpec> ExecutionLayer<E> {
&self,
start: u64,
count: u64,
) -> Result<Vec<Option<ExecutionPayloadBodyV1<E>>>, Error> {
) -> Result<Vec<Option<ExecutionPayloadBody<E>>>, Error> {
let _timer = metrics::start_timer(&metrics::EXECUTION_LAYER_GET_PAYLOAD_BODIES_BY_RANGE);
self.engine()
.request(|engine: &Engine| async move {
engine
.api
.get_payload_bodies_by_range_v1(start, count)
.await
engine.api.get_payload_bodies_by_range(start, count).await
})
.await
.map_err(Box::new)
@@ -2010,6 +2007,15 @@ impl<E: EthSpec> ExecutionLayer<E> {
.collect(),
)
.map_err(ApiError::DeserializeWithdrawalRequests)?;
let n_consolidations = electra_block.consolidation_requests.len();
let consolidation_requests = VariableList::new(
electra_block
.consolidation_requests
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
.map_err(|_| ApiError::TooManyConsolidationRequests(n_consolidations))?;
ExecutionPayload::Electra(ExecutionPayloadElectra {
parent_hash: electra_block.parent_hash,
fee_recipient: electra_block.fee_recipient,
@@ -2030,6 +2036,7 @@ impl<E: EthSpec> ExecutionLayer<E> {
excess_blob_gas: electra_block.excess_blob_gas,
deposit_requests,
withdrawal_requests,
consolidation_requests,
})
}
};

View File

@@ -659,8 +659,10 @@ impl<E: EthSpec> ExecutionBlockGenerator<E> {
withdrawals: pa.withdrawals.clone().into(),
blob_gas_used: 0,
excess_blob_gas: 0,
// TODO(electra): consider how to test these fields below
deposit_requests: vec![].into(),
withdrawal_requests: vec![].into(),
consolidation_requests: vec![].into(),
}),
_ => unreachable!(),
},

View File

@@ -589,6 +589,65 @@ pub async fn handle_rpc<E: EthSpec>(
.withdrawals()
.ok()
.map(|withdrawals| VariableList::from(withdrawals.clone())),
}));
}
None => response.push(None),
}
}
Ok(serde_json::to_value(response).unwrap())
}
ENGINE_GET_PAYLOAD_BODIES_BY_RANGE_V2 => {
#[derive(Deserialize)]
#[serde(transparent)]
struct Quantity(#[serde(with = "serde_utils::u64_hex_be")] pub u64);
let start = get_param::<Quantity>(params, 0)
.map_err(|s| (s, BAD_PARAMS_ERROR_CODE))?
.0;
let count = get_param::<Quantity>(params, 1)
.map_err(|s| (s, BAD_PARAMS_ERROR_CODE))?
.0;
let mut response = vec![];
for block_num in start..(start + count) {
let maybe_block = ctx
.execution_block_generator
.read()
.execution_block_with_txs_by_number(block_num);
match maybe_block {
Some(block) => {
let transactions = Transactions::<E>::new(
block
.transactions()
.iter()
.map(|transaction| VariableList::new(transaction.rlp().to_vec()))
.collect::<Result<_, _>>()
.map_err(|e| {
(
format!("failed to deserialize transaction: {:?}", e),
GENERIC_ERROR_CODE,
)
})?,
)
.map_err(|e| {
(
format!("failed to deserialize transactions: {:?}", e),
GENERIC_ERROR_CODE,
)
})?;
// TODO(electra): add testing for:
// deposit_requests
// withdrawal_requests
// consolidation_requests
response.push(Some(JsonExecutionPayloadBodyV2::<E> {
transactions,
withdrawals: block
.withdrawals()
.ok()
.map(|withdrawals| VariableList::from(withdrawals.clone())),
deposit_requests: block.deposit_requests().ok().map(
|deposit_requests| VariableList::from(deposit_requests.clone()),
),
@@ -597,6 +656,17 @@ pub async fn handle_rpc<E: EthSpec>(
VariableList::from(withdrawal_requests.clone())
},
),
consolidation_requests: block.consolidation_requests().ok().map(
|consolidation_requests| {
VariableList::from(
consolidation_requests
.clone()
.into_iter()
.map(Into::into)
.collect::<Vec<_>>(),
)
},
),
}));
}
None => response.push(None),

View File

@@ -47,7 +47,9 @@ pub const DEFAULT_ENGINE_CAPABILITIES: EngineCapabilities = EngineCapabilities {
forkchoice_updated_v2: true,
forkchoice_updated_v3: true,
get_payload_bodies_by_hash_v1: true,
get_payload_bodies_by_hash_v2: true,
get_payload_bodies_by_range_v1: true,
get_payload_bodies_by_range_v2: true,
get_payload_v1: true,
get_payload_v2: true,
get_payload_v3: true,