Commit Graph

452 Commits

Author SHA1 Message Date
realbigsean
13ffdd211d Beacon api + validator electra (#5744)
* Attestation superstruct changes for EIP 7549 (#5644)

* update

* experiment

* superstruct changes

* revert

* superstruct changes

* fix tests

* indexed attestation

* indexed attestation superstruct

* updated TODOs

* `superstruct` the `AttesterSlashing` (#5636)

* `superstruct` Attester Fork Variants

* Push a little further

* Deal with Encode / Decode of AttesterSlashing

* not so sure about this..

* Stop Encode/Decode Bounds from Propagating Out

* Tons of Changes..

* More Conversions to AttestationRef

* Add AsReference trait (#15)

* Add AsReference trait

* Fix some snafus

* Got it Compiling! :D

* Got Tests Building

* Get beacon chain tests compiling

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes

* Make EF Tests Fork-Agnostic (#5713)

* Finish EF Test Fork Agnostic (#5714)

* Superstruct `AggregateAndProof` (#5715)

* Upgrade `superstruct` to `0.8.0`

* superstruct `AggregateAndProof`

* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes

* cargo fmt

* Merge pull request #5726 from realbigsean/electra_attestation_changes

Merge unstable into Electra attestation changes

* process withdrawals updates

* cleanup withdrawals processing

* update `process_operations` deposit length check

* add apply_deposit changes

* add execution layer withdrawal request processing

* process deposit receipts

* add consolidation processing

* update process operations function

* exit updates

* clean up

* update slash_validator

* EIP7549 `get_attestation_indices` (#5657)

* get attesting indices electra impl

* fmt

* get tests to pass

* fmt

* fix some beacon chain tests

* fmt

* fix slasher test

* fmt got me again

* fix more tests

* fix tests

* Some small changes (#5739)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* cargo fmt (#5740)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* fix attestation verification

* Add new engine api methods

* Fix the versioning of v4 requests

* Handle new engine api methods in mock EL

* Note todo

* Fix todos

* Add support for electra fields in getPayloadBodies

* Add comments for potential versioning confusion

* udpates for aggregate attestation endpoint

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Sketch op pool changes

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

* Get `electra_op_pool` up to date (#5756)

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>

* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)

This reverts commit ab9e58aa3d.

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Compute on chain aggregate impl (#5752)

* add compute_on_chain_agg impl to op pool changes

* fmt

* get op pool tests to pass

* update beacon api aggregate attestationendpoint

* update the naive agg pool interface (#5760)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* updates after merge

* Fix bugs in cross-committee aggregation

* Add comment to max cover optimisation

* Fix assert

* Electra epoch processing

* add deposit limit for old deposit queue

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Merge pull request #5749 from sigp/electra_op_pool

Optimise Electra op pool aggregation

* don't fail on empty consolidations

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* update committee offset

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* update committee offset

* update committee offset

* update committee offset

* only increment the state deposit index on old deposit flow

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* use correct max eb in epoch cache initialization

* drop initiate validator ordering optimization

* fix initiate exit for single pass

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* accept new payload v4 in mock el

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Electra Fork Choice Tests (#5764)

* Fix Electra Fork Choice Tests (#5764)

* Fix Electra Fork Choice Tests (#5764)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Consolidation Sigs & Withdrawals

* Merge pull request #5766 from ethDreamer/two_fixes

Fix Consolidation Sigs & Withdrawals

* Merge branches 'block-processing-electra' and 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Send unagg attestation based on fork

* Fix ser/de

* Merge branch 'electra-engine-api' into beacon-api-electra

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* update electra readiness with new endpoints

* fix slashing handling

* Fix Bug In Block Processing with 0x02 Credentials

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request #5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* fix: serde rename camle case for execution payload body (#5846)

* Merge branch 'electra-engine-api' into beacon-api-electra

* Electra attestation changes rm decode impl (#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Remove Duplicate KZG Commitment Merkle Proof Code (#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* fix compile

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix slasher tests (#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations #5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* Fix Compilation Break

* Merge pull request #5973 from ethDreamer/beacon-api-electra

Fix Compilation Break

* Electra attestation changes sean review (#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Update per_block_processing.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request #5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request #5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra

* Merge branch 'block-processing-electra' of https://github.com/sigp/lighthouse into electra-epoch-proc

* Merge branch 'electra-epoch-proc' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* The great renaming receipt -> request

* Address some more review comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Update beacon_node/beacon_chain/src/electra_readiness.rs

* Update consensus/types/src/chain_spec.rs

* update GET requests

* update POST requests

* add client updates and test updates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-engine-api

* Merge branch 'electra-engine-api' of https://github.com/sigp/lighthouse into beacon-api-electra

* compile after merge

* unwrap -> unwrap_err

* self review

* fix tests

* convert op pool messages to electra in electra

* remove methods to post without content header

* filter instead of convert
2024-07-16 08:58:16 +10:00
Michael Sproul
2f0af2be89 Unimplement TreeHash for BeaconState (#6083)
* Unimplement `TreeHash` for `BeaconState`
2024-07-12 13:06:08 +00:00
Eitan Seri-Levi
69d84e785b Reject octet-stream content type when we expect json (#5862)
* reject octet-stream content type when we expect json

* added a test, fixed a bug with checking for 415's
2024-07-12 06:13:52 +00:00
realbigsean
858b01f4e3 Block processing electra (#5741)
* Attestation superstruct changes for EIP 7549 (#5644)

* update

* experiment

* superstruct changes

* revert

* superstruct changes

* fix tests

* indexed attestation

* indexed attestation superstruct

* updated TODOs

* `superstruct` the `AttesterSlashing` (#5636)

* `superstruct` Attester Fork Variants

* Push a little further

* Deal with Encode / Decode of AttesterSlashing

* not so sure about this..

* Stop Encode/Decode Bounds from Propagating Out

* Tons of Changes..

* More Conversions to AttestationRef

* Add AsReference trait (#15)

* Add AsReference trait

* Fix some snafus

* Got it Compiling! :D

* Got Tests Building

* Get beacon chain tests compiling

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes

* Make EF Tests Fork-Agnostic (#5713)

* Finish EF Test Fork Agnostic (#5714)

* Superstruct `AggregateAndProof` (#5715)

* Upgrade `superstruct` to `0.8.0`

* superstruct `AggregateAndProof`

* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes

* cargo fmt

* Merge pull request #5726 from realbigsean/electra_attestation_changes

Merge unstable into Electra attestation changes

* process withdrawals updates

* cleanup withdrawals processing

* update `process_operations` deposit length check

* add apply_deposit changes

* add execution layer withdrawal request processing

* process deposit receipts

* add consolidation processing

* update process operations function

* exit updates

* clean up

* update slash_validator

* EIP7549 `get_attestation_indices` (#5657)

* get attesting indices electra impl

* fmt

* get tests to pass

* fmt

* fix some beacon chain tests

* fmt

* fix slasher test

* fmt got me again

* fix more tests

* fix tests

* Some small changes (#5739)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* cargo fmt (#5740)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* fix attestation verification

* Sketch op pool changes

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

* Get `electra_op_pool` up to date (#5756)

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>

* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)

This reverts commit ab9e58aa3d.

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Compute on chain aggregate impl (#5752)

* add compute_on_chain_agg impl to op pool changes

* fmt

* get op pool tests to pass

* update the naive agg pool interface (#5760)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Fix bugs in cross-committee aggregation

* Add comment to max cover optimisation

* Fix assert

* Merge pull request #5749 from sigp/electra_op_pool

Optimise Electra op pool aggregation

* don't fail on empty consolidations

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* update committee offset

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* only increment the state deposit index on old deposit flow

* Fix Electra Fork Choice Tests (#5764)

* Fix Electra Fork Choice Tests (#5764)

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Fix Consolidation Sigs & Withdrawals

* Merge pull request #5766 from ethDreamer/two_fixes

Fix Consolidation Sigs & Withdrawals

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* fix slashing handling

* Fix Bug In Block Processing with 0x02 Credentials

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request #5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Electra attestation changes rm decode impl (#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Remove Duplicate KZG Commitment Merkle Proof Code (#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* fix compile

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Fix slasher tests (#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations #5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into block-processing-electra

* Electra attestation changes sean review (#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Update per_block_processing.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request #5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request #5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Merge branch 'electra_attestation_changes' of https://github.com/realbigsean/lighthouse into block-processing-electra

* fork enabled electra
2024-06-26 13:29:27 +00:00
ethDreamer
c52c598f69 Electra: Remaining Consensus Data Structures (#5712)
* Attestation superstruct changes for EIP 7549 (#5644)

* update

* experiment

* superstruct changes

* revert

* superstruct changes

* fix tests

* indexed attestation

* indexed attestation superstruct

* updated TODOs

* `superstruct` the `AttesterSlashing` (#5636)

* `superstruct` Attester Fork Variants

* Push a little further

* Deal with Encode / Decode of AttesterSlashing

* not so sure about this..

* Stop Encode/Decode Bounds from Propagating Out

* Tons of Changes..

* More Conversions to AttestationRef

* Add AsReference trait (#15)

* Add AsReference trait

* Fix some snafus

* Got it Compiling! :D

* Got Tests Building

* Get beacon chain tests compiling

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes

* Make EF Tests Fork-Agnostic (#5713)

* Finish EF Test Fork Agnostic (#5714)

* Superstruct `AggregateAndProof` (#5715)

* Upgrade `superstruct` to `0.8.0`

* superstruct `AggregateAndProof`

* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes

* cargo fmt

* Merge pull request #5726 from realbigsean/electra_attestation_changes

Merge unstable into Electra attestation changes

* EIP7549 `get_attestation_indices` (#5657)

* get attesting indices electra impl

* fmt

* get tests to pass

* fmt

* fix some beacon chain tests

* fmt

* fix slasher test

* fmt got me again

* fix more tests

* fix tests

* Some small changes (#5739)

* cargo fmt (#5740)

* Sketch op pool changes

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

* Get `electra_op_pool` up to date (#5756)

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>

* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)

This reverts commit ab9e58aa3d.

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool

* Compute on chain aggregate impl (#5752)

* add compute_on_chain_agg impl to op pool changes

* fmt

* get op pool tests to pass

* update the naive agg pool interface (#5760)

* Fix bugs in cross-committee aggregation

* Add comment to max cover optimisation

* Fix assert

* Merge pull request #5749 from sigp/electra_op_pool

Optimise Electra op pool aggregation

* update committee offset

* Fix Electra Fork Choice Tests (#5764)

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* fix slashing handling

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request #5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Electra attestation changes rm decode impl (#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Remove Duplicate KZG Commitment Merkle Proof Code (#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* fix compile

* Fix slasher tests (#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations #5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Electra attestation changes sean review (#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request #5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request #5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs
2024-06-24 21:08:07 +00:00
realbigsean
a74098044a Rust 1.79 lints (#5927)
* max_value -> MAX

* remove unnecesary closures

* a couple more max_value -> MAX

* a couple more max_value -> MAX

* Revert "a couple more max_value -> MAX"

This reverts commit 807fe7cae9.

* unused spec field -> phantom data

* ignore some dead code warnings

* update kurtosis repo location
2024-06-13 23:04:30 +00:00
Lion - dapplion
2c971fa9a1 Make beacon processor queue sizes dynamic (#5573)
* Make beacon processor queue sizes dynamic

* Update tests

* lint

* Review PR
2024-06-03 07:17:11 +00:00
chonghe
ffe29c087d Add bls_to_execution_change topic to eth1/v1/events (#5823)
* Add bls event

* Update events and types

* Add bls in event

* Event bls

* tests..rs

* change order

* another tests.rs

* Signed BLS

* Revert "another tests.rs"

This reverts commit 7f54e9c1ce.

* Revert "Signed BLS"

This reverts commit 1146bc734b.

* withdrawal_keyparis

* Fix genesis
2024-05-30 05:48:29 +00:00
chonghe
b5de925d8f Use JSON header by default for /eth/v1/beacon/deposit_snapshot (#5813)
* Fix with or

* Flip case
2024-05-20 02:17:43 +00:00
Eitan Seri-Levi
6636167503 Log block import source (#5738)
* the default target peers is 100

* add some comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into track-block-import-source

* add block import source

* revert

* update logging text

* fix tests

* lint

* use % instaed of to_string
2024-05-15 09:17:06 +00:00
Michael Sproul
d3d429ff5c Improve logging and metrics for block publication (#5699)
* Improve logging and metrics for block publication

* Add better buckets

* Bump SQL connection timeout for tests.
2024-05-03 08:24:49 +00:00
Eitan Seri-Levi
d3bf9a8956 Proposer and attester slashing sse events (#5327)
* default vc to block v3 endpoint and deprecate block-v3 flag

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into unstable

* add proposer and attester event variants

* add TOOOs

* add tests, event triggers

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into proposer-and-attester-slashing-sse-events

* revert

* revert

* remove double event tracking

* Merge branch 'unstable' into proposer-and-attester-slashing-sse-events

* remove todo, fix test

* resolve merge conflicts

* Merge branch 'proposer-and-attester-slashing-sse-events' of https://github.com/eserilev/lighthouse into proposer-and-attester-slashing-sse-events

* leftover debugging

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into proposer-and-attester-slashing-sse-events

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into proposer-and-attester-slashing-sse-events
2024-04-30 08:56:45 +00:00
Mac L
13f94ef0f3 Rename Merge to Bellatrix (#5601)
* Rename Merge to Bellatrix

* Remove tree-hash-cache which got readded from the rebase
2024-04-25 20:19:41 +00:00
Michael Sproul
61962898e2 In-memory tree states (#5533)
* Consensus changes

* EF tests

* lcli

* common and watch

* account manager

* cargo

* fork choice

* promise cache

* beacon chain

* interop genesis

* http api

* lighthouse

* op pool

* beacon chain misc

* parallel state cache

* store

* fix issues in store

* IT COMPILES

* Remove some unnecessary module qualification

* Revert Arced pubkey optimization (#5536)

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Fix caching, rebasing and some tests

* Remove unused deps

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Small cleanups

* Revert shuffling cache/promise cache changes

* Fix state advance bugs

* Fix shuffling tests

* Remove some resolved FIXMEs

* Remove StateProcessingStrategy

* Optimise withdrawals calculation

* Don't reorg if state cache is missed

* Remove inconsistent state func

* Fix beta compiler

* Rebase early, rebase often

* Fix state caching behaviour

* Update to milhouse release

* Fix on-disk consensus context format

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Squashed commit of the following:

commit 3a16649023
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Thu Apr 18 14:26:09 2024 +1000

    Fix on-disk consensus context format

* Keep indexed attestations, thanks Sean

* Merge branch 'on-disk-consensus-context' into tree-states-memory

* Merge branch 'unstable' into tree-states-memory

* Address half of Sean's review

* More simplifications from Sean's review

* Cache state after get_advanced_hot_state
2024-04-24 01:22:36 +00:00
Eitan Seri-Levi
6bac5ce12b Deprecate http-spec-fork and http-allow-sync-stalled (#5500)
* deprecate flags

* fmt

* remove backslash

* remove hidden flags from the book

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into deprecate-http-spec-fork-and-http-allow-sync-stalled

* add warn, re-add tests

* Apply suggestions from code review

* Merge remote-tracking branch 'origin/unstable' into deprecate-http-spec-fork-and-http-allow-sync-stalled

* Fix imports
2024-04-12 04:21:00 +00:00
pahor167
72af6fb83a Skip calculation of boosted_relay_value when builder_boost_factor pro… (#5352)
* Skip calculation of boosted_relay_value when builder_boost_factor provided

* PR comments
2024-04-10 12:05:13 +00:00
0xalex88
1b88d29807 Use hashset to filter validators ids in http_api (#5468)
* Use hashset to filter validators ids in http_api

* Update beacon_node/http_api/src/validators.rs
2024-04-09 03:49:03 +00:00
Eitan Seri-Levi
ee69e14db9 Add is_parent_strong proposer re-org check (#5417)
* initial fork choice additions

* add helper fns

* add is_parent_strong

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into add_is_parent_strong_check

* disabling proposer reorg should set parent_threshold to u64 max

* add new flag, is_parent_strong check in override fcu params

* cherry-pick changes

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into add_is_parent_strong_check

* cleanup

* fmt

* Minor review tweaks
2024-04-04 19:38:06 +00:00
Michael Sproul
feb531f85b Single-pass epoch processing and optimised block processing (#5279)
* Single-pass epoch processing (#4483, #4573)

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Delete unused epoch processing code (#5170)

* Delete unused epoch processing code

* Compare total deltas

* Remove unnecessary apply_pending

* cargo fmt

* Remove newline

* Use epoch cache in block packing (#5223)

* Remove progressive balances mode (#5224)

* inline inactivity_penalty_quotient_for_state

* drop previous_epoch_total_active_balance

* fc lint

* spec compliant process_sync_aggregate (#15)

* spec compliant process_sync_aggregate

* Update consensus/state_processing/src/per_block_processing/altair/sync_committee.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Delete the participation cache (#16)

* update help

* Fix op_pool tests

* Fix fork choice tests

* Merge remote-tracking branch 'sigp/unstable' into epoch-single-pass

* Simplify exit cache (#5280)

* Fix clippy on exit cache

* Clean up single-pass a bit (#5282)

* Address Mark's review of single-pass (#5386)

* Merge remote-tracking branch 'origin/unstable' into epoch-single-pass

* Address Sean's review comments (#5414)

* Address most of Sean's review comments

* Simplify total balance cache building

* Clean up unused junk

* Merge remote-tracking branch 'origin/unstable' into epoch-single-pass

* More self-review

* Merge remote-tracking branch 'origin/unstable' into epoch-single-pass

* Merge branch 'unstable' into epoch-single-pass

* Fix imports for beta compiler

* Fix tests, probably
2024-04-04 13:14:36 +00:00
Eitan Seri-Levi
f4cdcea7b1 Return not synced errors for endpoints that require syncing (#5136)
* add not synced filter into then blocks

* refactor
2024-04-04 01:36:23 +00:00
Mac L
969d12dc6f Use E for EthSpec globally (#5264)
* Use `E` for `EthSpec` globally

* Fix tests

* Merge branch 'unstable' into e-ethspec

* Merge branch 'unstable' into e-ethspec

# Conflicts:
#	beacon_node/execution_layer/src/engine_api.rs
#	beacon_node/execution_layer/src/engine_api/http.rs
#	beacon_node/execution_layer/src/engine_api/json_structures.rs
#	beacon_node/execution_layer/src/test_utils/handle_rpc.rs
#	beacon_node/store/src/partial_beacon_state.rs
#	consensus/types/src/beacon_block.rs
#	consensus/types/src/beacon_block_body.rs
#	consensus/types/src/beacon_state.rs
#	consensus/types/src/config_and_preset.rs
#	consensus/types/src/execution_payload.rs
#	consensus/types/src/execution_payload_header.rs
#	consensus/types/src/light_client_optimistic_update.rs
#	consensus/types/src/payload.rs
#	lcli/src/parse_ssz.rs
2024-04-02 15:12:25 +00:00
Mac L
f8fdb71f50 Add Electra fork boilerplate (#5122)
* Add Electra fork boilerplate

* Remove electra from spec tests

* Fix tests

* Remove sneaky log file

* Fix more tests

* Fix even more tests and add suggestions

* Remove unrelated lcli addition

* Update more tests

* Merge branch 'unstable' into electra

* Add comment for test-suite lcli override

* Merge branch 'unstable' into electra

* Cleanup

* Merge branch 'unstable' into electra

* Apply suggestions

* Merge branch 'unstable' into electra

* Merge sigp/unstable into electra

* Merge branch 'unstable' into electra
2024-04-02 12:35:02 +00:00
Eitan Seri-Levi
e4d4e439cb Add Capella & Deneb light client support (#4946)
* rebase and add comment

* conditional test

* test

* optimistic chould be working now

* finality should be working now

* try again

* try again

* clippy fix

* add lc bootstrap beacon api

* add lc optimistic/finality update to events

* fmt

* That error isn't occuring on my computer but I think this should fix it

* Merge branch 'unstable' into light_client_beacon_api_1

# Conflicts:
#	beacon_node/beacon_chain/src/events.rs
#	beacon_node/http_api/src/lib.rs
#	beacon_node/http_api/src/test_utils.rs
#	beacon_node/http_api/tests/main.rs
#	beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
#	beacon_node/lighthouse_network/src/rpc/methods.rs
#	beacon_node/lighthouse_network/src/service/api_types.rs
#	beacon_node/network/src/beacon_processor/worker/rpc_methods.rs
#	beacon_node/tests/test.rs
#	common/eth2/src/types.rs
#	lighthouse/src/main.rs

* Add missing test file

* Update light client types to comply with Altair light client spec.

* Fix test compilation

* Merge branch 'unstable' into light_client_beacon_api_1

* Support deserializing light client structures for the Bellatrix fork

* Move `get_light_client_bootstrap` logic to `BeaconChain`. `LightClientBootstrap` API to return `ForkVersionedResponse`.

* Misc fixes.
- log cleanup
- move http_api config mutation to `config::get_config` for consistency
- fix light client API responses

* Add light client bootstrap API test and fix existing ones.

* Merge branch 'unstable' into light_client_beacon_api_1

* Fix test for `light-client-server` http api config.

* Appease clippy

* Add Altair light client SSZ tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into light_client_beacon_api_1

* updates to light client header

* light client header from signed beacon block

* using options

* implement helper functions

* placeholder conversion from vec hash256 to exec branch

* add deneb

* using fixed vector

* remove unwraps

* by epoch

* compute merkle proof

* merkle proof

* update comments

* resolve merge conflicts

* linting

* Merge branch 'unstable' into light-client-ssz-tests

# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	consensus/types/src/light_client_bootstrap.rs
#	consensus/types/src/light_client_header.rs

* superstruct attempt

* superstruct changes

* lint

* altair

* update

* update

* changes to light_client_optimistic_ and finality

* merge unstable

* refactor

* resolved merge conflicts

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* block_to_light_client_header fork aware

* fmt

* comment fix

* comment fix

* include merge fork, update deserialize_by_fork, refactor

* fmt

* pass by ref to prevent clone

* rename merkle proof fn

* add FIXME

* LightClientHeader TestRandom

* fix comments

* fork version deserialize

* merge unstable

* move fn arguments, fork name calc

* use task executor

* remove unneeded fns

* remove dead code

* add manual ssz decoding/encoding and add ssz_tests_by_fork macro

* merge deneb types with tests

* merge ssz tests, revert code deletion, cleanup

* move chainspec

* update ssz tests

* fmt

* light client ssz tests

* change to superstruct

* changes from feedback

* linting

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* test fix

* cleanup

* Remove unused `derive`.

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* beta compiler fix

* merge
2024-03-25 11:01:56 +00:00
dknopik
0a6e4a11d7 Verify whether validators really are unknown during sync committee duty API request (#5174)
* Verify whether validators really are unknown during sync committee duty API request

* Merge branch 'unstable' into fix-4717

* Merge branch 'unstable' into fix-4717

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into fix-4717
2024-03-23 00:01:11 +00:00
zhiqiangxu
5121d655f7 chore: reduce scope of commitment (#5426)
* reduce scope of commitment

* avoid clone for last reference

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into mod_merge_single_blob
2024-03-22 18:16:52 +00:00
realbigsean
21cdc64bfe Improve parent lookup logging (#5451)
* upgrade parent lookup result processing logs to debug, use display instead of debug for BlockError in case a blob parent unknown error is hit, add block root to BlockIsAlreadyKnown

* fix compile

* fix compile

* fix compile
2024-03-22 18:16:13 +00:00
Eitan Seri-Levi
01ec42e75a Fix Rust beta compiler errors 1.78.0-beta.1 (#5439)
* remove redundant imports

* fix test

* contains key

* fmt

* Merge branch 'unstable' into fix-beta-compiler
2024-03-20 05:17:02 +00:00
Pawan Dhananjay
84a902a589 Reduce load on validator subscription channels (#5311)
* Fix tests

* Merge branch 'unstable' into unclog-channels

* Avoid reallocations

* Reduce subscription load on beacon node
2024-03-07 12:32:27 +00:00
Michael Sproul
bf118a17d4 Fix block v3 header decoding (#5366)
* Fix block v3 header decoding
2024-03-07 03:31:06 +00:00
João Oliveira
a89ff100af improve libp2p connected peer metrics (#5314)
* patch rust-yamux dep

* improve libp2p connected peer metrics
2024-02-28 03:52:55 +00:00
Jimmy Chen
50c423ad88 Revert libp2p metrics (#4870) (#5265)
* Revert "improve libp2p connected peer metrics (#4870)"

This reverts commit 0c3fef59b3.
2024-02-20 04:19:17 +00:00
Michael Sproul
f17fb291b7 Handle unknown head during attestation publishing (#5010)
* Handle unknown head during attestation publishing

* Merge remote-tracking branch 'origin/unstable' into queue-http-attestations

* Simplify task spawner

* Improve logging

* Add a test

* Improve error logging

* Merge remote-tracking branch 'origin/unstable' into queue-http-attestations

* Fix beta compiler warnings
2024-02-15 12:24:47 +00:00
Eitan Seri-Levi
e7ef2a3a54 validator liveness endpoint should accept string encoded indices (#5184)
* deserialize string indices as u64

* client should send quoted indices
2024-02-09 04:59:39 +00:00
João Oliveira
0c3fef59b3 improve libp2p connected peer metrics (#4870)
* improve libp2p connected peer metrics

* separate discv5 port from libp2p for NAT open

* use metric family for DISCOVERY_BYTES

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into improve-metrics
2024-02-08 02:40:54 +00:00
Jimmy Chen
39e9f7dc6b Fix Rust beta compiler errors (1.77) (#5180)
* Lint fixes

* More fixes for beta compiler.

* Format fixes

* Move `#[allow(dead_code)]` to field level.

* Remove old comment.

* Update beacon_node/execution_layer/src/test_utils/mod.rs

Co-authored-by: João Oliveira <hello@jxs.pt>

* remove duplicate line
2024-02-05 17:54:11 +00:00
Lion - dapplion
b035638f9b Compute recent lightclient updates (#4969)
* Compute recent lightclient updates

* Review PR

* Merge remote-tracking branch 'upstream/unstable' into lc-prod-recent-updates

* Review PR

* consistent naming

* add metrics

* revert dropping reprocessing queue

* Update light client optimistic update re-processing logic. (#7)

* Add light client server simulator tests. Co-authored by @dapplion.

* Merge branch 'unstable' into fork/dapplion/lc-prod-recent-updates

* Fix lint

* Enable light client server in simulator test.

* Fix test for light client optimistic updates and finality updates.
2024-01-31 05:25:51 +00:00
Eitan Seri-Levi
1d87edb03d Assume Content-Type is json for endpoints that require json (#4575)
* added default content type filter

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into unstable

* create custom warp json filter that ignores content type header

* cargo fmt and linting

* updated test

* updated test

* merge unstable

* merge conflicts

* workspace=true

* use Bytes instead of Buf

* resolve merge conflict

* resolve merge conflicts

* add extra error message context

* merge conflicts

* lint
2024-01-31 01:11:54 +00:00
Jack McPherson
abeb358f0b Remove custom SSZ beacon states route (#5065)
* Remove SSZ state root route

* Remove SSZ states route from client impl

* Patch tests

* Merge branch 'unstable' into 5063-delete-ssz-state-route

* Further remove dead code
2024-01-30 00:33:05 +00:00
Pawan Dhananjay
b55b58b3c6 Fix indices filter in blobs_sidecar http endpoint (#5118)
* add get blobs unit test

* Use a multi_key_query for blob_sidecar indices

* Fix test

* Remove env_logger

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2024-01-24 09:35:24 +11:00
Michael Sproul
a403138ed0 Reduce size of futures in HTTP API to prevent stack overflows (#5104)
* Box::pin a few big futures

* Arc the blocks early in publication

* Fix more tests
2024-01-23 15:32:07 +11:00
Michael Sproul
185646acb2 Fix PublishBlockRequest SSZ decoding (#5078)
* Fix PublishBlockRequest SSZ decoding

* Send header for block v1 ssz client

* Delete unused function
2024-01-19 12:56:30 +11:00
Eitan Seri-Levi
72bcf47dd0 add content-type octet stream helper fn (#5062) 2024-01-15 16:23:39 +11:00
João Oliveira
38df87c3c5 Switch libp2p sigp gossipsub fork (#4999)
* switch libp2p source to sigp fork

* Shift the connection closing inside RPC behaviour

* Tag specific commits

* Add slow peer scoring

* Fix test

* Use default yamux config

* Pin discv5 to our libp2p fork and cargo update

* Upgrade libp2p to enable yamux gains

* Add a comment specifying the branch being used

* cleanup build output from within container
(prevents CI warnings related to fs permissions)

* Remove revision tags add branches for testing, will revert back once we're happy

* Update to latest rust-libp2p version

* Pin forks

* Update cargo.lock

* Re-pin to panic-free rust

---------

Co-authored-by: Age Manning <Age@AgeManning.com>
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: antondlr <anton@delaruelle.net>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2024-01-10 16:26:52 +11:00
Michael Sproul
7e948eec9d Fix block v3 reward encodings (#5049)
* Fix block v3 reward encodings

* Use crates.io version
2024-01-10 10:44:07 +11:00
zhiqiangxu
4a65d28b3b remove needless Into (#5026) 2024-01-10 08:38:16 +11:00
realbigsean
f70c32ec70 create unified slashing cache (#5033)
* create unified slashing cache

* add observed slashable file

* fix broadcast validation tests

* revert block seen cache changes

* clean up slashable cache test

* check header signatures for RPC blobs

* don't throw error on RPC signature invalie
2024-01-08 10:30:57 -05:00
Eitan Seri-Levi
9c1505d082 Block v3 builder boost factor (#5035)
* builder boost factor

* default boost factor

* revert

* deprecate always_prefer_builder_payload, builder-profit-threshold, ignore_builder_override_suggestion_threshold and builder_comparison_factor flags

* revert

* set deprecated flags to no op, revert should_override_builder

* fix test, calc boosted relay value correctly, dont calculate if none

* Add deprecation warnings and restore CLI docs
2024-01-08 11:10:32 +11:00
Michael Sproul
af11e78ae1 Clean up blockv3 metadata and client (#5015)
* Improve block production v3 client

* Delete wayward line

* Overhaul JSON endpoint as well

* Rename timeout param

* Update tests

* I broke everything

* Ah this is an insane fix

* Remove unnecessary optionals

* Doc fix
2023-12-22 09:39:17 -05:00
realbigsean
c55608be10 suppress error on duplicate blobs (#4995) 2023-12-18 12:15:12 -05:00
Jimmy Chen
b0c374c1ca Update dependencies to get rid of the yanked deps (#4994)
* Initial attempt to upgrade hashbrown to latest to get rid of the crate warnings.

* Replace `.expect()` usage with use of `const`.

* Update ahash 0.7 as well

* Remove unsafe code.

* Update `lru` to 0.12 and fix release test errors.

* Set non-blocking socket

* Bump testcontainers to 0.15.

* Fix lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2023-12-15 18:31:59 +11:00