Electra: Remaining Consensus Data Structures (#5712)

* Attestation superstruct changes for EIP 7549 (#5644)

* update

* experiment

* superstruct changes

* revert

* superstruct changes

* fix tests

* indexed attestation

* indexed attestation superstruct

* updated TODOs

* `superstruct` the `AttesterSlashing` (#5636)

* `superstruct` Attester Fork Variants

* Push a little further

* Deal with Encode / Decode of AttesterSlashing

* not so sure about this..

* Stop Encode/Decode Bounds from Propagating Out

* Tons of Changes..

* More Conversions to AttestationRef

* Add AsReference trait (#15)

* Add AsReference trait

* Fix some snafus

* Got it Compiling! :D

* Got Tests Building

* Get beacon chain tests compiling

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes

* Make EF Tests Fork-Agnostic (#5713)

* Finish EF Test Fork Agnostic (#5714)

* Superstruct `AggregateAndProof` (#5715)

* Upgrade `superstruct` to `0.8.0`

* superstruct `AggregateAndProof`

* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes

* cargo fmt

* Merge pull request #5726 from realbigsean/electra_attestation_changes

Merge unstable into Electra attestation changes

* EIP7549 `get_attestation_indices` (#5657)

* get attesting indices electra impl

* fmt

* get tests to pass

* fmt

* fix some beacon chain tests

* fmt

* fix slasher test

* fmt got me again

* fix more tests

* fix tests

* Some small changes (#5739)

* cargo fmt (#5740)

* Sketch op pool changes

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

* Get `electra_op_pool` up to date (#5756)

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>

* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)

This reverts commit ab9e58aa3d.

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool

* Compute on chain aggregate impl (#5752)

* add compute_on_chain_agg impl to op pool changes

* fmt

* get op pool tests to pass

* update the naive agg pool interface (#5760)

* Fix bugs in cross-committee aggregation

* Add comment to max cover optimisation

* Fix assert

* Merge pull request #5749 from sigp/electra_op_pool

Optimise Electra op pool aggregation

* update committee offset

* Fix Electra Fork Choice Tests (#5764)

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* fix slashing handling

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request #5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Electra attestation changes rm decode impl (#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Remove Duplicate KZG Commitment Merkle Proof Code (#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* fix compile

* Fix slasher tests (#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations #5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Electra attestation changes sean review (#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request #5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request #5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs
This commit is contained in:
ethDreamer
2024-06-24 16:08:07 -05:00
committed by GitHub
parent 758b58c9e9
commit c52c598f69
118 changed files with 5076 additions and 1741 deletions

View File

@@ -14,8 +14,8 @@ use std::marker::PhantomData;
use std::time::Duration;
use types::{
consts::bellatrix::INTERVALS_PER_SLOT, AbstractExecPayload, AttestationShufflingId,
AttesterSlashing, BeaconBlockRef, BeaconState, BeaconStateError, ChainSpec, Checkpoint, Epoch,
EthSpec, ExecPayload, ExecutionBlockHash, Hash256, IndexedAttestation, RelativeEpoch,
AttesterSlashingRef, BeaconBlockRef, BeaconState, BeaconStateError, ChainSpec, Checkpoint,
Epoch, EthSpec, ExecPayload, ExecutionBlockHash, Hash256, IndexedAttestationRef, RelativeEpoch,
SignedBeaconBlock, Slot,
};
@@ -238,13 +238,13 @@ pub struct QueuedAttestation {
target_epoch: Epoch,
}
impl<E: EthSpec> From<&IndexedAttestation<E>> for QueuedAttestation {
fn from(a: &IndexedAttestation<E>) -> Self {
impl<'a, E: EthSpec> From<IndexedAttestationRef<'a, E>> for QueuedAttestation {
fn from(a: IndexedAttestationRef<'a, E>) -> Self {
Self {
slot: a.data.slot,
attesting_indices: a.attesting_indices[..].to_vec(),
block_root: a.data.beacon_block_root,
target_epoch: a.data.target.epoch,
slot: a.data().slot,
attesting_indices: a.attesting_indices_to_vec(),
block_root: a.data().beacon_block_root,
target_epoch: a.data().target.epoch,
}
}
}
@@ -940,7 +940,7 @@ where
/// https://github.com/ethereum/eth2.0-specs/blob/v0.12.1/specs/phase0/fork-choice.md#validate_on_attestation
fn validate_on_attestation(
&self,
indexed_attestation: &IndexedAttestation<E>,
indexed_attestation: IndexedAttestationRef<E>,
is_from_block: AttestationFromBlock,
) -> Result<(), InvalidAttestation> {
// There is no point in processing an attestation with an empty bitfield. Reject
@@ -948,20 +948,20 @@ where
//
// This is not in the specification, however it should be transparent to other nodes. We
// return early here to avoid wasting precious resources verifying the rest of it.
if indexed_attestation.attesting_indices.is_empty() {
if indexed_attestation.attesting_indices_is_empty() {
return Err(InvalidAttestation::EmptyAggregationBitfield);
}
let target = indexed_attestation.data.target;
let target = indexed_attestation.data().target;
if matches!(is_from_block, AttestationFromBlock::False) {
self.validate_target_epoch_against_current_time(target.epoch)?;
}
if target.epoch != indexed_attestation.data.slot.epoch(E::slots_per_epoch()) {
if target.epoch != indexed_attestation.data().slot.epoch(E::slots_per_epoch()) {
return Err(InvalidAttestation::BadTargetEpoch {
target: target.epoch,
slot: indexed_attestation.data.slot,
slot: indexed_attestation.data().slot,
});
}
@@ -983,9 +983,9 @@ where
// attestation and do not delay consideration for later.
let block = self
.proto_array
.get_block(&indexed_attestation.data.beacon_block_root)
.get_block(&indexed_attestation.data().beacon_block_root)
.ok_or(InvalidAttestation::UnknownHeadBlock {
beacon_block_root: indexed_attestation.data.beacon_block_root,
beacon_block_root: indexed_attestation.data().beacon_block_root,
})?;
// If an attestation points to a block that is from an earlier slot than the attestation,
@@ -993,7 +993,7 @@ where
// is from a prior epoch to the attestation, then the target root must be equal to the root
// of the block that is being attested to.
let expected_target = if target.epoch > block.slot.epoch(E::slots_per_epoch()) {
indexed_attestation.data.beacon_block_root
indexed_attestation.data().beacon_block_root
} else {
block.target_root
};
@@ -1007,10 +1007,10 @@ where
// Attestations must not be for blocks in the future. If this is the case, the attestation
// should not be considered.
if block.slot > indexed_attestation.data.slot {
if block.slot > indexed_attestation.data().slot {
return Err(InvalidAttestation::AttestsToFutureBlock {
block: block.slot,
attestation: indexed_attestation.data.slot,
attestation: indexed_attestation.data().slot,
});
}
@@ -1037,7 +1037,7 @@ where
pub fn on_attestation(
&mut self,
system_time_current_slot: Slot,
attestation: &IndexedAttestation<E>,
attestation: IndexedAttestationRef<E>,
is_from_block: AttestationFromBlock,
) -> Result<(), Error<T::Error>> {
self.update_time(system_time_current_slot)?;
@@ -1055,18 +1055,18 @@ where
// (1) becomes weird once we hit finality and fork choice drops the genesis block. (2) is
// fine because votes to the genesis block are not useful; all validators implicitly attest
// to genesis just by being present in the chain.
if attestation.data.beacon_block_root == Hash256::zero() {
if attestation.data().beacon_block_root == Hash256::zero() {
return Ok(());
}
self.validate_on_attestation(attestation, is_from_block)?;
if attestation.data.slot < self.fc_store.get_current_slot() {
for validator_index in attestation.attesting_indices.iter() {
if attestation.data().slot < self.fc_store.get_current_slot() {
for validator_index in attestation.attesting_indices_iter() {
self.proto_array.process_attestation(
*validator_index as usize,
attestation.data.beacon_block_root,
attestation.data.target.epoch,
attestation.data().beacon_block_root,
attestation.data().target.epoch,
)?;
}
} else {
@@ -1086,15 +1086,14 @@ where
/// Apply an attester slashing to fork choice.
///
/// We assume that the attester slashing provided to this function has already been verified.
pub fn on_attester_slashing(&mut self, slashing: &AttesterSlashing<E>) {
let attesting_indices_set = |att: &IndexedAttestation<E>| {
att.attesting_indices
.iter()
pub fn on_attester_slashing(&mut self, slashing: AttesterSlashingRef<'_, E>) {
let attesting_indices_set = |att: IndexedAttestationRef<'_, E>| {
att.attesting_indices_iter()
.copied()
.collect::<BTreeSet<_>>()
};
let att1_indices = attesting_indices_set(&slashing.attestation_1);
let att2_indices = attesting_indices_set(&slashing.attestation_2);
let att1_indices = attesting_indices_set(slashing.attestation_1());
let att2_indices = attesting_indices_set(slashing.attestation_2());
self.fc_store
.extend_equivocating_indices(att1_indices.intersection(&att2_indices).copied());
}