Electra: Remaining Consensus Data Structures (#5712)

* Attestation superstruct changes for EIP 7549 (#5644) * update * experiment * superstruct changes * revert * superstruct changes * fix tests * indexed attestation * indexed attestation superstruct * updated TODOs * `superstruct` the `AttesterSlashing` (#5636) * `superstruct` Attester Fork Variants * Push a little further * Deal with Encode / Decode of AttesterSlashing * not so sure about this.. * Stop Encode/Decode Bounds from Propagating Out * Tons of Changes.. * More Conversions to AttestationRef * Add AsReference trait (#15) * Add AsReference trait * Fix some snafus * Got it Compiling! :D * Got Tests Building * Get beacon chain tests compiling --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes * Make EF Tests Fork-Agnostic (#5713) * Finish EF Test Fork Agnostic (#5714) * Superstruct `AggregateAndProof` (#5715) * Upgrade `superstruct` to `0.8.0` * superstruct `AggregateAndProof` * Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes * cargo fmt * Merge pull request #5726 from realbigsean/electra_attestation_changes Merge unstable into Electra attestation changes * EIP7549 `get_attestation_indices` (#5657) * get attesting indices electra impl * fmt * get tests to pass * fmt * fix some beacon chain tests * fmt * fix slasher test * fmt got me again * fix more tests * fix tests * Some small changes (#5739) * cargo fmt (#5740) * Sketch op pool changes * fix get attesting indices (#5742) * fix get attesting indices * better errors * fix compile * only get committee index once * Ef test fixes (#5753) * attestation related ef test fixes * delete commented out stuff * Fix Aggregation Pool for Electra (#5754) * Fix Aggregation Pool for Electra * Remove Outdated Interface * fix ssz (#5755) * Get `electra_op_pool` up to date (#5756) * fix get attesting indices (#5742) * fix get attesting indices * better errors * fix compile * only get committee index once * Ef test fixes (#5753) * attestation related ef test fixes * delete commented out stuff * Fix Aggregation Pool for Electra (#5754) * Fix Aggregation Pool for Electra * Remove Outdated Interface * fix ssz (#5755) --------- Co-authored-by: realbigsean <sean@sigmaprime.io> * Revert "Get `electra_op_pool` up to date (#5756)" (#5757) This reverts commit ab9e58aa3d. * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool * Compute on chain aggregate impl (#5752) * add compute_on_chain_agg impl to op pool changes * fmt * get op pool tests to pass * update the naive agg pool interface (#5760) * Fix bugs in cross-committee aggregation * Add comment to max cover optimisation * Fix assert * Merge pull request #5749 from sigp/electra_op_pool Optimise Electra op pool aggregation * update committee offset * Fix Electra Fork Choice Tests (#5764) * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * fix slashing handling * Merge remote-tracking branch 'upstream/unstable' * Send unagg attestation based on fork * Publish all aggregates * just one more check bro plz.. * Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable Merge `unstable` into `electra_attestation_changes` * Merge pull request #5835 from realbigsean/fix-validator-logic Fix validator logic * Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling Electra slashing handling * Electra attestation changes rm decode impl (#5856) * Remove Crappy Decode impl for Attestation * Remove Inefficient Attestation Decode impl * Implement Schema Upgrade / Downgrade * Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Fix failing attestation tests and misc electra attestation cleanup (#5810) * - get attestation related beacon chain tests to pass - observed attestations are now keyed off of data + committee index - rename op pool attestationref to compactattestationref - remove unwraps in agg pool and use options instead - cherry pick some changes from ef-tests-electra * cargo fmt * fix failing test * Revert dockerfile changes * make committee_index return option * function args shouldnt be a ref to attestation ref * fmt * fix dup imports --------- Co-authored-by: realbigsean <seananderson33@GMAIL.com> * fix some todos (#5817) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * add consolidations to merkle calc for inclusion proof * Remove Duplicate KZG Commitment Merkle Proof Code (#5874) * Remove Duplicate KZG Commitment Merkle Proof Code * s/tree_lists/fields/ * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * fix compile * Fix slasher tests (#5906) * Fix electra tests * Add electra attestations to double vote tests * Update superstruct to 0.8 * Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes * Small cleanup in slasher tests * Clean up Electra observed aggregates (#5929) * Use consistent key in observed_attestations * Remove unwraps from observed aggregates * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * De-dup attestation constructor logic * Remove unwraps in Attestation construction * Dedup match_attestation_data * Remove outdated TODO * Use ForkName Ord in fork-choice tests * Use ForkName Ord in BeaconBlockBody * Make to_electra not fallible * Remove TestRandom impl for IndexedAttestation * Remove IndexedAttestation faulty Decode impl * Drop TestRandom impl * Add PendingAttestationInElectra * Indexed att on disk (#35) * indexed att on disk * fix lints * Update slasher/src/migrate.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * add electra fork enabled fn to ForkName impl (#36) * add electra fork enabled fn to ForkName impl * remove inadvertent file * Update common/eth2/src/types.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * Dedup attestation constructor logic in attester cache * Use if let Ok for committee_bits * Dedup Attestation constructor code * Diff reduction in tests * Fix beacon_chain tests * Diff reduction * Use Ord for ForkName in pubsub * Resolve into_attestation_and_indices todo * Remove stale TODO * Fix beacon_chain tests * Test spec invariant * Use electra_enabled in pubsub * Remove get_indexed_attestation_from_signed_aggregate * Use ok_or instead of if let else * committees are sorted * remove dup method `get_indexed_attestation_from_committees` * Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview Electra attestations #5712 review * update default persisted op pool deserialization * ensure aggregate and proof uses serde untagged on ref * Fork aware ssz static attestation tests * Electra attestation changes from Lions review (#5971) * dedup/cleanup and remove unneeded hashset use * remove irrelevant TODOs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Electra attestation changes sean review (#5972) * instantiate empty bitlist in unreachable code * clean up error conversion * fork enabled bool cleanup * remove a couple todos * return bools instead of options in `aggregate` and use the result * delete commented out code * use map macros in simple transformations * remove signers_disjoint_from * get ef tests compiling * get ef tests compiling * update intentionally excluded files * Avoid changing slasher schema for Electra * Delete slasher schema v4 * Fix clippy * Fix compilation of beacon_chain tests * Update database.rs * Add electra lightclient types * Update slasher/src/database.rs * fix imports * Merge pull request #5980 from dapplion/electra-lightclient Add electra lightclient types * Merge pull request #5975 from michaelsproul/electra-slasher-no-migration Avoid changing slasher schema for Electra * Update beacon_node/beacon_chain/src/attestation_verification.rs * Update beacon_node/beacon_chain/src/attestation_verification.rs
2026-03-10 12:11:59 +00:00 · 2024-06-24 16:08:07 -05:00
parent 758b58c9e9
commit c52c598f69
118 changed files with 5076 additions and 1741 deletions
--- a/consensus/state_processing/src/common/get_attesting_indices.rs
+++ b/consensus/state_processing/src/common/get_attesting_indices.rs
@@ -1,32 +1,172 @@
 use types::*;

-/// Returns validator indices which participated in the attestation, sorted by increasing index.
-pub fn get_attesting_indices<E: EthSpec>(
-    committee: &[usize],
-    bitlist: &BitList<E::MaxValidatorsPerCommittee>,
-) -> Result<Vec<u64>, BeaconStateError> {
-    if bitlist.len() != committee.len() {
-        return Err(BeaconStateError::InvalidBitfield);
+pub mod attesting_indices_base {
+    use crate::per_block_processing::errors::{AttestationInvalid as Invalid, BlockOperationError};
+    use types::*;
+
+    /// Convert `attestation` to (almost) indexed-verifiable form.
+    ///
+    /// Spec v0.12.1
+    pub fn get_indexed_attestation<E: EthSpec>(
+        committee: &[usize],
+        attestation: &AttestationBase<E>,
+    ) -> Result<IndexedAttestation<E>, BlockOperationError<Invalid>> {
+        let attesting_indices =
+            get_attesting_indices::<E>(committee, &attestation.aggregation_bits)?;
+        Ok(IndexedAttestation::Base(IndexedAttestationBase {
+            attesting_indices: VariableList::new(attesting_indices)?,
+            data: attestation.data.clone(),
+            signature: attestation.signature.clone(),
+        }))
    }

-    let mut indices = Vec::with_capacity(bitlist.num_set_bits());
-
-    for (i, validator_index) in committee.iter().enumerate() {
-        if let Ok(true) = bitlist.get(i) {
-            indices.push(*validator_index as u64)
+    /// Returns validator indices which participated in the attestation, sorted by increasing index.
+    pub fn get_attesting_indices<E: EthSpec>(
+        committee: &[usize],
+        bitlist: &BitList<E::MaxValidatorsPerCommittee>,
+    ) -> Result<Vec<u64>, BeaconStateError> {
+        if bitlist.len() != committee.len() {
+            return Err(BeaconStateError::InvalidBitfield);
        }
+
+        let mut indices = Vec::with_capacity(bitlist.num_set_bits());
+
+        for (i, validator_index) in committee.iter().enumerate() {
+            if let Ok(true) = bitlist.get(i) {
+                indices.push(*validator_index as u64)
+            }
+        }
+
+        indices.sort_unstable();
+
+        Ok(indices)
+    }
+}
+
+pub mod attesting_indices_electra {
+    use std::collections::HashSet;
+
+    use crate::per_block_processing::errors::{AttestationInvalid as Invalid, BlockOperationError};
+    use safe_arith::SafeArith;
+    use types::*;
+
+    /// Compute an Electra IndexedAttestation given a list of committees.
+    ///
+    /// Committees must be sorted by ascending order 0..committees_per_slot
+    pub fn get_indexed_attestation<E: EthSpec>(
+        committees: &[BeaconCommittee],
+        attestation: &AttestationElectra<E>,
+    ) -> Result<IndexedAttestation<E>, BlockOperationError<Invalid>> {
+        let attesting_indices = get_attesting_indices::<E>(
+            committees,
+            &attestation.aggregation_bits,
+            &attestation.committee_bits,
+        )?;
+
+        Ok(IndexedAttestation::Electra(IndexedAttestationElectra {
+            attesting_indices: VariableList::new(attesting_indices)?,
+            data: attestation.data.clone(),
+            signature: attestation.signature.clone(),
+        }))
    }

-    indices.sort_unstable();
+    pub fn get_indexed_attestation_from_state<E: EthSpec>(
+        beacon_state: &BeaconState<E>,
+        attestation: &AttestationElectra<E>,
+    ) -> Result<IndexedAttestation<E>, BlockOperationError<Invalid>> {
+        let committees = beacon_state.get_beacon_committees_at_slot(attestation.data.slot)?;
+        get_indexed_attestation(&committees, attestation)
+    }

-    Ok(indices)
+    /// Shortcut for getting the attesting indices while fetching the committee from the state's cache.
+    pub fn get_attesting_indices_from_state<E: EthSpec>(
+        state: &BeaconState<E>,
+        att: &AttestationElectra<E>,
+    ) -> Result<Vec<u64>, BeaconStateError> {
+        let committees = state.get_beacon_committees_at_slot(att.data.slot)?;
+        get_attesting_indices::<E>(&committees, &att.aggregation_bits, &att.committee_bits)
+    }
+
+    /// Returns validator indices which participated in the attestation, sorted by increasing index.
+    ///
+    /// Committees must be sorted by ascending order 0..committees_per_slot
+    pub fn get_attesting_indices<E: EthSpec>(
+        committees: &[BeaconCommittee],
+        aggregation_bits: &BitList<E::MaxValidatorsPerSlot>,
+        committee_bits: &BitVector<E::MaxCommitteesPerSlot>,
+    ) -> Result<Vec<u64>, BeaconStateError> {
+        let mut attesting_indices = vec![];
+
+        let committee_indices = get_committee_indices::<E>(committee_bits);
+
+        let mut committee_offset = 0;
+
+        let committee_count_per_slot = committees.len() as u64;
+        let mut participant_count = 0;
+        for index in committee_indices {
+            let beacon_committee = committees
+                .get(index as usize)
+                .ok_or(Error::NoCommitteeFound(index))?;
+
+            // This check is new to the spec's `process_attestation` in Electra.
+            if index >= committee_count_per_slot {
+                return Err(BeaconStateError::InvalidCommitteeIndex(index));
+            }
+            participant_count.safe_add_assign(beacon_committee.committee.len() as u64)?;
+            let committee_attesters = beacon_committee
+                .committee
+                .iter()
+                .enumerate()
+                .filter_map(|(i, &index)| {
+                    if let Ok(aggregation_bit_index) = committee_offset.safe_add(i) {
+                        if aggregation_bits.get(aggregation_bit_index).unwrap_or(false) {
+                            return Some(index as u64);
+                        }
+                    }
+                    None
+                })
+                .collect::<HashSet<u64>>();
+
+            attesting_indices.extend(committee_attesters);
+            committee_offset.safe_add_assign(beacon_committee.committee.len())?;
+        }
+
+        // This check is new to the spec's `process_attestation` in Electra.
+        if participant_count as usize != aggregation_bits.len() {
+            return Err(BeaconStateError::InvalidBitfield);
+        }
+
+        attesting_indices.sort_unstable();
+
+        Ok(attesting_indices)
+    }
+
+    pub fn get_committee_indices<E: EthSpec>(
+        committee_bits: &BitVector<E::MaxCommitteesPerSlot>,
+    ) -> Vec<CommitteeIndex> {
+        committee_bits
+            .iter()
+            .enumerate()
+            .filter_map(|(index, bit)| if bit { Some(index as u64) } else { None })
+            .collect()
+    }
 }

 /// Shortcut for getting the attesting indices while fetching the committee from the state's cache.
 pub fn get_attesting_indices_from_state<E: EthSpec>(
    state: &BeaconState<E>,
-    att: &Attestation<E>,
+    att: AttestationRef<E>,
 ) -> Result<Vec<u64>, BeaconStateError> {
-    let committee = state.get_beacon_committee(att.data.slot, att.data.index)?;
-    get_attesting_indices::<E>(committee.committee, &att.aggregation_bits)
+    match att {
+        AttestationRef::Base(att) => {
+            let committee = state.get_beacon_committee(att.data.slot, att.data.index)?;
+            attesting_indices_base::get_attesting_indices::<E>(
+                committee.committee,
+                &att.aggregation_bits,
+            )
+        }
+        AttestationRef::Electra(att) => {
+            attesting_indices_electra::get_attesting_indices_from_state::<E>(state, att)
+        }
+    }
 }
--- a/consensus/state_processing/src/common/get_indexed_attestation.rs
+++ b/consensus/state_processing/src/common/get_indexed_attestation.rs
@@ -1,21 +0,0 @@
-use super::get_attesting_indices;
-use crate::per_block_processing::errors::{AttestationInvalid as Invalid, BlockOperationError};
-use types::*;
-
-type Result<T> = std::result::Result<T, BlockOperationError<Invalid>>;
-
-/// Convert `attestation` to (almost) indexed-verifiable form.
-///
-/// Spec v0.12.1
-pub fn get_indexed_attestation<E: EthSpec>(
-    committee: &[usize],
-    attestation: &Attestation<E>,
-) -> Result<IndexedAttestation<E>> {
-    let attesting_indices = get_attesting_indices::<E>(committee, &attestation.aggregation_bits)?;
-
-    Ok(IndexedAttestation {
-        attesting_indices: VariableList::new(attesting_indices)?,
-        data: attestation.data.clone(),
-        signature: attestation.signature.clone(),
-    })
-}
--- a/consensus/state_processing/src/common/mod.rs
+++ b/consensus/state_processing/src/common/mod.rs
@@ -1,7 +1,6 @@
 mod deposit_data_tree;
 mod get_attestation_participation;
 mod get_attesting_indices;
-mod get_indexed_attestation;
 mod initiate_validator_exit;
 mod slash_validator;

@@ -11,8 +10,9 @@ pub mod update_progressive_balances_cache;

 pub use deposit_data_tree::DepositDataTree;
 pub use get_attestation_participation::get_attestation_participation_flag_indices;
-pub use get_attesting_indices::{get_attesting_indices, get_attesting_indices_from_state};
-pub use get_indexed_attestation::get_indexed_attestation;
+pub use get_attesting_indices::{
+    attesting_indices_base, attesting_indices_electra, get_attesting_indices_from_state,
+};
 pub use initiate_validator_exit::initiate_validator_exit;
 pub use slash_validator::slash_validator;