Electra: Remaining Consensus Data Structures (#5712)

* Attestation superstruct changes for EIP 7549 (#5644) * update * experiment * superstruct changes * revert * superstruct changes * fix tests * indexed attestation * indexed attestation superstruct * updated TODOs * `superstruct` the `AttesterSlashing` (#5636) * `superstruct` Attester Fork Variants * Push a little further * Deal with Encode / Decode of AttesterSlashing * not so sure about this.. * Stop Encode/Decode Bounds from Propagating Out * Tons of Changes.. * More Conversions to AttestationRef * Add AsReference trait (#15) * Add AsReference trait * Fix some snafus * Got it Compiling! :D * Got Tests Building * Get beacon chain tests compiling --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes * Make EF Tests Fork-Agnostic (#5713) * Finish EF Test Fork Agnostic (#5714) * Superstruct `AggregateAndProof` (#5715) * Upgrade `superstruct` to `0.8.0` * superstruct `AggregateAndProof` * Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes * cargo fmt * Merge pull request #5726 from realbigsean/electra_attestation_changes Merge unstable into Electra attestation changes * EIP7549 `get_attestation_indices` (#5657) * get attesting indices electra impl * fmt * get tests to pass * fmt * fix some beacon chain tests * fmt * fix slasher test * fmt got me again * fix more tests * fix tests * Some small changes (#5739) * cargo fmt (#5740) * Sketch op pool changes * fix get attesting indices (#5742) * fix get attesting indices * better errors * fix compile * only get committee index once * Ef test fixes (#5753) * attestation related ef test fixes * delete commented out stuff * Fix Aggregation Pool for Electra (#5754) * Fix Aggregation Pool for Electra * Remove Outdated Interface * fix ssz (#5755) * Get `electra_op_pool` up to date (#5756) * fix get attesting indices (#5742) * fix get attesting indices * better errors * fix compile * only get committee index once * Ef test fixes (#5753) * attestation related ef test fixes * delete commented out stuff * Fix Aggregation Pool for Electra (#5754) * Fix Aggregation Pool for Electra * Remove Outdated Interface * fix ssz (#5755) --------- Co-authored-by: realbigsean <sean@sigmaprime.io> * Revert "Get `electra_op_pool` up to date (#5756)" (#5757) This reverts commit ab9e58aa3d. * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool * Compute on chain aggregate impl (#5752) * add compute_on_chain_agg impl to op pool changes * fmt * get op pool tests to pass * update the naive agg pool interface (#5760) * Fix bugs in cross-committee aggregation * Add comment to max cover optimisation * Fix assert * Merge pull request #5749 from sigp/electra_op_pool Optimise Electra op pool aggregation * update committee offset * Fix Electra Fork Choice Tests (#5764) * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * fix slashing handling * Merge remote-tracking branch 'upstream/unstable' * Send unagg attestation based on fork * Publish all aggregates * just one more check bro plz.. * Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable Merge `unstable` into `electra_attestation_changes` * Merge pull request #5835 from realbigsean/fix-validator-logic Fix validator logic * Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling Electra slashing handling * Electra attestation changes rm decode impl (#5856) * Remove Crappy Decode impl for Attestation * Remove Inefficient Attestation Decode impl * Implement Schema Upgrade / Downgrade * Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Fix failing attestation tests and misc electra attestation cleanup (#5810) * - get attestation related beacon chain tests to pass - observed attestations are now keyed off of data + committee index - rename op pool attestationref to compactattestationref - remove unwraps in agg pool and use options instead - cherry pick some changes from ef-tests-electra * cargo fmt * fix failing test * Revert dockerfile changes * make committee_index return option * function args shouldnt be a ref to attestation ref * fmt * fix dup imports --------- Co-authored-by: realbigsean <seananderson33@GMAIL.com> * fix some todos (#5817) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * add consolidations to merkle calc for inclusion proof * Remove Duplicate KZG Commitment Merkle Proof Code (#5874) * Remove Duplicate KZG Commitment Merkle Proof Code * s/tree_lists/fields/ * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * fix compile * Fix slasher tests (#5906) * Fix electra tests * Add electra attestations to double vote tests * Update superstruct to 0.8 * Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes * Small cleanup in slasher tests * Clean up Electra observed aggregates (#5929) * Use consistent key in observed_attestations * Remove unwraps from observed aggregates * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * De-dup attestation constructor logic * Remove unwraps in Attestation construction * Dedup match_attestation_data * Remove outdated TODO * Use ForkName Ord in fork-choice tests * Use ForkName Ord in BeaconBlockBody * Make to_electra not fallible * Remove TestRandom impl for IndexedAttestation * Remove IndexedAttestation faulty Decode impl * Drop TestRandom impl * Add PendingAttestationInElectra * Indexed att on disk (#35) * indexed att on disk * fix lints * Update slasher/src/migrate.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * add electra fork enabled fn to ForkName impl (#36) * add electra fork enabled fn to ForkName impl * remove inadvertent file * Update common/eth2/src/types.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * Dedup attestation constructor logic in attester cache * Use if let Ok for committee_bits * Dedup Attestation constructor code * Diff reduction in tests * Fix beacon_chain tests * Diff reduction * Use Ord for ForkName in pubsub * Resolve into_attestation_and_indices todo * Remove stale TODO * Fix beacon_chain tests * Test spec invariant * Use electra_enabled in pubsub * Remove get_indexed_attestation_from_signed_aggregate * Use ok_or instead of if let else * committees are sorted * remove dup method `get_indexed_attestation_from_committees` * Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview Electra attestations #5712 review * update default persisted op pool deserialization * ensure aggregate and proof uses serde untagged on ref * Fork aware ssz static attestation tests * Electra attestation changes from Lions review (#5971) * dedup/cleanup and remove unneeded hashset use * remove irrelevant TODOs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Electra attestation changes sean review (#5972) * instantiate empty bitlist in unreachable code * clean up error conversion * fork enabled bool cleanup * remove a couple todos * return bools instead of options in `aggregate` and use the result * delete commented out code * use map macros in simple transformations * remove signers_disjoint_from * get ef tests compiling * get ef tests compiling * update intentionally excluded files * Avoid changing slasher schema for Electra * Delete slasher schema v4 * Fix clippy * Fix compilation of beacon_chain tests * Update database.rs * Add electra lightclient types * Update slasher/src/database.rs * fix imports * Merge pull request #5980 from dapplion/electra-lightclient Add electra lightclient types * Merge pull request #5975 from michaelsproul/electra-slasher-no-migration Avoid changing slasher schema for Electra * Update beacon_node/beacon_chain/src/attestation_verification.rs * Update beacon_node/beacon_chain/src/attestation_verification.rs
2026-04-17 04:48:21 +00:00 · 2024-06-24 16:08:07 -05:00
parent 758b58c9e9
commit c52c598f69
118 changed files with 5076 additions and 1741 deletions
--- a/validator_client/src/attestation_service.rs
+++ b/validator_client/src/attestation_service.rs
@@ -14,10 +14,7 @@ use std::ops::Deref;
 use std::sync::Arc;
 use tokio::time::{sleep, sleep_until, Duration, Instant};
 use tree_hash::TreeHash;
-use types::{
-    AggregateSignature, Attestation, AttestationData, BitList, ChainSpec, CommitteeIndex, EthSpec,
-    Slot,
-};
+use types::{Attestation, AttestationData, ChainSpec, CommitteeIndex, EthSpec, Slot};

 /// Builds an `AttestationService`.
 pub struct AttestationServiceBuilder<T: SlotClock + 'static, E: EthSpec> {
@@ -363,9 +360,7 @@ impl<T: SlotClock + 'static, E: EthSpec> AttestationService<T, E> {
            let attestation_data = attestation_data_ref;

            // Ensure that the attestation matches the duties.
-            #[allow(clippy::suspicious_operation_groupings)]
-            if duty.slot != attestation_data.slot || duty.committee_index != attestation_data.index
-            {
+            if !duty.match_attestation_data::<E>(attestation_data, &self.context.eth2_config.spec) {
                crit!(
                    log,
                    "Inconsistent validator duties during signing";
@@ -378,10 +373,26 @@ impl<T: SlotClock + 'static, E: EthSpec> AttestationService<T, E> {
                return None;
            }

-            let mut attestation = Attestation {
-                aggregation_bits: BitList::with_capacity(duty.committee_length as usize).unwrap(),
-                data: attestation_data.clone(),
-                signature: AggregateSignature::infinity(),
+            let mut attestation = match Attestation::<E>::empty_for_signing(
+                duty.committee_index,
+                duty.committee_length as usize,
+                attestation_data.slot,
+                attestation_data.beacon_block_root,
+                attestation_data.source,
+                attestation_data.target,
+                &self.context.eth2_config.spec,
+            ) {
+                Ok(attestation) => attestation,
+                Err(err) => {
+                    crit!(
+                        log,
+                        "Invalid validator duties during signing";
+                        "validator" => ?duty.pubkey,
+                        "duty" => ?duty,
+                        "err" => ?err,
+                    );
+                    return None;
+                }
            };

            match self
@@ -536,10 +547,7 @@ impl<T: SlotClock + 'static, E: EthSpec> AttestationService<T, E> {
            let duty = &duty_and_proof.duty;
            let selection_proof = duty_and_proof.selection_proof.as_ref()?;

-            let slot = attestation_data.slot;
-            let committee_index = attestation_data.index;
-
-            if duty.slot != slot || duty.committee_index != committee_index {
+            if !duty.match_attestation_data::<E>(attestation_data, &self.context.eth2_config.spec) {
                crit!(log, "Inconsistent validator duties during signing");
                return None;
            }
@@ -605,29 +613,29 @@ impl<T: SlotClock + 'static, E: EthSpec> AttestationService<T, E> {
            {
                Ok(()) => {
                    for signed_aggregate_and_proof in signed_aggregate_and_proofs {
-                        let attestation = &signed_aggregate_and_proof.message.aggregate;
+                        let attestation = signed_aggregate_and_proof.message().aggregate();
                        info!(
                            log,
                            "Successfully published attestation";
-                            "aggregator" => signed_aggregate_and_proof.message.aggregator_index,
-                            "signatures" => attestation.aggregation_bits.num_set_bits(),
-                            "head_block" => format!("{:?}", attestation.data.beacon_block_root),
-                            "committee_index" => attestation.data.index,
-                            "slot" => attestation.data.slot.as_u64(),
+                            "aggregator" => signed_aggregate_and_proof.message().aggregator_index(),
+                            "signatures" => attestation.num_set_aggregation_bits(),
+                            "head_block" => format!("{:?}", attestation.data().beacon_block_root),
+                            "committee_index" => attestation.committee_index(),
+                            "slot" => attestation.data().slot.as_u64(),
                            "type" => "aggregated",
                        );
                    }
                }
                Err(e) => {
                    for signed_aggregate_and_proof in signed_aggregate_and_proofs {
-                        let attestation = &signed_aggregate_and_proof.message.aggregate;
+                        let attestation = &signed_aggregate_and_proof.message().aggregate();
                        crit!(
                            log,
                            "Failed to publish attestation";
                            "error" => %e,
-                            "aggregator" => signed_aggregate_and_proof.message.aggregator_index,
-                            "committee_index" => attestation.data.index,
-                            "slot" => attestation.data.slot.as_u64(),
+                            "aggregator" => signed_aggregate_and_proof.message().aggregator_index(),
+                            "committee_index" => attestation.committee_index(),
+                            "slot" => attestation.data().slot.as_u64(),
                            "type" => "aggregated",
                        );
                    }
--- a/validator_client/src/block_service.rs
+++ b/validator_client/src/block_service.rs
@@ -919,8 +919,8 @@ impl<E: EthSpec> SignedBlock<E> {
    }
    pub fn num_attestations(&self) -> usize {
        match self {
-            SignedBlock::Full(block) => block.signed_block().message().body().attestations().len(),
-            SignedBlock::Blinded(block) => block.message().body().attestations().len(),
+            SignedBlock::Full(block) => block.signed_block().message().body().attestations_len(),
+            SignedBlock::Blinded(block) => block.message().body().attestations_len(),
        }
    }
 }
--- a/validator_client/src/http_api/tests/keystores.rs
+++ b/validator_client/src/http_api/tests/keystores.rs
@@ -13,7 +13,7 @@ use rand::{rngs::SmallRng, Rng, SeedableRng};
 use slashing_protection::interchange::{Interchange, InterchangeMetadata};
 use std::{collections::HashMap, path::Path};
 use tokio::runtime::Handle;
-use types::Address;
+use types::{attestation::AttestationBase, Address};

 fn new_keystore(password: ZeroizeString) -> Keystore {
    let keypair = Keypair::random();
@@ -1094,7 +1094,7 @@ async fn generic_migration_test(
        // Sign attestations on VC1.
        for (validator_index, mut attestation) in first_vc_attestations {
            let public_key = keystore_pubkey(&keystores[validator_index]);
-            let current_epoch = attestation.data.target.epoch;
+            let current_epoch = attestation.data().target.epoch;
            tester1
                .validator_store
                .sign_attestation(public_key, 0, &mut attestation, current_epoch)
@@ -1170,7 +1170,7 @@ async fn generic_migration_test(
        // Sign attestations on the second VC.
        for (validator_index, mut attestation, should_succeed) in second_vc_attestations {
            let public_key = keystore_pubkey(&keystores[validator_index]);
-            let current_epoch = attestation.data.target.epoch;
+            let current_epoch = attestation.data().target.epoch;
            match tester2
                .validator_store
                .sign_attestation(public_key, 0, &mut attestation, current_epoch)
@@ -1236,7 +1236,7 @@ async fn delete_nonexistent_keystores() {
 }

 fn make_attestation(source_epoch: u64, target_epoch: u64) -> Attestation<E> {
-    Attestation {
+    Attestation::Base(AttestationBase {
        aggregation_bits: BitList::with_capacity(
            <E as EthSpec>::MaxValidatorsPerCommittee::to_usize(),
        )
@@ -1253,7 +1253,7 @@ fn make_attestation(source_epoch: u64, target_epoch: u64) -> Attestation<E> {
            ..AttestationData::default()
        },
        signature: AggregateSignature::empty(),
-    }
+    })
 }

 #[tokio::test]
--- a/validator_client/src/signing_method.rs
+++ b/validator_client/src/signing_method.rs
@@ -38,7 +38,7 @@ pub enum SignableMessage<'a, E: EthSpec, Payload: AbstractExecPayload<E> = FullP
    RandaoReveal(Epoch),
    BeaconBlock(&'a BeaconBlock<E, Payload>),
    AttestationData(&'a AttestationData),
-    SignedAggregateAndProof(&'a AggregateAndProof<E>),
+    SignedAggregateAndProof(AggregateAndProofRef<'a, E>),
    SelectionProof(Slot),
    SyncSelectionProof(&'a SyncAggregatorSelectionData),
    SyncCommitteeSignature {
--- a/validator_client/src/signing_method/web3signer.rs
+++ b/validator_client/src/signing_method/web3signer.rs
@@ -43,7 +43,7 @@ pub enum Web3SignerObject<'a, E: EthSpec, Payload: AbstractExecPayload<E>> {
    AggregationSlot {
        slot: Slot,
    },
-    AggregateAndProof(&'a AggregateAndProof<E>),
+    AggregateAndProof(AggregateAndProofRef<'a, E>),
    Attestation(&'a AttestationData),
    BeaconBlock {
        version: ForkName,
--- a/validator_client/src/validator_store.rs
+++ b/validator_client/src/validator_store.rs
@@ -647,9 +647,9 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
        current_epoch: Epoch,
    ) -> Result<(), Error> {
        // Make sure the target epoch is not higher than the current epoch to avoid potential attacks.
-        if attestation.data.target.epoch > current_epoch {
+        if attestation.data().target.epoch > current_epoch {
            return Err(Error::GreaterThanCurrentEpoch {
-                epoch: attestation.data.target.epoch,
+                epoch: attestation.data().target.epoch,
                current_epoch,
            });
        }
@@ -658,7 +658,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
        let signing_method = self.doppelganger_checked_signing_method(validator_pubkey)?;

        // Checking for slashing conditions.
-        let signing_epoch = attestation.data.target.epoch;
+        let signing_epoch = attestation.data().target.epoch;
        let signing_context = self.signing_context(Domain::BeaconAttester, signing_epoch);
        let domain_hash = signing_context.domain_hash(&self.spec);
        let slashing_status = if signing_method
@@ -666,7 +666,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
        {
            self.slashing_protection.check_and_insert_attestation(
                &validator_pubkey,
-                &attestation.data,
+                attestation.data(),
                domain_hash,
            )
        } else {
@@ -678,7 +678,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
            Ok(Safe::Valid) => {
                let signature = signing_method
                    .get_signature::<E, BlindedPayload<E>>(
-                        SignableMessage::AttestationData(&attestation.data),
+                        SignableMessage::AttestationData(attestation.data()),
                        signing_context,
                        &self.spec,
                        &self.task_executor,
@@ -720,7 +720,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
                crit!(
                    self.log,
                    "Not signing slashable attestation";
-                    "attestation" => format!("{:?}", attestation.data),
+                    "attestation" => format!("{:?}", attestation.data()),
                    "error" => format!("{:?}", e)
                );
                metrics::inc_counter_vec(
@@ -798,19 +798,16 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
        aggregate: Attestation<E>,
        selection_proof: SelectionProof,
    ) -> Result<SignedAggregateAndProof<E>, Error> {
-        let signing_epoch = aggregate.data.target.epoch;
+        let signing_epoch = aggregate.data().target.epoch;
        let signing_context = self.signing_context(Domain::AggregateAndProof, signing_epoch);

-        let message = AggregateAndProof {
-            aggregator_index,
-            aggregate,
-            selection_proof: selection_proof.into(),
-        };
+        let message =
+            AggregateAndProof::from_attestation(aggregator_index, aggregate, selection_proof);

        let signing_method = self.doppelganger_checked_signing_method(validator_pubkey)?;
        let signature = signing_method
            .get_signature::<E, BlindedPayload<E>>(
-                SignableMessage::SignedAggregateAndProof(&message),
+                SignableMessage::SignedAggregateAndProof(message.to_ref()),
                signing_context,
                &self.spec,
                &self.task_executor,
@@ -819,7 +816,9 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {

        metrics::inc_counter_vec(&metrics::SIGNED_AGGREGATES_TOTAL, &[metrics::SUCCESS]);

-        Ok(SignedAggregateAndProof { message, signature })
+        Ok(SignedAggregateAndProof::from_aggregate_and_proof(
+            message, signature,
+        ))
    }

    /// Produces a `SelectionProof` for the `slot`, signed by with corresponding secret key to