Electra: Remaining Consensus Data Structures (#5712)

* Attestation superstruct changes for EIP 7549 (#5644) * update * experiment * superstruct changes * revert * superstruct changes * fix tests * indexed attestation * indexed attestation superstruct * updated TODOs * `superstruct` the `AttesterSlashing` (#5636) * `superstruct` Attester Fork Variants * Push a little further * Deal with Encode / Decode of AttesterSlashing * not so sure about this.. * Stop Encode/Decode Bounds from Propagating Out * Tons of Changes.. * More Conversions to AttestationRef * Add AsReference trait (#15) * Add AsReference trait * Fix some snafus * Got it Compiling! :D * Got Tests Building * Get beacon chain tests compiling --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes * Make EF Tests Fork-Agnostic (#5713) * Finish EF Test Fork Agnostic (#5714) * Superstruct `AggregateAndProof` (#5715) * Upgrade `superstruct` to `0.8.0` * superstruct `AggregateAndProof` * Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes * cargo fmt * Merge pull request #5726 from realbigsean/electra_attestation_changes Merge unstable into Electra attestation changes * EIP7549 `get_attestation_indices` (#5657) * get attesting indices electra impl * fmt * get tests to pass * fmt * fix some beacon chain tests * fmt * fix slasher test * fmt got me again * fix more tests * fix tests * Some small changes (#5739) * cargo fmt (#5740) * Sketch op pool changes * fix get attesting indices (#5742) * fix get attesting indices * better errors * fix compile * only get committee index once * Ef test fixes (#5753) * attestation related ef test fixes * delete commented out stuff * Fix Aggregation Pool for Electra (#5754) * Fix Aggregation Pool for Electra * Remove Outdated Interface * fix ssz (#5755) * Get `electra_op_pool` up to date (#5756) * fix get attesting indices (#5742) * fix get attesting indices * better errors * fix compile * only get committee index once * Ef test fixes (#5753) * attestation related ef test fixes * delete commented out stuff * Fix Aggregation Pool for Electra (#5754) * Fix Aggregation Pool for Electra * Remove Outdated Interface * fix ssz (#5755) --------- Co-authored-by: realbigsean <sean@sigmaprime.io> * Revert "Get `electra_op_pool` up to date (#5756)" (#5757) This reverts commit ab9e58aa3d. * Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool * Compute on chain aggregate impl (#5752) * add compute_on_chain_agg impl to op pool changes * fmt * get op pool tests to pass * update the naive agg pool interface (#5760) * Fix bugs in cross-committee aggregation * Add comment to max cover optimisation * Fix assert * Merge pull request #5749 from sigp/electra_op_pool Optimise Electra op pool aggregation * update committee offset * Fix Electra Fork Choice Tests (#5764) * Subscribe to the correct subnets for electra attestations (#5782) * subscribe to the correct att subnets for electra * subscribe to the correct att subnets for electra * cargo fmt * fix slashing handling * Merge remote-tracking branch 'upstream/unstable' * Send unagg attestation based on fork * Publish all aggregates * just one more check bro plz.. * Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable Merge `unstable` into `electra_attestation_changes` * Merge pull request #5835 from realbigsean/fix-validator-logic Fix validator logic * Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling Electra slashing handling * Electra attestation changes rm decode impl (#5856) * Remove Crappy Decode impl for Attestation * Remove Inefficient Attestation Decode impl * Implement Schema Upgrade / Downgrade * Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs Co-authored-by: Michael Sproul <micsproul@gmail.com> --------- Co-authored-by: Michael Sproul <micsproul@gmail.com> * Fix failing attestation tests and misc electra attestation cleanup (#5810) * - get attestation related beacon chain tests to pass - observed attestations are now keyed off of data + committee index - rename op pool attestationref to compactattestationref - remove unwraps in agg pool and use options instead - cherry pick some changes from ef-tests-electra * cargo fmt * fix failing test * Revert dockerfile changes * make committee_index return option * function args shouldnt be a ref to attestation ref * fmt * fix dup imports --------- Co-authored-by: realbigsean <seananderson33@GMAIL.com> * fix some todos (#5817) * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * add consolidations to merkle calc for inclusion proof * Remove Duplicate KZG Commitment Merkle Proof Code (#5874) * Remove Duplicate KZG Commitment Merkle Proof Code * s/tree_lists/fields/ * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * fix compile * Fix slasher tests (#5906) * Fix electra tests * Add electra attestations to double vote tests * Update superstruct to 0.8 * Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes * Small cleanup in slasher tests * Clean up Electra observed aggregates (#5929) * Use consistent key in observed_attestations * Remove unwraps from observed aggregates * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * De-dup attestation constructor logic * Remove unwraps in Attestation construction * Dedup match_attestation_data * Remove outdated TODO * Use ForkName Ord in fork-choice tests * Use ForkName Ord in BeaconBlockBody * Make to_electra not fallible * Remove TestRandom impl for IndexedAttestation * Remove IndexedAttestation faulty Decode impl * Drop TestRandom impl * Add PendingAttestationInElectra * Indexed att on disk (#35) * indexed att on disk * fix lints * Update slasher/src/migrate.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> --------- Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com> Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * add electra fork enabled fn to ForkName impl (#36) * add electra fork enabled fn to ForkName impl * remove inadvertent file * Update common/eth2/src/types.rs Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com> * Dedup attestation constructor logic in attester cache * Use if let Ok for committee_bits * Dedup Attestation constructor code * Diff reduction in tests * Fix beacon_chain tests * Diff reduction * Use Ord for ForkName in pubsub * Resolve into_attestation_and_indices todo * Remove stale TODO * Fix beacon_chain tests * Test spec invariant * Use electra_enabled in pubsub * Remove get_indexed_attestation_from_signed_aggregate * Use ok_or instead of if let else * committees are sorted * remove dup method `get_indexed_attestation_from_committees` * Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview Electra attestations #5712 review * update default persisted op pool deserialization * ensure aggregate and proof uses serde untagged on ref * Fork aware ssz static attestation tests * Electra attestation changes from Lions review (#5971) * dedup/cleanup and remove unneeded hashset use * remove irrelevant TODOs * Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes * Electra attestation changes sean review (#5972) * instantiate empty bitlist in unreachable code * clean up error conversion * fork enabled bool cleanup * remove a couple todos * return bools instead of options in `aggregate` and use the result * delete commented out code * use map macros in simple transformations * remove signers_disjoint_from * get ef tests compiling * get ef tests compiling * update intentionally excluded files * Avoid changing slasher schema for Electra * Delete slasher schema v4 * Fix clippy * Fix compilation of beacon_chain tests * Update database.rs * Add electra lightclient types * Update slasher/src/database.rs * fix imports * Merge pull request #5980 from dapplion/electra-lightclient Add electra lightclient types * Merge pull request #5975 from michaelsproul/electra-slasher-no-migration Avoid changing slasher schema for Electra * Update beacon_node/beacon_chain/src/attestation_verification.rs * Update beacon_node/beacon_chain/src/attestation_verification.rs
2026-03-09 19:51:47 +00:00 · 2024-06-24 16:08:07 -05:00
parent 758b58c9e9
commit c52c598f69
118 changed files with 5076 additions and 1741 deletions
--- a/consensus/state_processing/src/per_block_processing/tests.rs
+++ b/consensus/state_processing/src/per_block_processing/tests.rs
@@ -388,8 +388,13 @@ async fn invalid_attestation_no_committee_for_index() {
        .clone()
        .deconstruct()
        .0;
-    head_block.to_mut().body_mut().attestations_mut()[0]
-        .data
+    head_block
+        .to_mut()
+        .body_mut()
+        .attestations_mut()
+        .next()
+        .unwrap()
+        .data_mut()
        .index += 1;
    let mut ctxt = ConsensusContext::new(state.slot());
    let result = process_operations::process_attestations(
@@ -423,11 +428,22 @@ async fn invalid_attestation_wrong_justified_checkpoint() {
        .clone()
        .deconstruct()
        .0;
-    let old_justified_checkpoint = head_block.body().attestations()[0].data.source;
+    let old_justified_checkpoint = head_block
+        .body()
+        .attestations()
+        .next()
+        .unwrap()
+        .data()
+        .source;
    let mut new_justified_checkpoint = old_justified_checkpoint;
    new_justified_checkpoint.epoch += Epoch::new(1);
-    head_block.to_mut().body_mut().attestations_mut()[0]
-        .data
+    head_block
+        .to_mut()
+        .body_mut()
+        .attestations_mut()
+        .next()
+        .unwrap()
+        .data_mut()
        .source = new_justified_checkpoint;

    let mut ctxt = ConsensusContext::new(state.slot());
@@ -467,8 +483,14 @@ async fn invalid_attestation_bad_aggregation_bitfield_len() {
        .clone()
        .deconstruct()
        .0;
-    head_block.to_mut().body_mut().attestations_mut()[0].aggregation_bits =
-        Bitfield::with_capacity(spec.target_committee_size).unwrap();
+    *head_block
+        .to_mut()
+        .body_mut()
+        .attestations_mut()
+        .next()
+        .unwrap()
+        .aggregation_bits_base_mut()
+        .unwrap() = Bitfield::with_capacity(spec.target_committee_size).unwrap();

    let mut ctxt = ConsensusContext::new(state.slot());
    let result = process_operations::process_attestations(
@@ -501,7 +523,13 @@ async fn invalid_attestation_bad_signature() {
        .clone()
        .deconstruct()
        .0;
-    head_block.to_mut().body_mut().attestations_mut()[0].signature = AggregateSignature::empty();
+    *head_block
+        .to_mut()
+        .body_mut()
+        .attestations_mut()
+        .next()
+        .unwrap()
+        .signature_mut() = AggregateSignature::empty();

    let mut ctxt = ConsensusContext::new(state.slot());
    let result = process_operations::process_attestations(
@@ -536,10 +564,15 @@ async fn invalid_attestation_included_too_early() {
        .clone()
        .deconstruct()
        .0;
-    let new_attesation_slot = head_block.body().attestations()[0].data.slot
+    let new_attesation_slot = head_block.body().attestations().next().unwrap().data().slot
        + Slot::new(MainnetEthSpec::slots_per_epoch());
-    head_block.to_mut().body_mut().attestations_mut()[0]
-        .data
+    head_block
+        .to_mut()
+        .body_mut()
+        .attestations_mut()
+        .next()
+        .unwrap()
+        .data_mut()
        .slot = new_attesation_slot;

    let mut ctxt = ConsensusContext::new(state.slot());
@@ -579,10 +612,15 @@ async fn invalid_attestation_included_too_late() {
        .clone()
        .deconstruct()
        .0;
-    let new_attesation_slot = head_block.body().attestations()[0].data.slot
+    let new_attesation_slot = head_block.body().attestations().next().unwrap().data().slot
        - Slot::new(MainnetEthSpec::slots_per_epoch());
-    head_block.to_mut().body_mut().attestations_mut()[0]
-        .data
+    head_block
+        .to_mut()
+        .body_mut()
+        .attestations_mut()
+        .next()
+        .unwrap()
+        .data_mut()
        .slot = new_attesation_slot;

    let mut ctxt = ConsensusContext::new(state.slot());
@@ -619,8 +657,13 @@ async fn invalid_attestation_target_epoch_slot_mismatch() {
        .clone()
        .deconstruct()
        .0;
-    head_block.to_mut().body_mut().attestations_mut()[0]
-        .data
+    head_block
+        .to_mut()
+        .body_mut()
+        .attestations_mut()
+        .next()
+        .unwrap()
+        .data_mut()
        .target
        .epoch += Epoch::new(1);

@@ -655,7 +698,7 @@ async fn valid_insert_attester_slashing() {
    let mut ctxt = ConsensusContext::new(state.slot());
    let result = process_operations::process_attester_slashings(
        &mut state,
-        &[attester_slashing],
+        [attester_slashing.to_ref()].into_iter(),
        VerifySignatures::True,
        &mut ctxt,
        &spec,
@@ -671,13 +714,20 @@ async fn invalid_attester_slashing_not_slashable() {
    let harness = get_harness::<MainnetEthSpec>(EPOCH_OFFSET, VALIDATOR_COUNT).await;

    let mut attester_slashing = harness.make_attester_slashing(vec![1, 2]);
-    attester_slashing.attestation_1 = attester_slashing.attestation_2.clone();
+    match &mut attester_slashing {
+        AttesterSlashing::Base(ref mut attester_slashing) => {
+            attester_slashing.attestation_1 = attester_slashing.attestation_2.clone();
+        }
+        AttesterSlashing::Electra(ref mut attester_slashing) => {
+            attester_slashing.attestation_1 = attester_slashing.attestation_2.clone();
+        }
+    }

    let mut state = harness.get_current_state();
    let mut ctxt = ConsensusContext::new(state.slot());
    let result = process_operations::process_attester_slashings(
        &mut state,
-        &[attester_slashing],
+        [attester_slashing.to_ref()].into_iter(),
        VerifySignatures::True,
        &mut ctxt,
        &spec,
@@ -699,13 +749,20 @@ async fn invalid_attester_slashing_1_invalid() {
    let harness = get_harness::<MainnetEthSpec>(EPOCH_OFFSET, VALIDATOR_COUNT).await;

    let mut attester_slashing = harness.make_attester_slashing(vec![1, 2]);
-    attester_slashing.attestation_1.attesting_indices = VariableList::from(vec![2, 1]);
+    match &mut attester_slashing {
+        AttesterSlashing::Base(ref mut attester_slashing) => {
+            attester_slashing.attestation_1.attesting_indices = VariableList::from(vec![2, 1]);
+        }
+        AttesterSlashing::Electra(ref mut attester_slashing) => {
+            attester_slashing.attestation_1.attesting_indices = VariableList::from(vec![2, 1]);
+        }
+    }

    let mut state = harness.get_current_state();
    let mut ctxt = ConsensusContext::new(state.slot());
    let result = process_operations::process_attester_slashings(
        &mut state,
-        &[attester_slashing],
+        [attester_slashing.to_ref()].into_iter(),
        VerifySignatures::True,
        &mut ctxt,
        &spec,
@@ -730,13 +787,20 @@ async fn invalid_attester_slashing_2_invalid() {
    let harness = get_harness::<MainnetEthSpec>(EPOCH_OFFSET, VALIDATOR_COUNT).await;

    let mut attester_slashing = harness.make_attester_slashing(vec![1, 2]);
-    attester_slashing.attestation_2.attesting_indices = VariableList::from(vec![2, 1]);
+    match &mut attester_slashing {
+        AttesterSlashing::Base(ref mut attester_slashing) => {
+            attester_slashing.attestation_2.attesting_indices = VariableList::from(vec![2, 1]);
+        }
+        AttesterSlashing::Electra(ref mut attester_slashing) => {
+            attester_slashing.attestation_2.attesting_indices = VariableList::from(vec![2, 1]);
+        }
+    }

    let mut state = harness.get_current_state();
    let mut ctxt = ConsensusContext::new(state.slot());
    let result = process_operations::process_attester_slashings(
        &mut state,
-        &[attester_slashing],
+        [attester_slashing.to_ref()].into_iter(),
        VerifySignatures::True,
        &mut ctxt,
        &spec,