Files
lighthouse/beacon_node/operation_pool/src/attestation.rs
ethDreamer c52c598f69 Electra: Remaining Consensus Data Structures (#5712)
* Attestation superstruct changes for EIP 7549 (#5644)

* update

* experiment

* superstruct changes

* revert

* superstruct changes

* fix tests

* indexed attestation

* indexed attestation superstruct

* updated TODOs

* `superstruct` the `AttesterSlashing` (#5636)

* `superstruct` Attester Fork Variants

* Push a little further

* Deal with Encode / Decode of AttesterSlashing

* not so sure about this..

* Stop Encode/Decode Bounds from Propagating Out

* Tons of Changes..

* More Conversions to AttestationRef

* Add AsReference trait (#15)

* Add AsReference trait

* Fix some snafus

* Got it Compiling! :D

* Got Tests Building

* Get beacon chain tests compiling

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes

* Make EF Tests Fork-Agnostic (#5713)

* Finish EF Test Fork Agnostic (#5714)

* Superstruct `AggregateAndProof` (#5715)

* Upgrade `superstruct` to `0.8.0`

* superstruct `AggregateAndProof`

* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes

* cargo fmt

* Merge pull request #5726 from realbigsean/electra_attestation_changes

Merge unstable into Electra attestation changes

* EIP7549 `get_attestation_indices` (#5657)

* get attesting indices electra impl

* fmt

* get tests to pass

* fmt

* fix some beacon chain tests

* fmt

* fix slasher test

* fmt got me again

* fix more tests

* fix tests

* Some small changes (#5739)

* cargo fmt (#5740)

* Sketch op pool changes

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

* Get `electra_op_pool` up to date (#5756)

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>

* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)

This reverts commit ab9e58aa3d.

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool

* Compute on chain aggregate impl (#5752)

* add compute_on_chain_agg impl to op pool changes

* fmt

* get op pool tests to pass

* update the naive agg pool interface (#5760)

* Fix bugs in cross-committee aggregation

* Add comment to max cover optimisation

* Fix assert

* Merge pull request #5749 from sigp/electra_op_pool

Optimise Electra op pool aggregation

* update committee offset

* Fix Electra Fork Choice Tests (#5764)

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* fix slashing handling

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request #5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Electra attestation changes rm decode impl (#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Remove Duplicate KZG Commitment Merkle Proof Code (#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* fix compile

* Fix slasher tests (#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations #5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Electra attestation changes sean review (#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request #5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request #5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs
2024-06-24 21:08:07 +00:00

213 lines
8.7 KiB
Rust

use crate::attestation_storage::{CompactAttestationRef, CompactIndexedAttestation};
use crate::max_cover::MaxCover;
use crate::reward_cache::RewardCache;
use state_processing::common::{
attesting_indices_base::get_attesting_indices, base, get_attestation_participation_flag_indices,
};
use std::collections::HashMap;
use types::{
beacon_state::BeaconStateBase,
consts::altair::{PARTICIPATION_FLAG_WEIGHTS, WEIGHT_DENOMINATOR},
Attestation, BeaconState, BitList, ChainSpec, EthSpec,
};
#[derive(Debug, Clone)]
pub struct AttMaxCover<'a, E: EthSpec> {
/// Underlying attestation.
pub att: CompactAttestationRef<'a, E>,
/// Mapping of validator indices and their rewards.
pub fresh_validators_rewards: HashMap<u64, u64>,
}
impl<'a, E: EthSpec> AttMaxCover<'a, E> {
pub fn new(
att: CompactAttestationRef<'a, E>,
state: &BeaconState<E>,
reward_cache: &'a RewardCache,
total_active_balance: u64,
spec: &ChainSpec,
) -> Option<Self> {
if let BeaconState::Base(ref base_state) = state {
Self::new_for_base(att, state, base_state, total_active_balance, spec)
} else {
Self::new_for_altair_deneb(att, state, reward_cache, spec)
}
}
/// Initialise an attestation cover object for base/phase0 hard fork.
pub fn new_for_base(
att: CompactAttestationRef<'a, E>,
state: &BeaconState<E>,
base_state: &BeaconStateBase<E>,
total_active_balance: u64,
spec: &ChainSpec,
) -> Option<Self> {
let fresh_validators = earliest_attestation_validators(&att, state, base_state);
let committee = state
.get_beacon_committee(att.data.slot, att.data.index)
.ok()?;
let indices = get_attesting_indices::<E>(committee.committee, &fresh_validators).ok()?;
let sqrt_total_active_balance = base::SqrtTotalActiveBalance::new(total_active_balance);
let fresh_validators_rewards: HashMap<u64, u64> = indices
.iter()
.copied()
.flat_map(|validator_index| {
let effective_balance =
state.get_effective_balance(validator_index as usize).ok()?;
let reward =
base::get_base_reward(effective_balance, sqrt_total_active_balance, spec)
.ok()?
.checked_div(spec.proposer_reward_quotient)?;
Some((validator_index, reward))
})
.collect();
Some(Self {
att,
fresh_validators_rewards,
})
}
/// Initialise an attestation cover object for Altair or later.
pub fn new_for_altair_deneb(
att: CompactAttestationRef<'a, E>,
state: &BeaconState<E>,
reward_cache: &'a RewardCache,
spec: &ChainSpec,
) -> Option<Self> {
let att_data = att.attestation_data();
let inclusion_delay = state.slot().as_u64().checked_sub(att_data.slot.as_u64())?;
let att_participation_flags =
get_attestation_participation_flag_indices(state, &att_data, inclusion_delay, spec)
.ok()?;
let fresh_validators_rewards = att
.indexed
.attesting_indices()
.iter()
.filter_map(|&index| {
if reward_cache
.has_attested_in_epoch(index, att_data.target.epoch)
.ok()?
{
return None;
}
let mut proposer_reward_numerator = 0;
let base_reward = state.get_base_reward(index as usize).ok()?;
for (flag_index, weight) in PARTICIPATION_FLAG_WEIGHTS.iter().enumerate() {
if att_participation_flags.contains(&flag_index) {
proposer_reward_numerator += base_reward.checked_mul(*weight)?;
}
}
let proposer_reward = proposer_reward_numerator
.checked_div(WEIGHT_DENOMINATOR.checked_mul(spec.proposer_reward_quotient)?)?;
Some((index, proposer_reward)).filter(|_| proposer_reward != 0)
})
.collect();
Some(Self {
att,
fresh_validators_rewards,
})
}
}
impl<'a, E: EthSpec> MaxCover for AttMaxCover<'a, E> {
type Object = Attestation<E>;
type Intermediate = CompactAttestationRef<'a, E>;
type Set = HashMap<u64, u64>;
fn intermediate(&self) -> &CompactAttestationRef<'a, E> {
&self.att
}
fn convert_to_object(att_ref: &CompactAttestationRef<'a, E>) -> Attestation<E> {
att_ref.clone_as_attestation()
}
fn covering_set(&self) -> &HashMap<u64, u64> {
&self.fresh_validators_rewards
}
/// Sneaky: we keep all the attestations together in one bucket, even though
/// their aggregation bitfields refer to different committees. In order to avoid
/// confusing committees when updating covering sets, we update only those attestations
/// whose slot and index match the attestation being included in the solution, by the logic
/// that a slot and index uniquely identify a committee.
///
/// We completely remove any validator covered by another attestation. This is close to optimal
/// because including two attestations on chain to satisfy different participation bits is
/// impossible without the validator double voting. I.e. it is only suboptimal in the presence
/// of slashable voting, which is rare.
///
/// Post-Electra this optimisation is still OK. The `self.att.data.index` will always be 0 for
/// all Electra attestations, so when a new attestation is added to the solution, we will
/// remove its validators from all attestations at the same slot. It may happen that the
/// included attestation and the attestation being updated have no validators in common, in
/// which case the `retain` will be a no-op. We could consider optimising this in future by only
/// executing the `retain` when the `committee_bits` of the two attestations intersect.
fn update_covering_set(
&mut self,
best_att: &CompactAttestationRef<'a, E>,
covered_validators: &HashMap<u64, u64>,
) {
if self.att.data.slot == best_att.data.slot && self.att.data.index == best_att.data.index {
self.fresh_validators_rewards
.retain(|k, _| !covered_validators.contains_key(k))
}
}
fn score(&self) -> usize {
self.fresh_validators_rewards.values().sum::<u64>() as usize
}
}
/// Extract the validators for which `attestation` would be their earliest in the epoch.
///
/// The reward paid to a proposer for including an attestation is proportional to the number
/// of validators for which the included attestation is their first in the epoch. The attestation
/// is judged against the state's `current_epoch_attestations` or `previous_epoch_attestations`
/// depending on when it was created, and all those validators who have already attested are
/// removed from the `aggregation_bits` before returning it.
///
/// This isn't optimal, but with the Altair fork this code is obsolete and not worth upgrading.
pub fn earliest_attestation_validators<E: EthSpec>(
attestation: &CompactAttestationRef<E>,
state: &BeaconState<E>,
base_state: &BeaconStateBase<E>,
) -> BitList<E::MaxValidatorsPerCommittee> {
// Bitfield of validators whose attestations are new/fresh.
let mut new_validators = match attestation.indexed {
CompactIndexedAttestation::Base(indexed_att) => indexed_att.aggregation_bits.clone(),
// This code path is obsolete post altair fork, so we just return an empty bitlist here.
CompactIndexedAttestation::Electra(_) => return BitList::with_capacity(0).unwrap(),
};
let state_attestations = if attestation.checkpoint.target_epoch == state.current_epoch() {
&base_state.current_epoch_attestations
} else if attestation.checkpoint.target_epoch == state.previous_epoch() {
&base_state.previous_epoch_attestations
} else {
return BitList::with_capacity(0).unwrap();
};
state_attestations
.iter()
// In a single epoch, an attester should only be attesting for one slot and index.
.filter(|existing_attestation| {
existing_attestation.data.slot == attestation.data.slot
&& existing_attestation.data.index == attestation.data.index
})
.for_each(|existing_attestation| {
// Remove the validators who have signed the existing attestation (they are not new)
new_validators.difference_inplace(&existing_attestation.aggregation_bits);
});
new_validators
}