Electra: Remaining Consensus Data Structures (#5712)

* Attestation superstruct changes for EIP 7549 (#5644)

* update

* experiment

* superstruct changes

* revert

* superstruct changes

* fix tests

* indexed attestation

* indexed attestation superstruct

* updated TODOs

* `superstruct` the `AttesterSlashing` (#5636)

* `superstruct` Attester Fork Variants

* Push a little further

* Deal with Encode / Decode of AttesterSlashing

* not so sure about this..

* Stop Encode/Decode Bounds from Propagating Out

* Tons of Changes..

* More Conversions to AttestationRef

* Add AsReference trait (#15)

* Add AsReference trait

* Fix some snafus

* Got it Compiling! :D

* Got Tests Building

* Get beacon chain tests compiling

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'upstream/unstable' into electra_attestation_changes

* Make EF Tests Fork-Agnostic (#5713)

* Finish EF Test Fork Agnostic (#5714)

* Superstruct `AggregateAndProof` (#5715)

* Upgrade `superstruct` to `0.8.0`

* superstruct `AggregateAndProof`

* Merge remote-tracking branch 'sigp/unstable' into electra_attestation_changes

* cargo fmt

* Merge pull request #5726 from realbigsean/electra_attestation_changes

Merge unstable into Electra attestation changes

* EIP7549 `get_attestation_indices` (#5657)

* get attesting indices electra impl

* fmt

* get tests to pass

* fmt

* fix some beacon chain tests

* fmt

* fix slasher test

* fmt got me again

* fix more tests

* fix tests

* Some small changes (#5739)

* cargo fmt (#5740)

* Sketch op pool changes

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

* Get `electra_op_pool` up to date (#5756)

* fix get attesting indices (#5742)

* fix get attesting indices

* better errors

* fix compile

* only get committee index once

* Ef test fixes (#5753)

* attestation related ef test fixes

* delete commented out stuff

* Fix Aggregation Pool for Electra (#5754)

* Fix Aggregation Pool for Electra

* Remove Outdated Interface

* fix ssz (#5755)

---------

Co-authored-by: realbigsean <sean@sigmaprime.io>

* Revert "Get `electra_op_pool` up to date (#5756)" (#5757)

This reverts commit ab9e58aa3d.

* Merge branch 'electra_attestation_changes' of https://github.com/sigp/lighthouse into electra_op_pool

* Compute on chain aggregate impl (#5752)

* add compute_on_chain_agg impl to op pool changes

* fmt

* get op pool tests to pass

* update the naive agg pool interface (#5760)

* Fix bugs in cross-committee aggregation

* Add comment to max cover optimisation

* Fix assert

* Merge pull request #5749 from sigp/electra_op_pool

Optimise Electra op pool aggregation

* update committee offset

* Fix Electra Fork Choice Tests (#5764)

* Subscribe to the correct subnets for electra attestations (#5782)

* subscribe to the correct att subnets for electra

* subscribe to the correct att subnets for electra

* cargo fmt

* fix slashing handling

* Merge remote-tracking branch 'upstream/unstable'

* Send unagg attestation based on fork

* Publish all aggregates

* just one more check bro plz..

* Merge pull request #5832 from ethDreamer/electra_attestation_changes_merge_unstable

Merge `unstable` into `electra_attestation_changes`

* Merge pull request #5835 from realbigsean/fix-validator-logic

Fix validator logic

* Merge pull request #5816 from realbigsean/electra-attestation-slashing-handling

Electra slashing handling

* Electra attestation changes rm decode impl (#5856)

* Remove Crappy Decode impl for Attestation

* Remove Inefficient Attestation Decode impl

* Implement Schema Upgrade / Downgrade

* Update beacon_node/beacon_chain/src/schema_change/migration_schema_v20.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Fix failing attestation tests and misc electra attestation cleanup (#5810)

* - get attestation related beacon chain tests to pass
- observed attestations are now keyed off of data + committee index
- rename op pool attestationref to compactattestationref
- remove unwraps in agg pool and use options instead
- cherry pick some changes from ef-tests-electra

* cargo fmt

* fix failing test

* Revert dockerfile changes

* make committee_index return option

* function args shouldnt be a ref to attestation ref

* fmt

* fix dup imports

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* fix some todos (#5817)

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* add consolidations to merkle calc for inclusion proof

* Remove Duplicate KZG Commitment Merkle Proof Code (#5874)

* Remove Duplicate KZG Commitment Merkle Proof Code

* s/tree_lists/fields/

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* fix compile

* Fix slasher tests (#5906)

* Fix electra tests

* Add electra attestations to double vote tests

* Update superstruct to 0.8

* Merge remote-tracking branch 'origin/unstable' into electra_attestation_changes

* Small cleanup in slasher tests

* Clean up Electra observed aggregates (#5929)

* Use consistent key in observed_attestations

* Remove unwraps from observed aggregates

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* De-dup attestation constructor logic

* Remove unwraps in Attestation construction

* Dedup match_attestation_data

* Remove outdated TODO

* Use ForkName Ord in fork-choice tests

* Use ForkName Ord in BeaconBlockBody

* Make to_electra not fallible

* Remove TestRandom impl for IndexedAttestation

* Remove IndexedAttestation faulty Decode impl

* Drop TestRandom impl

* Add PendingAttestationInElectra

* Indexed att on disk (#35)

* indexed att on disk

* fix lints

* Update slasher/src/migrate.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* add electra fork enabled fn to ForkName impl (#36)

* add electra fork enabled fn to ForkName impl

* remove inadvertent file

* Update common/eth2/src/types.rs

Co-authored-by: ethDreamer <37123614+ethDreamer@users.noreply.github.com>

* Dedup attestation constructor logic in attester cache

* Use if let Ok for committee_bits

* Dedup Attestation constructor code

* Diff reduction in tests

* Fix beacon_chain tests

* Diff reduction

* Use Ord for ForkName in pubsub

* Resolve into_attestation_and_indices todo

* Remove stale TODO

* Fix beacon_chain tests

* Test spec invariant

* Use electra_enabled in pubsub

* Remove get_indexed_attestation_from_signed_aggregate

* Use ok_or instead of if let else

* committees are sorted

* remove dup method `get_indexed_attestation_from_committees`

* Merge pull request #5940 from dapplion/electra_attestation_changes_lionreview

Electra attestations #5712 review

* update default persisted op pool deserialization

* ensure aggregate and proof uses serde untagged on ref

* Fork aware ssz static attestation tests

* Electra attestation changes from Lions review (#5971)

* dedup/cleanup and remove unneeded hashset use

* remove irrelevant TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra_attestation_changes

* Electra attestation changes sean review (#5972)

* instantiate empty bitlist in unreachable code

* clean up error conversion

* fork enabled bool cleanup

* remove a couple todos

* return bools instead of options in `aggregate` and use the result

* delete commented out code

* use map macros in simple transformations

* remove signers_disjoint_from

* get ef tests compiling

* get ef tests compiling

* update intentionally excluded files

* Avoid changing slasher schema for Electra

* Delete slasher schema v4

* Fix clippy

* Fix compilation of beacon_chain tests

* Update database.rs

* Add electra lightclient types

* Update slasher/src/database.rs

* fix imports

* Merge pull request #5980 from dapplion/electra-lightclient

Add electra lightclient types

* Merge pull request #5975 from michaelsproul/electra-slasher-no-migration

Avoid changing slasher schema for Electra

* Update beacon_node/beacon_chain/src/attestation_verification.rs

* Update beacon_node/beacon_chain/src/attestation_verification.rs
This commit is contained in:
ethDreamer
2024-06-24 16:08:07 -05:00
committed by GitHub
parent 758b58c9e9
commit c52c598f69
118 changed files with 5076 additions and 1741 deletions

View File

@@ -29,6 +29,7 @@ tree_hash = { workspace = true }
tree_hash_derive = { workspace = true }
types = { workspace = true }
strum = { workspace = true }
ssz_types = { workspace = true }
# MDBX is pinned at the last version with Windows and macOS support.
mdbx = { package = "libmdbx", git = "https://github.com/sigp/libmdbx-rs", tag = "v0.1.4", optional = true }

View File

@@ -226,12 +226,12 @@ impl TargetArrayChunk for MinTargetChunk {
) -> Result<AttesterSlashingStatus<E>, Error> {
let min_target =
self.chunk
.get_target(validator_index, attestation.data.source.epoch, config)?;
if attestation.data.target.epoch > min_target {
.get_target(validator_index, attestation.data().source.epoch, config)?;
if attestation.data().target.epoch > min_target {
let existing_attestation =
db.get_attestation_for_validator(txn, validator_index, min_target)?;
if attestation.data.source.epoch < existing_attestation.data.source.epoch {
if attestation.data().source.epoch < existing_attestation.data().source.epoch {
Ok(AttesterSlashingStatus::SurroundsExisting(Box::new(
existing_attestation,
)))
@@ -329,12 +329,12 @@ impl TargetArrayChunk for MaxTargetChunk {
) -> Result<AttesterSlashingStatus<E>, Error> {
let max_target =
self.chunk
.get_target(validator_index, attestation.data.source.epoch, config)?;
if attestation.data.target.epoch < max_target {
.get_target(validator_index, attestation.data().source.epoch, config)?;
if attestation.data().target.epoch < max_target {
let existing_attestation =
db.get_attestation_for_validator(txn, validator_index, max_target)?;
if existing_attestation.data.source.epoch < attestation.data.source.epoch {
if existing_attestation.data().source.epoch < attestation.data().source.epoch {
Ok(AttesterSlashingStatus::SurroundedByExisting(Box::new(
existing_attestation,
)))
@@ -428,7 +428,7 @@ pub fn apply_attestation_for_validator<E: EthSpec, T: TargetArrayChunk>(
current_epoch: Epoch,
config: &Config,
) -> Result<AttesterSlashingStatus<E>, Error> {
let mut chunk_index = config.chunk_index(attestation.data.source.epoch);
let mut chunk_index = config.chunk_index(attestation.data().source.epoch);
let mut current_chunk = get_chunk_for_update(
db,
txn,
@@ -446,7 +446,7 @@ pub fn apply_attestation_for_validator<E: EthSpec, T: TargetArrayChunk>(
}
let Some(mut start_epoch) =
T::first_start_epoch(attestation.data.source.epoch, current_epoch, config)
T::first_start_epoch(attestation.data().source.epoch, current_epoch, config)
else {
return Ok(slashing_status);
};
@@ -465,7 +465,7 @@ pub fn apply_attestation_for_validator<E: EthSpec, T: TargetArrayChunk>(
chunk_index,
validator_index,
start_epoch,
attestation.data.target.epoch,
attestation.data().target.epoch,
current_epoch,
config,
)?;
@@ -492,7 +492,7 @@ pub fn update<E: EthSpec>(
let mut chunk_attestations = BTreeMap::new();
for attestation in batch {
chunk_attestations
.entry(config.chunk_index(attestation.indexed.data.source.epoch))
.entry(config.chunk_index(attestation.indexed.data().source.epoch))
.or_insert_with(Vec::new)
.push(attestation);
}

View File

@@ -47,7 +47,8 @@ impl<E: EthSpec> AttestationBatch<E> {
self.attestations.push(Arc::downgrade(&indexed_record));
let attestation_data_hash = indexed_record.record.attestation_data_hash;
for &validator_index in &indexed_record.indexed.attesting_indices {
for &validator_index in indexed_record.indexed.attesting_indices_iter() {
self.attesters
.entry((validator_index, attestation_data_hash))
.and_modify(|existing_entry| {
@@ -56,8 +57,8 @@ impl<E: EthSpec> AttestationBatch<E> {
// smaller indexed attestation. Single-bit attestations will usually be removed
// completely by this process, and aggregates will only be retained if they
// are not redundant with respect to a larger aggregate seen in the same batch.
if existing_entry.indexed.attesting_indices.len()
< indexed_record.indexed.attesting_indices.len()
if existing_entry.indexed.attesting_indices_len()
< indexed_record.indexed.attesting_indices_len()
{
*existing_entry = indexed_record.clone();
}

View File

@@ -80,18 +80,20 @@ impl<E: EthSpec> IndexedAttesterRecord<E> {
#[derive(Debug, Clone, Encode, Decode, TreeHash)]
struct IndexedAttestationHeader<E: EthSpec> {
pub attesting_indices: VariableList<u64, E::MaxValidatorsPerCommittee>,
pub attesting_indices: VariableList<u64, E::MaxValidatorsPerSlot>,
pub data_root: Hash256,
pub signature: AggregateSignature,
}
impl<E: EthSpec> From<IndexedAttestation<E>> for AttesterRecord {
fn from(indexed_attestation: IndexedAttestation<E>) -> AttesterRecord {
let attestation_data_hash = indexed_attestation.data.tree_hash_root();
let attestation_data_hash = indexed_attestation.data().tree_hash_root();
let attesting_indices =
VariableList::new(indexed_attestation.attesting_indices_to_vec()).unwrap_or_default();
let header = IndexedAttestationHeader::<E> {
attesting_indices: indexed_attestation.attesting_indices,
attesting_indices,
data_root: attestation_data_hash,
signature: indexed_attestation.signature,
signature: indexed_attestation.signature().clone(),
};
let indexed_attestation_hash = header.tree_hash_root();
AttesterRecord {
@@ -104,15 +106,15 @@ impl<E: EthSpec> From<IndexedAttestation<E>> for AttesterRecord {
#[cfg(test)]
mod test {
use super::*;
use crate::test_utils::indexed_att;
use crate::test_utils::indexed_att_electra;
// Check correctness of fast hashing
#[test]
fn fast_hash() {
let data = vec![
indexed_att(vec![], 0, 0, 0),
indexed_att(vec![1, 2, 3], 12, 14, 1),
indexed_att(vec![4], 0, 5, u64::MAX),
indexed_att_electra(vec![], 0, 0, 0),
indexed_att_electra(vec![1, 2, 3], 12, 14, 1),
indexed_att_electra(vec![4], 0, 5, u64::MAX),
];
for att in data {
assert_eq!(

View File

@@ -166,8 +166,7 @@ impl Config {
validator_chunk_index: usize,
) -> impl Iterator<Item = u64> + 'a {
attestation
.attesting_indices
.iter()
.attesting_indices_iter()
.filter(move |v| self.validator_chunk_index(**v) == validator_chunk_index)
.copied()
}

View File

@@ -13,12 +13,15 @@ use parking_lot::Mutex;
use serde::de::DeserializeOwned;
use slog::{info, Logger};
use ssz::{Decode, Encode};
use ssz_derive::{Decode, Encode};
use std::borrow::{Borrow, Cow};
use std::marker::PhantomData;
use std::sync::Arc;
use tree_hash::TreeHash;
use types::{
Epoch, EthSpec, Hash256, IndexedAttestation, ProposerSlashing, SignedBeaconBlockHeader, Slot,
AggregateSignature, AttestationData, ChainSpec, Epoch, EthSpec, Hash256, IndexedAttestation,
IndexedAttestationBase, IndexedAttestationElectra, ProposerSlashing, SignedBeaconBlockHeader,
Slot, VariableList,
};
/// Current database schema version, to check compatibility of on-disk DB with software.
@@ -69,6 +72,7 @@ pub struct SlasherDB<E: EthSpec> {
/// LRU cache mapping indexed attestation IDs to their attestation data roots.
attestation_root_cache: Mutex<LruCache<IndexedAttestationId, Hash256>>,
pub(crate) config: Arc<Config>,
pub(crate) spec: Arc<ChainSpec>,
_phantom: PhantomData<E>,
}
@@ -235,6 +239,43 @@ impl AsRef<[u8]> for IndexedAttestationId {
}
}
/// Indexed attestation that abstracts over Phase0 and Electra variants by using a plain `Vec` for
/// the attesting indices.
///
/// This allows us to avoid rewriting the entire indexed attestation database at Electra, which
/// saves a lot of execution time. The bytes that it encodes to are the same as the bytes that a
/// regular IndexedAttestation encodes to, because SSZ doesn't care about the length-bound.
#[derive(Debug, PartialEq, Decode, Encode)]
pub struct IndexedAttestationOnDisk {
attesting_indices: Vec<u64>,
data: AttestationData,
signature: AggregateSignature,
}
impl IndexedAttestationOnDisk {
fn into_indexed_attestation<E: EthSpec>(
self,
spec: &ChainSpec,
) -> Result<IndexedAttestation<E>, Error> {
let fork_at_target_epoch = spec.fork_name_at_epoch(self.data.target.epoch);
if fork_at_target_epoch.electra_enabled() {
let attesting_indices = VariableList::new(self.attesting_indices)?;
Ok(IndexedAttestation::Electra(IndexedAttestationElectra {
attesting_indices,
data: self.data,
signature: self.signature,
}))
} else {
let attesting_indices = VariableList::new(self.attesting_indices)?;
Ok(IndexedAttestation::Base(IndexedAttestationBase {
attesting_indices,
data: self.data,
signature: self.signature,
}))
}
}
}
/// Bincode deserialization specialised to `Cow<[u8]>`.
fn bincode_deserialize<T: DeserializeOwned>(bytes: Cow<[u8]>) -> Result<T, Error> {
Ok(bincode::deserialize(bytes.borrow())?)
@@ -245,7 +286,7 @@ fn ssz_decode<T: Decode>(bytes: Cow<[u8]>) -> Result<T, Error> {
}
impl<E: EthSpec> SlasherDB<E> {
pub fn open(config: Arc<Config>, log: Logger) -> Result<Self, Error> {
pub fn open(config: Arc<Config>, spec: Arc<ChainSpec>, log: Logger) -> Result<Self, Error> {
info!(log, "Opening slasher database"; "backend" => %config.backend);
std::fs::create_dir_all(&config.database_path)?;
@@ -268,6 +309,7 @@ impl<E: EthSpec> SlasherDB<E> {
databases,
attestation_root_cache,
config,
spec,
_phantom: PhantomData,
};
@@ -438,7 +480,7 @@ impl<E: EthSpec> SlasherDB<E> {
) -> Result<u64, Error> {
// Look-up ID by hash.
let id_key = IndexedAttestationIdKey::new(
indexed_attestation.data.target.epoch,
indexed_attestation.data().target.epoch,
indexed_attestation_hash,
);
@@ -457,6 +499,7 @@ impl<E: EthSpec> SlasherDB<E> {
};
let attestation_key = IndexedAttestationId::new(indexed_att_id);
// IndexedAttestationOnDisk and IndexedAttestation have compatible encodings.
let data = indexed_attestation.as_ssz_bytes();
cursor.put(attestation_key.as_ref(), &data)?;
@@ -481,7 +524,8 @@ impl<E: EthSpec> SlasherDB<E> {
.ok_or(Error::MissingIndexedAttestation {
id: indexed_attestation_id.as_u64(),
})?;
ssz_decode(bytes)
let indexed_attestation_on_disk: IndexedAttestationOnDisk = ssz_decode(bytes)?;
indexed_attestation_on_disk.into_indexed_attestation(&self.spec)
}
fn get_attestation_data_root(
@@ -500,7 +544,7 @@ impl<E: EthSpec> SlasherDB<E> {
// Otherwise, load the indexed attestation, compute the root and cache it.
let indexed_attestation = self.get_indexed_attestation(txn, indexed_id)?;
let attestation_data_root = indexed_attestation.data.tree_hash_root();
let attestation_data_root = indexed_attestation.data().tree_hash_root();
cache.put(indexed_id, attestation_data_root);
@@ -536,7 +580,7 @@ impl<E: EthSpec> SlasherDB<E> {
indexed_attestation_id: IndexedAttestationId,
) -> Result<AttesterSlashingStatus<E>, Error> {
// See if there's an existing attestation for this attester.
let target_epoch = attestation.data.target.epoch;
let target_epoch = attestation.data().target.epoch;
let prev_max_target = self.get_attester_max_target(validator_index, txn)?;
@@ -771,3 +815,93 @@ impl<E: EthSpec> SlasherDB<E> {
Ok(())
}
}
#[cfg(test)]
mod test {
use super::*;
use types::{Checkpoint, ForkName, MainnetEthSpec, Unsigned};
type E = MainnetEthSpec;
fn indexed_attestation_on_disk_roundtrip_test(
spec: &ChainSpec,
make_attestation: fn(
Vec<u64>,
AttestationData,
AggregateSignature,
) -> IndexedAttestation<E>,
committee_len: u64,
) {
let attestation_data = AttestationData {
slot: Slot::new(1000),
index: 0,
beacon_block_root: Hash256::repeat_byte(0xaa),
source: Checkpoint {
epoch: Epoch::new(0),
root: Hash256::repeat_byte(0xbb),
},
target: Checkpoint {
epoch: Epoch::new(31),
root: Hash256::repeat_byte(0xcc),
},
};
let attesting_indices = (0..committee_len).collect::<Vec<_>>();
let signature = AggregateSignature::infinity();
let fork_attestation = make_attestation(
attesting_indices.clone(),
attestation_data.clone(),
signature.clone(),
);
let on_disk = IndexedAttestationOnDisk {
attesting_indices,
data: attestation_data,
signature,
};
let encoded = on_disk.as_ssz_bytes();
assert_eq!(encoded, fork_attestation.as_ssz_bytes());
let decoded_on_disk = IndexedAttestationOnDisk::from_ssz_bytes(&encoded).unwrap();
assert_eq!(decoded_on_disk, on_disk);
let decoded = on_disk.into_indexed_attestation(spec).unwrap();
assert_eq!(decoded, fork_attestation);
}
/// Check that `IndexedAttestationOnDisk` and `IndexedAttestation` have compatible encodings.
#[test]
fn indexed_attestation_on_disk_roundtrip_base() {
let spec = ForkName::Base.make_genesis_spec(E::default_spec());
let make_attestation = |attesting_indices, data, signature| {
IndexedAttestation::<E>::Base(IndexedAttestationBase {
attesting_indices: VariableList::new(attesting_indices).unwrap(),
data,
signature,
})
};
indexed_attestation_on_disk_roundtrip_test(
&spec,
make_attestation,
<E as EthSpec>::MaxValidatorsPerCommittee::to_u64(),
)
}
#[test]
fn indexed_attestation_on_disk_roundtrip_electra() {
let spec = ForkName::Electra.make_genesis_spec(E::default_spec());
let make_attestation = |attesting_indices, data, signature| {
IndexedAttestation::<E>::Electra(IndexedAttestationElectra {
attesting_indices: VariableList::new(attesting_indices).unwrap(),
data,
signature,
})
};
indexed_attestation_on_disk_roundtrip_test(
&spec,
make_attestation,
<E as EthSpec>::MaxValidatorsPerSlot::to_u64(),
)
}
}

View File

@@ -13,6 +13,7 @@ pub enum Error {
DatabaseIOError(io::Error),
DatabasePermissionsError(filesystem::Error),
SszDecodeError(ssz::DecodeError),
SszTypesError(ssz_types::Error),
BincodeError(bincode::Error),
ArithError(safe_arith::ArithError),
ChunkIndexOutOfBounds(usize),
@@ -100,6 +101,12 @@ impl From<ssz::DecodeError> for Error {
}
}
impl From<ssz_types::Error> for Error {
fn from(e: ssz_types::Error) -> Self {
Error::SszTypesError(e)
}
}
impl From<bincode::Error> for Error {
fn from(e: bincode::Error) -> Self {
Error::BincodeError(e)

View File

@@ -28,7 +28,8 @@ pub use database::{
};
pub use error::Error;
use types::{AttesterSlashing, EthSpec, IndexedAttestation, ProposerSlashing};
use types::{AttesterSlashing, AttesterSlashingBase, AttesterSlashingElectra};
use types::{EthSpec, IndexedAttestation, ProposerSlashing};
#[derive(Debug, PartialEq)]
pub enum AttesterSlashingStatus<E: EthSpec> {
@@ -59,14 +60,34 @@ impl<E: EthSpec> AttesterSlashingStatus<E> {
match self {
NotSlashable => None,
AlreadyDoubleVoted => None,
DoubleVote(existing) | SurroundedByExisting(existing) => Some(AttesterSlashing {
attestation_1: *existing,
attestation_2: new_attestation.clone(),
}),
SurroundsExisting(existing) => Some(AttesterSlashing {
attestation_1: new_attestation.clone(),
attestation_2: *existing,
}),
DoubleVote(existing) | SurroundedByExisting(existing) => {
match (&*existing, new_attestation) {
(IndexedAttestation::Base(existing_att), IndexedAttestation::Base(new)) => {
Some(AttesterSlashing::Base(AttesterSlashingBase {
attestation_1: existing_att.clone(),
attestation_2: new.clone(),
}))
}
// A slashing involving an electra attestation type must return an `AttesterSlashingElectra` type
(_, _) => Some(AttesterSlashing::Electra(AttesterSlashingElectra {
attestation_1: existing.clone().to_electra(),
attestation_2: new_attestation.clone().to_electra(),
})),
}
}
SurroundsExisting(existing) => match (&*existing, new_attestation) {
(IndexedAttestation::Base(existing_att), IndexedAttestation::Base(new)) => {
Some(AttesterSlashing::Base(AttesterSlashingBase {
attestation_1: new.clone(),
attestation_2: existing_att.clone(),
}))
}
// A slashing involving an electra attestation type must return an `AttesterSlashingElectra` type
(_, _) => Some(AttesterSlashing::Electra(AttesterSlashingElectra {
attestation_1: new_attestation.clone().to_electra(),
attestation_2: existing.clone().to_electra(),
})),
},
}
}
}

View File

@@ -13,7 +13,8 @@ use slog::{debug, error, info, Logger};
use std::collections::HashSet;
use std::sync::Arc;
use types::{
AttesterSlashing, Epoch, EthSpec, IndexedAttestation, ProposerSlashing, SignedBeaconBlockHeader,
AttesterSlashing, ChainSpec, Epoch, EthSpec, IndexedAttestation, ProposerSlashing,
SignedBeaconBlockHeader,
};
#[derive(Debug)]
@@ -28,10 +29,10 @@ pub struct Slasher<E: EthSpec> {
}
impl<E: EthSpec> Slasher<E> {
pub fn open(config: Config, log: Logger) -> Result<Self, Error> {
pub fn open(config: Config, spec: Arc<ChainSpec>, log: Logger) -> Result<Self, Error> {
config.validate()?;
let config = Arc::new(config);
let db = SlasherDB::open(config.clone(), log.clone())?;
let db = SlasherDB::open(config.clone(), spec, log.clone())?;
let attester_slashings = Mutex::new(HashSet::new());
let proposer_slashings = Mutex::new(HashSet::new());
let attestation_queue = AttestationQueue::default();
@@ -299,7 +300,7 @@ impl<E: EthSpec> Slasher<E> {
self.log,
"Found double-vote slashing";
"validator_index" => validator_index,
"epoch" => slashing.attestation_1.data.target.epoch,
"epoch" => slashing.attestation_1().data().target.epoch,
);
slashings.insert(slashing);
}
@@ -325,8 +326,8 @@ impl<E: EthSpec> Slasher<E> {
for indexed_record in batch {
let attestation = &indexed_record.indexed;
let target_epoch = attestation.data.target.epoch;
let source_epoch = attestation.data.source.epoch;
let target_epoch = attestation.data().target.epoch;
let source_epoch = attestation.data().source.epoch;
if source_epoch > target_epoch
|| source_epoch + self.config.history_length as u64 <= current_epoch

View File

@@ -1,18 +1,21 @@
use std::collections::HashSet;
use std::sync::Arc;
use types::{
AggregateSignature, AttestationData, AttesterSlashing, BeaconBlockHeader, Checkpoint, Epoch,
Hash256, IndexedAttestation, MainnetEthSpec, Signature, SignedBeaconBlockHeader, Slot,
indexed_attestation::{IndexedAttestationBase, IndexedAttestationElectra},
AggregateSignature, AttestationData, AttesterSlashing, AttesterSlashingBase,
AttesterSlashingElectra, BeaconBlockHeader, ChainSpec, Checkpoint, Epoch, EthSpec, Hash256,
IndexedAttestation, MainnetEthSpec, Signature, SignedBeaconBlockHeader, Slot,
};
pub type E = MainnetEthSpec;
pub fn indexed_att(
pub fn indexed_att_electra(
attesting_indices: impl AsRef<[u64]>,
source_epoch: u64,
target_epoch: u64,
target_root: u64,
) -> IndexedAttestation<E> {
IndexedAttestation {
IndexedAttestation::Electra(IndexedAttestationElectra {
attesting_indices: attesting_indices.as_ref().to_vec().into(),
data: AttestationData {
slot: Slot::new(0),
@@ -28,16 +31,50 @@ pub fn indexed_att(
},
},
signature: AggregateSignature::empty(),
}
})
}
pub fn indexed_att(
attesting_indices: impl AsRef<[u64]>,
source_epoch: u64,
target_epoch: u64,
target_root: u64,
) -> IndexedAttestation<E> {
IndexedAttestation::Base(IndexedAttestationBase {
attesting_indices: attesting_indices.as_ref().to_vec().into(),
data: AttestationData {
slot: Slot::new(0),
index: 0,
beacon_block_root: Hash256::zero(),
source: Checkpoint {
epoch: Epoch::new(source_epoch),
root: Hash256::from_low_u64_be(0),
},
target: Checkpoint {
epoch: Epoch::new(target_epoch),
root: Hash256::from_low_u64_be(target_root),
},
},
signature: AggregateSignature::empty(),
})
}
pub fn att_slashing(
attestation_1: &IndexedAttestation<E>,
attestation_2: &IndexedAttestation<E>,
) -> AttesterSlashing<E> {
AttesterSlashing {
attestation_1: attestation_1.clone(),
attestation_2: attestation_2.clone(),
match (attestation_1, attestation_2) {
(IndexedAttestation::Base(att1), IndexedAttestation::Base(att2)) => {
AttesterSlashing::Base(AttesterSlashingBase {
attestation_1: att1.clone(),
attestation_2: att2.clone(),
})
}
// A slashing involving an electra attestation type must return an electra AttesterSlashing type
(_, _) => AttesterSlashing::Electra(AttesterSlashingElectra {
attestation_1: attestation_1.clone().to_electra(),
attestation_2: attestation_2.clone().to_electra(),
}),
}
}
@@ -59,14 +96,16 @@ pub fn slashed_validators_from_slashings(slashings: &HashSet<AttesterSlashing<E>
slashings
.iter()
.flat_map(|slashing| {
let att1 = &slashing.attestation_1;
let att2 = &slashing.attestation_2;
let att1 = slashing.attestation_1();
let att2 = slashing.attestation_2();
assert!(
att1.is_double_vote(att2) || att1.is_surround_vote(att2),
"invalid slashing: {:#?}",
slashing
);
hashset_intersection(&att1.attesting_indices, &att2.attesting_indices)
let attesting_indices_1 = att1.attesting_indices_to_vec();
let attesting_indices_2 = att2.attesting_indices_to_vec();
hashset_intersection(&attesting_indices_1, &attesting_indices_2)
})
.collect()
}
@@ -83,9 +122,11 @@ pub fn slashed_validators_from_attestations(
}
if att1.is_double_vote(att2) || att1.is_surround_vote(att2) {
let attesting_indices_1 = att1.attesting_indices_to_vec();
let attesting_indices_2 = att2.attesting_indices_to_vec();
slashed_validators.extend(hashset_intersection(
&att1.attesting_indices,
&att2.attesting_indices,
&attesting_indices_1,
&attesting_indices_2,
));
}
}
@@ -105,3 +146,7 @@ pub fn block(slot: u64, proposer_index: u64, block_root: u64) -> SignedBeaconBlo
signature: Signature::empty(),
}
}
pub fn chain_spec() -> Arc<ChainSpec> {
Arc::new(E::default_spec())
}

View File

@@ -5,7 +5,10 @@ use maplit::hashset;
use rayon::prelude::*;
use slasher::{
config::DEFAULT_CHUNK_SIZE,
test_utils::{att_slashing, indexed_att, slashed_validators_from_slashings, E},
test_utils::{
att_slashing, chain_spec, indexed_att, indexed_att_electra,
slashed_validators_from_slashings, E,
},
Config, Slasher,
};
use std::collections::HashSet;
@@ -15,23 +18,35 @@ use types::{AttesterSlashing, Epoch, IndexedAttestation};
#[test]
fn double_vote_single_val() {
let v = vec![99];
let att1 = indexed_att(&v, 0, 1, 0);
let att2 = indexed_att(&v, 0, 1, 1);
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 1);
slasher_test_indiv(&attestations, &slashings, 1000);
for (att1, att2) in [
(indexed_att(&v, 0, 1, 0), indexed_att(&v, 0, 1, 1)),
(
indexed_att_electra(&v, 0, 1, 0),
indexed_att_electra(&v, 0, 1, 1),
),
] {
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 1);
slasher_test_indiv(&attestations, &slashings, 1000);
}
}
#[test]
fn double_vote_multi_vals() {
let v = vec![0, 1, 2];
let att1 = indexed_att(&v, 0, 1, 0);
let att2 = indexed_att(&v, 0, 1, 1);
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 1);
slasher_test_indiv(&attestations, &slashings, 1000);
for (att1, att2) in [
(indexed_att(&v, 0, 1, 0), indexed_att(&v, 0, 1, 1)),
(
indexed_att_electra(&v, 0, 1, 0),
indexed_att_electra(&v, 0, 1, 1),
),
] {
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 1);
slasher_test_indiv(&attestations, &slashings, 1000);
}
}
// A subset of validators double vote.
@@ -39,12 +54,18 @@ fn double_vote_multi_vals() {
fn double_vote_some_vals() {
let v1 = vec![0, 1, 2, 3, 4, 5, 6];
let v2 = vec![0, 2, 4, 6];
let att1 = indexed_att(v1, 0, 1, 0);
let att2 = indexed_att(v2, 0, 1, 1);
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 1);
slasher_test_indiv(&attestations, &slashings, 1000);
for (att1, att2) in [
(indexed_att(&v1, 0, 1, 0), indexed_att(&v2, 0, 1, 1)),
(
indexed_att_electra(&v1, 0, 1, 0),
indexed_att_electra(&v2, 0, 1, 1),
),
] {
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 1);
slasher_test_indiv(&attestations, &slashings, 1000);
}
}
// A subset of validators double vote, others vote twice for the same thing.
@@ -53,13 +74,23 @@ fn double_vote_some_vals_repeat() {
let v1 = vec![0, 1, 2, 3, 4, 5, 6];
let v2 = vec![0, 2, 4, 6];
let v3 = vec![1, 3, 5];
let att1 = indexed_att(v1, 0, 1, 0);
let att2 = indexed_att(v2, 0, 1, 1);
let att3 = indexed_att(v3, 0, 1, 0);
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2, att3];
slasher_test_indiv(&attestations, &slashings, 1);
slasher_test_indiv(&attestations, &slashings, 1000);
for (att1, att2, att3) in [
(
indexed_att(&v1, 0, 1, 0),
indexed_att(&v2, 0, 1, 1),
indexed_att(&v3, 0, 1, 0),
),
(
indexed_att_electra(&v1, 0, 1, 0),
indexed_att_electra(&v2, 0, 1, 1),
indexed_att_electra(&v3, 0, 1, 0),
),
] {
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2, att3];
slasher_test_indiv(&attestations, &slashings, 1);
slasher_test_indiv(&attestations, &slashings, 1000);
}
}
// Nobody double votes, nobody gets slashed.
@@ -67,11 +98,17 @@ fn double_vote_some_vals_repeat() {
fn no_double_vote_same_target() {
let v1 = vec![0, 1, 2, 3, 4, 5, 6];
let v2 = vec![0, 1, 2, 3, 4, 5, 7, 8];
let att1 = indexed_att(v1, 0, 1, 0);
let att2 = indexed_att(v2, 0, 1, 0);
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &hashset! {}, 1);
slasher_test_indiv(&attestations, &hashset! {}, 1000);
for (att1, att2) in [
(indexed_att(&v1, 0, 1, 0), indexed_att(&v2, 0, 1, 0)),
(
indexed_att_electra(&v1, 0, 1, 0),
indexed_att_electra(&v2, 0, 1, 0),
),
] {
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &hashset! {}, 1);
slasher_test_indiv(&attestations, &hashset! {}, 1000);
}
}
// Two groups votes for different things, no slashings.
@@ -79,73 +116,133 @@ fn no_double_vote_same_target() {
fn no_double_vote_distinct_vals() {
let v1 = vec![0, 1, 2, 3];
let v2 = vec![4, 5, 6, 7];
let att1 = indexed_att(v1, 0, 1, 0);
let att2 = indexed_att(v2, 0, 1, 1);
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &hashset! {}, 1);
slasher_test_indiv(&attestations, &hashset! {}, 1000);
for (att1, att2) in [
(indexed_att(&v1, 0, 1, 0), indexed_att(&v2, 0, 1, 0)),
(
indexed_att_electra(&v1, 0, 1, 0),
indexed_att_electra(&v2, 0, 1, 1),
),
] {
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &hashset! {}, 1);
slasher_test_indiv(&attestations, &hashset! {}, 1000);
}
}
#[test]
fn no_double_vote_repeated() {
let v = vec![0, 1, 2, 3, 4];
let att1 = indexed_att(v, 0, 1, 0);
let att2 = att1.clone();
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &hashset! {}, 1);
slasher_test_batch(&attestations, &hashset! {}, 1);
parallel_slasher_test(&attestations, hashset! {}, 1);
for att1 in [indexed_att(&v, 0, 1, 0), indexed_att_electra(&v, 0, 1, 0)] {
let att2 = att1.clone();
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &hashset! {}, 1);
slasher_test_batch(&attestations, &hashset! {}, 1);
parallel_slasher_test(&attestations, hashset! {}, 1);
}
}
#[test]
fn surrounds_existing_single_val_single_chunk() {
let v = vec![0];
let att1 = indexed_att(&v, 1, 2, 0);
let att2 = indexed_att(&v, 0, 3, 0);
let slashings = hashset![att_slashing(&att2, &att1)];
slasher_test_indiv(&[att1, att2], &slashings, 3);
for (att1, att2) in [
(indexed_att(&v, 1, 2, 0), indexed_att(&v, 0, 3, 0)),
(indexed_att(&v, 1, 2, 0), indexed_att_electra(&v, 0, 3, 0)),
(
indexed_att_electra(&v, 1, 2, 0),
indexed_att_electra(&v, 0, 3, 0),
),
] {
let slashings = hashset![att_slashing(&att2, &att1)];
slasher_test_indiv(&[att1, att2], &slashings, 3);
}
}
#[test]
fn surrounds_existing_multi_vals_single_chunk() {
let validators = vec![0, 16, 1024, 300_000, 300_001];
let att1 = indexed_att(validators.clone(), 1, 2, 0);
let att2 = indexed_att(validators, 0, 3, 0);
let slashings = hashset![att_slashing(&att2, &att1)];
slasher_test_indiv(&[att1, att2], &slashings, 3);
for (att1, att2) in [
(
indexed_att(&validators, 1, 2, 0),
indexed_att(&validators, 0, 3, 0),
),
(
indexed_att(&validators, 1, 2, 0),
indexed_att_electra(&validators, 0, 3, 0),
),
(
indexed_att_electra(&validators, 1, 2, 0),
indexed_att_electra(&validators, 0, 3, 0),
),
] {
let slashings = hashset![att_slashing(&att2, &att1)];
slasher_test_indiv(&[att1, att2], &slashings, 3);
}
}
#[test]
fn surrounds_existing_many_chunks() {
let v = vec![0];
let chunk_size = DEFAULT_CHUNK_SIZE as u64;
let att1 = indexed_att(&v, 3 * chunk_size, 3 * chunk_size + 1, 0);
let att2 = indexed_att(&v, 0, 3 * chunk_size + 2, 0);
let slashings = hashset![att_slashing(&att2, &att1)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 4 * chunk_size);
for (att1, att2) in [
(
indexed_att(&v, 3 * chunk_size, 3 * chunk_size + 1, 0),
indexed_att(&v, 0, 3 * chunk_size + 2, 0),
),
(
indexed_att(&v, 3 * chunk_size, 3 * chunk_size + 1, 0),
indexed_att_electra(&v, 0, 3 * chunk_size + 2, 0),
),
(
indexed_att_electra(&v, 3 * chunk_size, 3 * chunk_size + 1, 0),
indexed_att_electra(&v, 0, 3 * chunk_size + 2, 0),
),
] {
let slashings = hashset![att_slashing(&att2, &att1)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 4 * chunk_size);
}
}
#[test]
fn surrounded_by_single_val_single_chunk() {
let v = vec![0];
let att1 = indexed_att(&v, 0, 15, 0);
let att2 = indexed_att(&v, 1, 14, 0);
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 15);
for (att1, att2) in [
(indexed_att(&v, 0, 15, 0), indexed_att(&v, 1, 14, 0)),
(indexed_att(&v, 0, 15, 0), indexed_att_electra(&v, 1, 14, 0)),
(
indexed_att_electra(&v, 0, 15, 0),
indexed_att_electra(&v, 1, 14, 0),
),
] {
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 15);
}
}
#[test]
fn surrounded_by_single_val_multi_chunk() {
let v = vec![0];
let chunk_size = DEFAULT_CHUNK_SIZE as u64;
let att1 = indexed_att(&v, 0, 3 * chunk_size, 0);
let att2 = indexed_att(&v, chunk_size, chunk_size + 1, 0);
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 3 * chunk_size);
slasher_test_indiv(&attestations, &slashings, 4 * chunk_size);
for (att1, att2) in [
(
indexed_att(&v, 0, 3 * chunk_size, 0),
indexed_att(&v, chunk_size, chunk_size + 1, 0),
),
(
indexed_att(&v, 0, 3 * chunk_size, 0),
indexed_att_electra(&v, chunk_size, chunk_size + 1, 0),
),
(
indexed_att_electra(&v, 0, 3 * chunk_size, 0),
indexed_att_electra(&v, chunk_size, chunk_size + 1, 0),
),
] {
let slashings = hashset![att_slashing(&att1, &att2)];
let attestations = vec![att1, att2];
slasher_test_indiv(&attestations, &slashings, 3 * chunk_size);
slasher_test_indiv(&attestations, &slashings, 4 * chunk_size);
}
}
// Process each attestation individually, and confirm that the slashings produced are as expected.
@@ -174,7 +271,8 @@ fn slasher_test(
) {
let tempdir = tempdir().unwrap();
let config = Config::new(tempdir.path().into());
let slasher = Slasher::open(config, test_logger()).unwrap();
let spec = chain_spec();
let slasher = Slasher::open(config, spec, test_logger()).unwrap();
let current_epoch = Epoch::new(current_epoch);
for (i, attestation) in attestations.iter().enumerate() {
@@ -203,7 +301,8 @@ fn parallel_slasher_test(
) {
let tempdir = tempdir().unwrap();
let config = Config::new(tempdir.path().into());
let slasher = Slasher::open(config, test_logger()).unwrap();
let spec = chain_spec();
let slasher = Slasher::open(config, spec, test_logger()).unwrap();
let current_epoch = Epoch::new(current_epoch);
attestations

View File

@@ -2,7 +2,7 @@
use logging::test_logger;
use slasher::{
test_utils::{block as test_block, E},
test_utils::{block as test_block, chain_spec, E},
Config, Slasher,
};
use tempfile::tempdir;
@@ -12,7 +12,8 @@ use types::{Epoch, EthSpec};
fn empty_pruning() {
let tempdir = tempdir().unwrap();
let config = Config::new(tempdir.path().into());
let slasher = Slasher::<E>::open(config, test_logger()).unwrap();
let spec = chain_spec();
let slasher = Slasher::<E>::open(config, spec, test_logger()).unwrap();
slasher.prune_database(Epoch::new(0)).unwrap();
}
@@ -24,8 +25,9 @@ fn block_pruning() {
let mut config = Config::new(tempdir.path().into());
config.chunk_size = 2;
config.history_length = 2;
let spec = chain_spec();
let slasher = Slasher::<E>::open(config.clone(), test_logger()).unwrap();
let slasher = Slasher::<E>::open(config.clone(), spec, test_logger()).unwrap();
let current_epoch = Epoch::from(2 * config.history_length);
// Pruning the empty database should be safe.

View File

@@ -4,7 +4,7 @@ use logging::test_logger;
use rand::prelude::*;
use slasher::{
test_utils::{
block, indexed_att, slashed_validators_from_attestations,
block, chain_spec, indexed_att, slashed_validators_from_attestations,
slashed_validators_from_slashings, E,
},
Config, Slasher,
@@ -49,7 +49,9 @@ fn random_test(seed: u64, test_config: TestConfig) {
config.chunk_size = 1 << chunk_size_exponent;
config.history_length = 1 << rng.gen_range(chunk_size_exponent..chunk_size_exponent + 3);
let slasher = Slasher::<E>::open(config.clone(), test_logger()).unwrap();
let spec = chain_spec();
let slasher = Slasher::<E>::open(config.clone(), spec, test_logger()).unwrap();
let validators = (0..num_validators as u64).collect::<Vec<u64>>();

View File

@@ -1,7 +1,10 @@
#![cfg(any(feature = "mdbx", feature = "lmdb"))]
use logging::test_logger;
use slasher::{test_utils::indexed_att, Config, Slasher};
use slasher::{
test_utils::{chain_spec, indexed_att},
Config, Slasher,
};
use tempfile::tempdir;
use types::Epoch;
@@ -9,11 +12,12 @@ use types::Epoch;
fn attestation_pruning_empty_wrap_around() {
let tempdir = tempdir().unwrap();
let mut config = Config::new(tempdir.path().into());
let spec = chain_spec();
config.validator_chunk_size = 1;
config.chunk_size = 16;
config.history_length = 16;
let slasher = Slasher::open(config.clone(), test_logger()).unwrap();
let slasher = Slasher::open(config.clone(), spec, test_logger()).unwrap();
let v = vec![0];
let history_length = config.history_length as u64;