gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-04-17 12:58:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

Batch BLS verification for attestations (#2399)

Browse Source 
## Issue Addressed

NA

## Proposed Changes

Adds the ability to verify batches of aggregated/unaggregated attestations from the network.

When the `BeaconProcessor` finds there are messages in the aggregated or unaggregated attestation queues, it will first check the length of the queue:

- `== 1` verify the attestation individually.
- `>= 2` take up to 64 of those attestations and verify them in a batch.

Notably, we only perform batch verification if the queue has a backlog. We don't apply any artificial delays to attestations to try and force them into batches. 

### Batching Details

To assist with implementing batches we modify `beacon_chain::attestation_verification` to have two distinct categories for attestations:

- *Indexed* attestations: those which have passed initial validation and were valid enough for us to derive an `IndexedAttestation`.
- *Verified* attestations: those attestations which were indexed *and also* passed signature verification. These are well-formed, interesting messages which were signed by validators.

The batching functions accept `n` attestations and then return `n` attestation verification `Result`s, where those `Result`s can be any combination of `Ok` or `Err`. In other words, we attempt to verify as many attestations as possible and return specific per-attestation results so peer scores can be updated, if required.

When we batch verify attestations, we first try to map all those attestations to *indexed* attestations. If any of those attestations were able to be indexed, we then perform batch BLS verification on those indexed attestations. If the batch verification succeeds, we convert them into *verified* attestations, disabling individual signature checking. If the batch fails, we convert to verified attestations with individual signature checking enabled.

Ultimately, we optimistically try to do a batch verification of attestation signatures and fall-back to individual verification if it fails. This opens an attach vector for "poisoning" the attestations and causing us to waste a batch verification. I argue that peer scoring should do a good-enough job of defending against this and the typical-case gains massively outweigh the worst-case losses.

## Additional Info

Before this PR, attestation verification took the attestations by value (instead of by reference). It turns out that this was unnecessary and, in my opinion, resulted in some undesirable ergonomics (e.g., we had to pass the attestation back in the `Err` variant to avoid clones). In this PR I've modified attestation verification so that it now takes a reference.

I refactored the `beacon_chain/tests/attestation_verification.rs` tests so they use a builder-esque "tester" struct instead of a weird macro. It made it easier for me to test individual/batch with the same set of tests and I think it was a nice tidy-up. Notably, I did this last to try and make sure my new refactors to *actual* production code would pass under the existing test suite.
This commit is contained in:
Paul Hauner
2021-09-22 08:49:41 +00:00
parent 9667dc2f03
commit be11437c27
13 changed files with 1962 additions and 1037 deletions
Download Patch File Download Diff File Expand all files Collapse all files

451
beacon_node/beacon_chain/src/attestation_verification.rs
View File

@@ -18,13 +18,16 @@
//! types::Attestation types::SignedAggregateAndProof
//! | |
//! ▼ ▼
//! VerifiedUnaggregatedAttestation VerifiedAggregatedAttestation
//! IndexedUnaggregatedAttestation IndexedAggregatedAttestation
//! | |
//! VerifiedUnaggregatedAttestation VerifiedAggregatedAttestation
//! | |
//! -------------------------------------
//! |
//! ▼
//! impl SignatureVerifiedAttestation
//! impl VerifiedAttestation
//! ```
mod batch;
use crate::{
beacon_chain::{MAXIMUM_GOSSIP_CLOCK_DISPARITY, VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT},
@@ -53,6 +56,8 @@ use types::{
SelectionProof, SignedAggregateAndProof, Slot, SubnetId,
};
pub use batch::{batch_verify_aggregated_attestations, batch_verify_unaggregated_attestations};
/// Returned when an attestation was not successfully verified. It might not have been verified for
/// two reasons:
///
@@ -254,53 +259,105 @@ impl From<BeaconChainError> for Error {
}
}
/// Wraps a `SignedAggregateAndProof` that has been verified for propagation on the gossip network.
pub struct VerifiedAggregatedAttestation<T: BeaconChainTypes> {
signed_aggregate: SignedAggregateAndProof<T::EthSpec>,
/// Used to avoid double-checking signatures.
#[derive(Copy, Clone)]
enum CheckAttestationSignature {
Yes,
No,
}
/// Wraps a `SignedAggregateAndProof` that has been verified up until the point that an
/// `IndexedAttestation` can be derived.
///
/// These attestations have *not* undergone signature verification.
struct IndexedAggregatedAttestation<'a, T: BeaconChainTypes> {
signed_aggregate: &'a SignedAggregateAndProof<T::EthSpec>,
indexed_attestation: IndexedAttestation<T::EthSpec>,
attestation_root: Hash256,
}
/// Wraps a `Attestation` that has been verified up until the point that an `IndexedAttestation` can
/// be derived.
///
/// These attestations have *not* undergone signature verification.
struct IndexedUnaggregatedAttestation<'a, T: BeaconChainTypes> {
attestation: &'a Attestation<T::EthSpec>,
indexed_attestation: IndexedAttestation<T::EthSpec>,
subnet_id: SubnetId,
validator_index: u64,
}
/// Wraps a `SignedAggregateAndProof` that has been fully verified for propagation on the gossip
/// network.
pub struct VerifiedAggregatedAttestation<'a, T: BeaconChainTypes> {
signed_aggregate: &'a SignedAggregateAndProof<T::EthSpec>,
indexed_attestation: IndexedAttestation<T::EthSpec>,
}
/// Wraps an `Attestation` that has been verified for propagation on the gossip network.
pub struct VerifiedUnaggregatedAttestation<T: BeaconChainTypes> {
attestation: Attestation<T::EthSpec>,
impl<'a, T: BeaconChainTypes> VerifiedAggregatedAttestation<'a, T> {
pub fn into_indexed_attestation(self) -> IndexedAttestation<T::EthSpec> {
self.indexed_attestation
}
}
/// Wraps an `Attestation` that has been fully verified for propagation on the gossip network.
pub struct VerifiedUnaggregatedAttestation<'a, T: BeaconChainTypes> {
attestation: &'a Attestation<T::EthSpec>,
indexed_attestation: IndexedAttestation<T::EthSpec>,
subnet_id: SubnetId,
}
impl<'a, T: BeaconChainTypes> VerifiedUnaggregatedAttestation<'a, T> {
pub fn into_indexed_attestation(self) -> IndexedAttestation<T::EthSpec> {
self.indexed_attestation
}
}
/// Custom `Clone` implementation is to avoid the restrictive trait bounds applied by the usual derive
/// macro.
impl<T: BeaconChainTypes> Clone for VerifiedUnaggregatedAttestation<T> {
impl<'a, T: BeaconChainTypes> Clone for IndexedUnaggregatedAttestation<'a, T> {
fn clone(&self) -> Self {
Self {
attestation: self.attestation.clone(),
attestation: self.attestation,
indexed_attestation: self.indexed_attestation.clone(),
subnet_id: self.subnet_id,
validator_index: self.validator_index,
}
}
}
/// A helper trait implemented on wrapper types that can be progressed to a state where they can be
/// verified for application to fork choice.
pub trait SignatureVerifiedAttestation<T: BeaconChainTypes> {
pub trait VerifiedAttestation<T: BeaconChainTypes> {
fn attestation(&self) -> &Attestation<T::EthSpec>;
fn indexed_attestation(&self) -> &IndexedAttestation<T::EthSpec>;
}
impl<'a, T: BeaconChainTypes> SignatureVerifiedAttestation<T> for VerifiedAggregatedAttestation<T> {
impl<'a, T: BeaconChainTypes> VerifiedAttestation<T> for VerifiedAggregatedAttestation<'a, T> {
fn attestation(&self) -> &Attestation<T::EthSpec> {
self.attestation()
}
fn indexed_attestation(&self) -> &IndexedAttestation<T::EthSpec> {
&self.indexed_attestation
}
}
impl<T: BeaconChainTypes> SignatureVerifiedAttestation<T> for VerifiedUnaggregatedAttestation<T> {
impl<'a, T: BeaconChainTypes> VerifiedAttestation<T> for VerifiedUnaggregatedAttestation<'a, T> {
fn attestation(&self) -> &Attestation<T::EthSpec> {
self.attestation
}
fn indexed_attestation(&self) -> &IndexedAttestation<T::EthSpec> {
&self.indexed_attestation
}
}
/// Information about invalid attestations which might still be slashable despite being invalid.
pub enum AttestationSlashInfo<T: BeaconChainTypes, TErr> {
pub enum AttestationSlashInfo<'a, T: BeaconChainTypes, TErr> {
/// The attestation is invalid, but its signature wasn't checked.
SignatureNotChecked(Attestation<T::EthSpec>, TErr),
SignatureNotChecked(&'a Attestation<T::EthSpec>, TErr),
/// As for `SignatureNotChecked`, but we know the `IndexedAttestation`.
SignatureNotCheckedIndexed(IndexedAttestation<T::EthSpec>, TErr),
/// The attestation's signature is invalid, so it will never be slashable.
@@ -324,7 +381,7 @@ fn process_slash_info<T: BeaconChainTypes>(
if let Some(slasher) = chain.slasher.as_ref() {
let (indexed_attestation, check_signature, err) = match slash_info {
SignatureNotChecked(attestation, err) => {
match obtain_indexed_attestation_and_committees_per_slot(chain, &attestation) {
match obtain_indexed_attestation_and_committees_per_slot(chain, attestation) {
Ok((indexed, _)) => (indexed, true, err),
Err(e) => {
debug!(
@@ -367,13 +424,13 @@ fn process_slash_info<T: BeaconChainTypes>(
}
}
impl<T: BeaconChainTypes> VerifiedAggregatedAttestation<T> {
impl<'a, T: BeaconChainTypes> IndexedAggregatedAttestation<'a, T> {
/// Returns `Ok(Self)` if the `signed_aggregate` is valid to be (re)published on the gossip
/// network.
pub fn verify(
signed_aggregate: SignedAggregateAndProof<T::EthSpec>,
signed_aggregate: &'a SignedAggregateAndProof<T::EthSpec>,
chain: &BeaconChain<T>,
) -> Result<Self, (Error, SignedAggregateAndProof<T::EthSpec>)> {
) -> Result<Self, Error> {
Self::verify_slashable(signed_aggregate, chain)
.map(|verified_aggregate| {
if let Some(slasher) = chain.slasher.as_ref() {
@@ -381,9 +438,7 @@ impl<T: BeaconChainTypes> VerifiedAggregatedAttestation<T> {
}
verified_aggregate
})
.map_err(|(slash_info, original_aggregate)| {
(process_slash_info(slash_info, chain), original_aggregate)
})
.map_err(|slash_info| process_slash_info(slash_info, chain))
}
/// Run the checks that happen before an indexed attestation is constructed.
@@ -467,6 +522,56 @@ impl<T: BeaconChainTypes> VerifiedAggregatedAttestation<T> {
}
}
/// Verify the attestation, producing extra information about whether it might be slashable.
pub fn verify_slashable(
signed_aggregate: &'a SignedAggregateAndProof<T::EthSpec>,
chain: &BeaconChain<T>,
) -> Result<Self, AttestationSlashInfo<'a, T, Error>> {
use AttestationSlashInfo::*;
let attestation = &signed_aggregate.message.aggregate;
let aggregator_index = signed_aggregate.message.aggregator_index;
let attestation_root = match Self::verify_early_checks(signed_aggregate, chain) {
Ok(root) => root,
Err(e) => return Err(SignatureNotChecked(&signed_aggregate.message.aggregate, e)),
};
let indexed_attestation =
match map_attestation_committee(chain, attestation, |(committee, _)| {
// Note: this clones the signature which is known to be a relatively slow operation.
//
// Future optimizations should remove this clone.
let selection_proof =
SelectionProof::from(signed_aggregate.message.selection_proof.clone());
if !selection_proof
.is_aggregator(committee.committee.len(), &chain.spec)
.map_err(|e| Error::BeaconChainError(e.into()))?
{
return Err(Error::InvalidSelectionProof { aggregator_index });
}
// Ensure the aggregator is a member of the committee for which it is aggregating.
if !committee.committee.contains(&(aggregator_index as usize)) {
return Err(Error::AggregatorNotInCommittee { aggregator_index });
}
get_indexed_attestation(committee.committee, attestation)
.map_err(|e| BeaconChainError::from(e).into())
}) {
Ok(indexed_attestation) => indexed_attestation,
Err(e) => return Err(SignatureNotChecked(&signed_aggregate.message.aggregate, e)),
};
Ok(IndexedAggregatedAttestation {
signed_aggregate,
indexed_attestation,
attestation_root,
})
}
}
impl<'a, T: BeaconChainTypes> VerifiedAggregatedAttestation<'a, T> {
/// Run the checks that happen after the indexed attestation and signature have been checked.
fn verify_late_checks(
signed_aggregate: &SignedAggregateAndProof<T::EthSpec>,
@@ -508,82 +613,70 @@ impl<T: BeaconChainTypes> VerifiedAggregatedAttestation<T> {
Ok(())
}
/// Verify the attestation, producing extra information about whether it might be slashable.
// NOTE: Clippy considers the return too complex. This tuple is not used elsewhere so it is not
// worth creating an alias.
#[allow(clippy::type_complexity)]
pub fn verify_slashable(
signed_aggregate: SignedAggregateAndProof<T::EthSpec>,
/// Verify the `signed_aggregate`.
pub fn verify(
signed_aggregate: &'a SignedAggregateAndProof<T::EthSpec>,
chain: &BeaconChain<T>,
) -> Result<
Self,
(
AttestationSlashInfo<T, Error>,
SignedAggregateAndProof<T::EthSpec>,
),
> {
) -> Result<Self, Error> {
let indexed = IndexedAggregatedAttestation::verify(signed_aggregate, chain)?;
Self::from_indexed(indexed, chain, CheckAttestationSignature::Yes)
}
/// Complete the verification of an indexed attestation.
fn from_indexed(
signed_aggregate: IndexedAggregatedAttestation<'a, T>,
chain: &BeaconChain<T>,
check_signature: CheckAttestationSignature,
) -> Result<Self, Error> {
Self::verify_slashable(signed_aggregate, chain, check_signature)
.map(|verified_aggregate| verified_aggregate.apply_to_slasher(chain))
.map_err(|slash_info| process_slash_info(slash_info, chain))
}
fn apply_to_slasher(self, chain: &BeaconChain<T>) -> Self {
if let Some(slasher) = chain.slasher.as_ref() {
slasher.accept_attestation(self.indexed_attestation.clone());
}
self
}
/// Verify the attestation, producing extra information about whether it might be slashable.
fn verify_slashable(
signed_aggregate: IndexedAggregatedAttestation<'a, T>,
chain: &BeaconChain<T>,
check_signature: CheckAttestationSignature,
) -> Result<Self, AttestationSlashInfo<'a, T, Error>> {
use AttestationSlashInfo::*;
let attestation = &signed_aggregate.message.aggregate;
let aggregator_index = signed_aggregate.message.aggregator_index;
let attestation_root = match Self::verify_early_checks(&signed_aggregate, chain) {
Ok(root) => root,
Err(e) => {
return Err((
SignatureNotChecked(signed_aggregate.message.aggregate.clone(), e),
let IndexedAggregatedAttestation {
signed_aggregate,
indexed_attestation,
attestation_root,
} = signed_aggregate;
match check_signature {
CheckAttestationSignature::Yes => {
// Ensure that all signatures are valid.
if let Err(e) = verify_signed_aggregate_signatures(
chain,
signed_aggregate,
))
}
};
let indexed_attestation =
match map_attestation_committee(chain, attestation, |(committee, _)| {
// Note: this clones the signature which is known to be a relatively slow operation.
//
// Future optimizations should remove this clone.
let selection_proof =
SelectionProof::from(signed_aggregate.message.selection_proof.clone());
if !selection_proof
.is_aggregator(committee.committee.len(), &chain.spec)
.map_err(|e| Error::BeaconChainError(e.into()))?
{
return Err(Error::InvalidSelectionProof { aggregator_index });
}
// Ensure the aggregator is a member of the committee for which it is aggregating.
if !committee.committee.contains(&(aggregator_index as usize)) {
return Err(Error::AggregatorNotInCommittee { aggregator_index });
}
get_indexed_attestation(committee.committee, attestation)
.map_err(|e| BeaconChainError::from(e).into())
}) {
Ok(indexed_attestation) => indexed_attestation,
Err(e) => {
return Err((
SignatureNotChecked(signed_aggregate.message.aggregate.clone(), e),
signed_aggregate,
))
}
};
// Ensure that all signatures are valid.
if let Err(e) =
verify_signed_aggregate_signatures(chain, &signed_aggregate, &indexed_attestation)
&indexed_attestation,
)
.and_then(|is_valid| {
if !is_valid {
Err(Error::InvalidSignature)
} else {
Ok(())
}
})
{
return Err((SignatureInvalid(e), signed_aggregate));
}
}) {
return Err(SignatureInvalid(e));
}
}
CheckAttestationSignature::No => (),
};
if let Err(e) = Self::verify_late_checks(&signed_aggregate, attestation_root, chain) {
return Err((SignatureValid(indexed_attestation, e), signed_aggregate));
if let Err(e) = Self::verify_late_checks(signed_aggregate, attestation_root, chain) {
return Err(SignatureValid(indexed_attestation, e));
}
Ok(VerifiedAggregatedAttestation {
@@ -592,11 +685,6 @@ impl<T: BeaconChainTypes> VerifiedAggregatedAttestation<T> {
})
}
/// A helper function to add this aggregate to `beacon_chain.op_pool`.
pub fn add_to_pool(self, chain: &BeaconChain<T>) -> Result<Self, Error> {
chain.add_to_block_inclusion_pool(self)
}
/// Returns the underlying `attestation` for the `signed_aggregate`.
pub fn attestation(&self) -> &Attestation<T::EthSpec> {
&self.signed_aggregate.message.aggregate
@@ -604,11 +692,11 @@ impl<T: BeaconChainTypes> VerifiedAggregatedAttestation<T> {
/// Returns the underlying `signed_aggregate`.
pub fn aggregate(&self) -> &SignedAggregateAndProof<T::EthSpec> {
&self.signed_aggregate
self.signed_aggregate
}
}
impl<T: BeaconChainTypes> VerifiedUnaggregatedAttestation<T> {
impl<'a, T: BeaconChainTypes> IndexedUnaggregatedAttestation<'a, T> {
/// Run the checks that happen before an indexed attestation is constructed.
pub fn verify_early_checks(
attestation: &Attestation<T::EthSpec>,
@@ -699,6 +787,75 @@ impl<T: BeaconChainTypes> VerifiedUnaggregatedAttestation<T> {
Ok((validator_index, expected_subnet_id))
}
/// Returns `Ok(Self)` if the `attestation` is valid to be (re)published on the gossip
/// network.
///
/// `subnet_id` is the subnet from which we received this attestation. This function will
/// verify that it was received on the correct subnet.
pub fn verify(
attestation: &'a Attestation<T::EthSpec>,
subnet_id: Option<SubnetId>,
chain: &BeaconChain<T>,
) -> Result<Self, Error> {
Self::verify_slashable(attestation, subnet_id, chain)
.map(|verified_unaggregated| {
if let Some(slasher) = chain.slasher.as_ref() {
slasher.accept_attestation(verified_unaggregated.indexed_attestation.clone());
}
verified_unaggregated
})
.map_err(|slash_info| process_slash_info(slash_info, chain))
}
/// Verify the attestation, producing extra information about whether it might be slashable.
pub fn verify_slashable(
attestation: &'a Attestation<T::EthSpec>,
subnet_id: Option<SubnetId>,
chain: &BeaconChain<T>,
) -> Result<Self, AttestationSlashInfo<'a, T, Error>> {
use AttestationSlashInfo::*;
if let Err(e) = Self::verify_early_checks(attestation, chain) {
return Err(SignatureNotChecked(attestation, e));
}
let (indexed_attestation, committees_per_slot) =
match obtain_indexed_attestation_and_committees_per_slot(chain, attestation) {
Ok(x) => x,
Err(e) => {
return Err(SignatureNotChecked(attestation, e));
}
};
let (validator_index, expected_subnet_id) = match Self::verify_middle_checks(
attestation,
&indexed_attestation,
committees_per_slot,
subnet_id,
chain,
) {
Ok(t) => t,
Err(e) => return Err(SignatureNotCheckedIndexed(indexed_attestation, e)),
};
Ok(Self {
attestation,
indexed_attestation,
subnet_id: expected_subnet_id,
validator_index,
})
}
/// Returns a mutable reference to the underlying attestation.
///
/// Only use during testing since modifying the `IndexedAttestation` can cause the attestation
/// to no-longer be valid.
pub fn __indexed_attestation_mut(&mut self) -> &mut IndexedAttestation<T::EthSpec> {
&mut self.indexed_attestation
}
}
impl<'a, T: BeaconChainTypes> VerifiedUnaggregatedAttestation<'a, T> {
/// Run the checks that apply after the signature has been checked.
fn verify_late_checks(
attestation: &Attestation<T::EthSpec>,
@@ -725,88 +882,70 @@ impl<T: BeaconChainTypes> VerifiedUnaggregatedAttestation<T> {
Ok(())
}
/// Returns `Ok(Self)` if the `attestation` is valid to be (re)published on the gossip
/// network.
///
/// `subnet_id` is the subnet from which we received this attestation. This function will
/// verify that it was received on the correct subnet.
/// Verify the `unaggregated_attestation`.
pub fn verify(
attestation: Attestation<T::EthSpec>,
unaggregated_attestation: &'a Attestation<T::EthSpec>,
subnet_id: Option<SubnetId>,
chain: &BeaconChain<T>,
) -> Result<Self, (Error, Attestation<T::EthSpec>)> {
Self::verify_slashable(attestation, subnet_id, chain)
.map(|verified_unaggregated| {
if let Some(slasher) = chain.slasher.as_ref() {
slasher.accept_attestation(verified_unaggregated.indexed_attestation.clone());
}
verified_unaggregated
})
.map_err(|(slash_info, original_attestation)| {
(process_slash_info(slash_info, chain), original_attestation)
})
) -> Result<Self, Error> {
let indexed =
IndexedUnaggregatedAttestation::verify(unaggregated_attestation, subnet_id, chain)?;
Self::from_indexed(indexed, chain, CheckAttestationSignature::Yes)
}
/// Complete the verification of an indexed attestation.
fn from_indexed(
attestation: IndexedUnaggregatedAttestation<'a, T>,
chain: &BeaconChain<T>,
check_signature: CheckAttestationSignature,
) -> Result<Self, Error> {
Self::verify_slashable(attestation, chain, check_signature)
.map(|verified_unaggregated| verified_unaggregated.apply_to_slasher(chain))
.map_err(|slash_info| process_slash_info(slash_info, chain))
}
fn apply_to_slasher(self, chain: &BeaconChain<T>) -> Self {
if let Some(slasher) = chain.slasher.as_ref() {
slasher.accept_attestation(self.indexed_attestation.clone());
}
self
}
/// Verify the attestation, producing extra information about whether it might be slashable.
// NOTE: Clippy considers the return too complex. This tuple is not used elsewhere so it is not
// worth creating an alias.
#[allow(clippy::type_complexity)]
pub fn verify_slashable(
attestation: Attestation<T::EthSpec>,
subnet_id: Option<SubnetId>,
fn verify_slashable(
attestation: IndexedUnaggregatedAttestation<'a, T>,
chain: &BeaconChain<T>,
) -> Result<Self, (AttestationSlashInfo<T, Error>, Attestation<T::EthSpec>)> {
check_signature: CheckAttestationSignature,
) -> Result<Self, AttestationSlashInfo<'a, T, Error>> {
use AttestationSlashInfo::*;
if let Err(e) = Self::verify_early_checks(&attestation, chain) {
return Err((SignatureNotChecked(attestation.clone(), e), attestation));
}
let (indexed_attestation, committees_per_slot) =
match obtain_indexed_attestation_and_committees_per_slot(chain, &attestation) {
Ok(x) => x,
Err(e) => {
return Err((SignatureNotChecked(attestation.clone(), e), attestation));
}
};
let (validator_index, expected_subnet_id) = match Self::verify_middle_checks(
&attestation,
&indexed_attestation,
committees_per_slot,
let IndexedUnaggregatedAttestation {
attestation,
indexed_attestation,
subnet_id,
chain,
) {
Ok(t) => t,
Err(e) => {
return Err((
SignatureNotCheckedIndexed(indexed_attestation, e),
attestation,
))
validator_index,
} = attestation;
match check_signature {
CheckAttestationSignature::Yes => {
if let Err(e) = verify_attestation_signature(chain, &indexed_attestation) {
return Err(SignatureInvalid(e));
}
}
CheckAttestationSignature::No => (),
};
// The aggregate signature of the attestation is valid.
if let Err(e) = verify_attestation_signature(chain, &indexed_attestation) {
return Err((SignatureInvalid(e), attestation));
}
if let Err(e) = Self::verify_late_checks(&attestation, validator_index, chain) {
return Err((SignatureValid(indexed_attestation, e), attestation));
if let Err(e) = Self::verify_late_checks(attestation, validator_index, chain) {
return Err(SignatureValid(indexed_attestation, e));
}
Ok(Self {
attestation,
indexed_attestation,
subnet_id: expected_subnet_id,
subnet_id,
})
}
/// A helper function to add this attestation to `beacon_chain.naive_aggregation_pool`.
pub fn add_to_pool(self, chain: &BeaconChain<T>) -> Result<Self, Error> {
chain.add_to_naive_aggregation_pool(self)
}
/// Returns the correct subnet for the attestation.
pub fn subnet_id(&self) -> SubnetId {
self.subnet_id
@@ -814,7 +953,7 @@ impl<T: BeaconChainTypes> VerifiedUnaggregatedAttestation<T> {
/// Returns the wrapped `attestation`.
pub fn attestation(&self) -> &Attestation<T::EthSpec> {
&self.attestation
self.attestation
}
/// Returns the wrapped `indexed_attestation`.

222
beacon_node/beacon_chain/src/attestation_verification/batch.rs Normal file
View File

@@ -0,0 +1,222 @@
//! These two `batch_...` functions provide verification of batches of attestations. They provide
//! significant CPU-time savings by performing batch verification of BLS signatures.
//!
//! In each function, attestations are "indexed" (i.e., the `IndexedAttestation` is computed), to
//! determine if they should progress to signature verification. Then, all attestations which were
//! successfully indexed have their signatures verified in a batch. If that signature batch fails
//! then all attestation signatures are verified independently.
//!
//! The outcome of each function is a `Vec<Result>` with a one-to-one mapping to the attestations
//! supplied as input. Each result provides the exact success or failure result of the corresponding
//! attestation, with no loss of fidelity when compared to individual verification.
use super::{
CheckAttestationSignature, Error, IndexedAggregatedAttestation, IndexedUnaggregatedAttestation,
VerifiedAggregatedAttestation, VerifiedUnaggregatedAttestation,
};
use crate::{
beacon_chain::VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT, metrics, BeaconChain, BeaconChainError,
BeaconChainTypes,
};
use bls::verify_signature_sets;
use state_processing::signature_sets::{
indexed_attestation_signature_set_from_pubkeys, signed_aggregate_selection_proof_signature_set,
signed_aggregate_signature_set,
};
use std::borrow::Cow;
use types::*;
/// Verify aggregated attestations using batch BLS signature verification.
///
/// See module-level docs for more info.
pub fn batch_verify_aggregated_attestations<'a, T, I>(
aggregates: I,
chain: &BeaconChain<T>,
) -> Result<Vec<Result<VerifiedAggregatedAttestation<'a, T>, Error>>, Error>
where
T: BeaconChainTypes,
I: Iterator<Item = &'a SignedAggregateAndProof<T::EthSpec>> + ExactSizeIterator,
{
let mut num_indexed = 0;
let mut num_failed = 0;
// Perform indexing of all attestations, collecting the results.
let indexing_results = aggregates
.map(|aggregate| {
let result = IndexedAggregatedAttestation::verify(aggregate, chain);
if result.is_ok() {
num_indexed += 1;
} else {
num_failed += 1;
}
result
})
.collect::<Vec<_>>();
// May be set to `No` if batch verification succeeds.
let mut check_signatures = CheckAttestationSignature::Yes;
// Perform batch BLS verification, if any attestation signatures are worth checking.
if num_indexed > 0 {
let signature_setup_timer =
metrics::start_timer(&metrics::ATTESTATION_PROCESSING_BATCH_AGG_SIGNATURE_SETUP_TIMES);
let pubkey_cache = chain
.validator_pubkey_cache
.try_read_for(VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT)
.ok_or(BeaconChainError::ValidatorPubkeyCacheLockTimeout)?;
let fork = chain.with_head(|head| Ok::<_, BeaconChainError>(head.beacon_state.fork()))?;
let mut signature_sets = Vec::with_capacity(num_indexed * 3);
// Iterate, flattening to get only the `Ok` values.
for indexed in indexing_results.iter().flatten() {
let signed_aggregate = &indexed.signed_aggregate;
let indexed_attestation = &indexed.indexed_attestation;
signature_sets.push(
signed_aggregate_selection_proof_signature_set(
|validator_index| pubkey_cache.get(validator_index).map(Cow::Borrowed),
signed_aggregate,
&fork,
chain.genesis_validators_root,
&chain.spec,
)
.map_err(BeaconChainError::SignatureSetError)?,
);
signature_sets.push(
signed_aggregate_signature_set(
|validator_index| pubkey_cache.get(validator_index).map(Cow::Borrowed),
signed_aggregate,
&fork,
chain.genesis_validators_root,
&chain.spec,
)
.map_err(BeaconChainError::SignatureSetError)?,
);
signature_sets.push(
indexed_attestation_signature_set_from_pubkeys(
|validator_index| pubkey_cache.get(validator_index).map(Cow::Borrowed),
&indexed_attestation.signature,
indexed_attestation,
&fork,
chain.genesis_validators_root,
&chain.spec,
)
.map_err(BeaconChainError::SignatureSetError)?,
);
}
metrics::stop_timer(signature_setup_timer);
let _signature_verification_timer =
metrics::start_timer(&metrics::ATTESTATION_PROCESSING_BATCH_AGG_SIGNATURE_TIMES);
if verify_signature_sets(signature_sets.iter()) {
// Since all the signatures verified in a batch, there's no reason for them to be
// checked again later.
check_signatures = CheckAttestationSignature::No
}
}
// Complete the attestation verification, potentially verifying all signatures independently.
let final_results = indexing_results
.into_iter()
.map(|result| match result {
Ok(indexed) => {
VerifiedAggregatedAttestation::from_indexed(indexed, chain, check_signatures)
}
Err(e) => Err(e),
})
.collect();
Ok(final_results)
}
/// Verify unaggregated attestations using batch BLS signature verification.
///
/// See module-level docs for more info.
pub fn batch_verify_unaggregated_attestations<'a, T, I>(
attestations: I,
chain: &BeaconChain<T>,
) -> Result<Vec<Result<VerifiedUnaggregatedAttestation<'a, T>, Error>>, Error>
where
T: BeaconChainTypes,
I: Iterator<Item = (&'a Attestation<T::EthSpec>, Option<SubnetId>)> + ExactSizeIterator,
{
let mut num_partially_verified = 0;
let mut num_failed = 0;
// Perform partial verification of all attestations, collecting the results.
let partial_results = attestations
.map(|(attn, subnet_opt)| {
let result = IndexedUnaggregatedAttestation::verify(attn, subnet_opt, chain);
if result.is_ok() {
num_partially_verified += 1;
} else {
num_failed += 1;
}
result
})
.collect::<Vec<_>>();
// May be set to `No` if batch verification succeeds.
let mut check_signatures = CheckAttestationSignature::Yes;
// Perform batch BLS verification, if any attestation signatures are worth checking.
if num_partially_verified > 0 {
let signature_setup_timer = metrics::start_timer(
&metrics::ATTESTATION_PROCESSING_BATCH_UNAGG_SIGNATURE_SETUP_TIMES,
);
let pubkey_cache = chain
.validator_pubkey_cache
.try_read_for(VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT)
.ok_or(BeaconChainError::ValidatorPubkeyCacheLockTimeout)?;
let fork = chain.with_head(|head| Ok::<_, BeaconChainError>(head.beacon_state.fork()))?;
let mut signature_sets = Vec::with_capacity(num_partially_verified);
// Iterate, flattening to get only the `Ok` values.
for partially_verified in partial_results.iter().flatten() {
let indexed_attestation = &partially_verified.indexed_attestation;
let signature_set = indexed_attestation_signature_set_from_pubkeys(
|validator_index| pubkey_cache.get(validator_index).map(Cow::Borrowed),
&indexed_attestation.signature,
indexed_attestation,
&fork,
chain.genesis_validators_root,
&chain.spec,
)
.map_err(BeaconChainError::SignatureSetError)?;
signature_sets.push(signature_set);
}
metrics::stop_timer(signature_setup_timer);
let _signature_verification_timer =
metrics::start_timer(&metrics::ATTESTATION_PROCESSING_BATCH_UNAGG_SIGNATURE_TIMES);
if verify_signature_sets(signature_sets.iter()) {
// Since all the signatures verified in a batch, there's no reason for them to be
// checked again later.
check_signatures = CheckAttestationSignature::No
}
}
// Complete the attestation verification, potentially verifying all signatures independently.
let final_results = partial_results
.into_iter()
.map(|result| match result {
Ok(partial) => {
VerifiedUnaggregatedAttestation::from_indexed(partial, chain, check_signatures)
}
Err(e) => Err(e),
})
.collect();
Ok(final_results)
}

68
beacon_node/beacon_chain/src/beacon_chain.rs
View File

@@ -1,5 +1,6 @@
use crate::attestation_verification::{
Error as AttestationError, SignatureVerifiedAttestation, VerifiedAggregatedAttestation,
batch_verify_aggregated_attestations, batch_verify_unaggregated_attestations,
Error as AttestationError, VerifiedAggregatedAttestation, VerifiedAttestation,
VerifiedUnaggregatedAttestation,
};
use crate::attester_cache::{AttesterCache, AttesterCacheKey};
@@ -1510,17 +1511,32 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
})
}
/// Performs the same validation as `Self::verify_unaggregated_attestation_for_gossip`, but for
/// multiple attestations using batch BLS verification. Batch verification can provide
/// significant CPU-time savings compared to individual verification.
pub fn batch_verify_unaggregated_attestations_for_gossip<'a, I>(
&self,
attestations: I,
) -> Result<
Vec<Result<VerifiedUnaggregatedAttestation<'a, T>, AttestationError>>,
AttestationError,
>
where
I: Iterator<Item = (&'a Attestation<T::EthSpec>, Option<SubnetId>)> + ExactSizeIterator,
{
batch_verify_unaggregated_attestations(attestations, self)
}
/// Accepts some `Attestation` from the network and attempts to verify it, returning `Ok(_)` if
/// it is valid to be (re)broadcast on the gossip network.
///
/// The attestation must be "unaggregated", that is it must have exactly one
/// aggregation bit set.
pub fn verify_unaggregated_attestation_for_gossip(
pub fn verify_unaggregated_attestation_for_gossip<'a>(
&self,
unaggregated_attestation: Attestation<T::EthSpec>,
unaggregated_attestation: &'a Attestation<T::EthSpec>,
subnet_id: Option<SubnetId>,
) -> Result<VerifiedUnaggregatedAttestation<T>, (AttestationError, Attestation<T::EthSpec>)>
{
) -> Result<VerifiedUnaggregatedAttestation<'a, T>, AttestationError> {
metrics::inc_counter(&metrics::UNAGGREGATED_ATTESTATION_PROCESSING_REQUESTS);
let _timer =
metrics::start_timer(&metrics::UNAGGREGATED_ATTESTATION_GOSSIP_VERIFICATION_TIMES);
@@ -1539,15 +1555,25 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
)
}
/// Performs the same validation as `Self::verify_aggregated_attestation_for_gossip`, but for
/// multiple attestations using batch BLS verification. Batch verification can provide
/// significant CPU-time savings compared to individual verification.
pub fn batch_verify_aggregated_attestations_for_gossip<'a, I>(
&self,
aggregates: I,
) -> Result<Vec<Result<VerifiedAggregatedAttestation<'a, T>, AttestationError>>, AttestationError>
where
I: Iterator<Item = &'a SignedAggregateAndProof<T::EthSpec>> + ExactSizeIterator,
{
batch_verify_aggregated_attestations(aggregates, self)
}
/// Accepts some `SignedAggregateAndProof` from the network and attempts to verify it,
/// returning `Ok(_)` if it is valid to be (re)broadcast on the gossip network.
pub fn verify_aggregated_attestation_for_gossip(
pub fn verify_aggregated_attestation_for_gossip<'a>(
&self,
signed_aggregate: SignedAggregateAndProof<T::EthSpec>,
) -> Result<
VerifiedAggregatedAttestation<T>,
(AttestationError, SignedAggregateAndProof<T::EthSpec>),
> {
signed_aggregate: &'a SignedAggregateAndProof<T::EthSpec>,
) -> Result<VerifiedAggregatedAttestation<'a, T>, AttestationError> {
metrics::inc_counter(&metrics::AGGREGATED_ATTESTATION_PROCESSING_REQUESTS);
let _timer =
metrics::start_timer(&metrics::AGGREGATED_ATTESTATION_GOSSIP_VERIFICATION_TIMES);
@@ -1597,13 +1623,13 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
/// Accepts some attestation-type object and attempts to verify it in the context of fork
/// choice. If it is valid it is applied to `self.fork_choice`.
///
/// Common items that implement `SignatureVerifiedAttestation`:
/// Common items that implement `VerifiedAttestation`:
///
/// - `VerifiedUnaggregatedAttestation`
/// - `VerifiedAggregatedAttestation`
pub fn apply_attestation_to_fork_choice(
&self,
verified: &impl SignatureVerifiedAttestation<T>,
verified: &impl VerifiedAttestation<T>,
) -> Result<(), Error> {
let _timer = metrics::start_timer(&metrics::FORK_CHOICE_PROCESS_ATTESTATION_TIMES);
@@ -1623,8 +1649,8 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
/// and no error is returned.
pub fn add_to_naive_aggregation_pool(
&self,
unaggregated_attestation: VerifiedUnaggregatedAttestation<T>,
) -> Result<VerifiedUnaggregatedAttestation<T>, AttestationError> {
unaggregated_attestation: &impl VerifiedAttestation<T>,
) -> Result<(), AttestationError> {
let _timer = metrics::start_timer(&metrics::ATTESTATION_PROCESSING_APPLY_TO_AGG_POOL);
let attestation = unaggregated_attestation.attestation();
@@ -1660,7 +1686,7 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
}
};
Ok(unaggregated_attestation)
Ok(())
}
/// Accepts a `VerifiedSyncCommitteeMessage` and attempts to apply it to the "naive
@@ -1727,13 +1753,13 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
Ok(verified_sync_committee_message)
}
/// Accepts a `VerifiedAggregatedAttestation` and attempts to apply it to `self.op_pool`.
/// Accepts a `VerifiedAttestation` and attempts to apply it to `self.op_pool`.
///
/// The op pool is used by local block producers to pack blocks with operations.
pub fn add_to_block_inclusion_pool(
&self,
signed_aggregate: VerifiedAggregatedAttestation<T>,
) -> Result<VerifiedAggregatedAttestation<T>, AttestationError> {
verified_attestation: &impl VerifiedAttestation<T>,
) -> Result<(), AttestationError> {
let _timer = metrics::start_timer(&metrics::ATTESTATION_PROCESSING_APPLY_TO_OP_POOL);
// If there's no eth1 chain then it's impossible to produce blocks and therefore
@@ -1745,7 +1771,7 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
self.op_pool
.insert_attestation(
// TODO: address this clone.
signed_aggregate.attestation().clone(),
verified_attestation.attestation().clone(),
&fork,
self.genesis_validators_root,
&self.spec,
@@ -1753,7 +1779,7 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
.map_err(Error::from)?;
}
Ok(signed_aggregate)
Ok(())
}
/// Accepts a `VerifiedSyncContribution` and attempts to apply it to `self.op_pool`.

20
beacon_node/beacon_chain/src/metrics.rs
View File

@@ -199,6 +199,26 @@ lazy_static! {
"Time spent on the signature verification of attestation processing"
);
/*
* Batch Attestation Processing
*/
pub static ref ATTESTATION_PROCESSING_BATCH_AGG_SIGNATURE_SETUP_TIMES: Result<Histogram> = try_create_histogram(
"beacon_attestation_processing_batch_agg_signature_setup_times",
"Time spent on setting up for the signature verification of batch aggregate processing"
);
pub static ref ATTESTATION_PROCESSING_BATCH_AGG_SIGNATURE_TIMES: Result<Histogram> = try_create_histogram(
"beacon_attestation_processing_batch_agg_signature_times",
"Time spent on the signature verification of batch aggregate attestation processing"
);
pub static ref ATTESTATION_PROCESSING_BATCH_UNAGG_SIGNATURE_SETUP_TIMES: Result<Histogram> = try_create_histogram(
"beacon_attestation_processing_batch_unagg_signature_setup_times",
"Time spent on setting up for the signature verification of batch unaggregate processing"
);
pub static ref ATTESTATION_PROCESSING_BATCH_UNAGG_SIGNATURE_TIMES: Result<Histogram> = try_create_histogram(
"beacon_attestation_processing_batch_unagg_signature_times",
"Time spent on the signature verification of batch unaggregate attestation processing"
);
/*
* Shuffling cache
*/

51
beacon_node/beacon_chain/src/test_utils.rs
View File

@@ -1159,29 +1159,42 @@ where
}
pub fn process_attestations(&self, attestations: HarnessAttestations<E>) {
for (unaggregated_attestations, maybe_signed_aggregate) in attestations.into_iter() {
for (attestation, subnet_id) in unaggregated_attestations {
self.chain
.verify_unaggregated_attestation_for_gossip(
attestation.clone(),
Some(subnet_id),
)
.unwrap()
.add_to_pool(&self.chain)
.unwrap();
let num_validators = self.validator_keypairs.len();
let mut unaggregated = Vec::with_capacity(num_validators);
// This is an over-allocation, but it should be fine. It won't be *that* memory hungry and
// it's nice to have fast tests.
let mut aggregated = Vec::with_capacity(num_validators);
for (unaggregated_attestations, maybe_signed_aggregate) in attestations.iter() {
for (attn, subnet) in unaggregated_attestations {
unaggregated.push((attn, Some(*subnet)));
}
if let Some(signed_aggregate) = maybe_signed_aggregate {
let attn = self
.chain
.verify_aggregated_attestation_for_gossip(signed_aggregate)
.unwrap();
self.chain.apply_attestation_to_fork_choice(&attn).unwrap();
self.chain.add_to_block_inclusion_pool(attn).unwrap();
if let Some(a) = maybe_signed_aggregate {
aggregated.push(a)
}
}
for result in self
.chain
.batch_verify_unaggregated_attestations_for_gossip(unaggregated.into_iter())
.unwrap()
{
let verified = result.unwrap();
self.chain.add_to_naive_aggregation_pool(&verified).unwrap();
}
for result in self
.chain
.batch_verify_aggregated_attestations_for_gossip(aggregated.into_iter())
.unwrap()
{
let verified = result.unwrap();
self.chain
.apply_attestation_to_fork_choice(&verified)
.unwrap();
self.chain.add_to_block_inclusion_pool(&verified).unwrap();
}
}
pub fn set_current_slot(&self, slot: Slot) {