gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-03 00:31:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Optimize validator duties (#2243)

Browse Source 
## Issue Addressed

Closes #2052

## Proposed Changes

- Refactor the attester/proposer duties endpoints in the BN
    - Performance improvements
    - Fixes some potential inconsistencies with the dependent root fields.
    - Removes `http_api::beacon_proposer_cache` and just uses the one on the `BeaconChain` instead.
    - Move the code for the proposer/attester duties endpoints into separate files, for readability.
- Refactor the `DutiesService` in the VC
    - Required to reduce the delay on broadcasting new blocks.
    - Gets rid of the `ValidatorDuty` shim struct that came about when we adopted the standard API.
    - Separate block/attestation duty tasks so that they don't block each other when one is slow.
- In the VC, use `PublicKeyBytes` to represent validators instead of `PublicKey`. `PublicKey` is a legit crypto object whilst `PublicKeyBytes` is just a byte-array, it's much faster to clone/hash `PublicKeyBytes` and this change has had a significant impact on runtimes.
    - Unfortunately this has created lots of dust changes.
 - In the BN, store `PublicKeyBytes` in the `beacon_proposer_cache` and allow access to them. The HTTP API always sends `PublicKeyBytes` over the wire and the conversion from `PublicKey` -> `PublickeyBytes` is non-trivial, especially when queries have 100s/1000s of validators (like Pyrmont).
 - Add the `state_processing::state_advance` mod which dedups a lot of the "apply `n` skip slots to the state" code.
    - This also fixes a bug with some functions which were failing to include a state root as per [this comment](072695284f/consensus/state_processing/src/state_advance.rs (L69-L74)). I couldn't find any instance of this bug that resulted in anything more severe than keying a shuffling cache by the wrong block root.
 - Swap the VC block service to use `mpsc` from `tokio` instead of `futures`. This is consistent with the rest of the code base.
    
~~This PR *reduces* the size of the codebase 🎉~~ It *used* to reduce the size of the code base before I added more comments. 

## Observations on Prymont

- Proposer duties times down from peaks of 450ms to consistent <1ms.
- Current epoch attester duties times down from >1s peaks to a consistent 20-30ms.
- Block production down from +600ms to 100-200ms.

## Additional Info

- ~~Blocked on #2241~~
- ~~Blocked on #2234~~

## TODO

- [x] ~~Refactor this into some smaller PRs?~~ Leaving this as-is for now.
- [x] Address `per_slot_processing` roots.
- [x] Investigate slow next epoch times. Not getting added to cache on block processing?
- [x] Consider [this](072695284f/beacon_node/store/src/hot_cold_store.rs (L811-L812)) in the scenario of replacing the state roots


Co-authored-by: pawan <pawandhananjay@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
This commit is contained in:
Paul Hauner
2021-03-17 05:09:57 +00:00
parent 6a69b20be1
commit 015ab7d0a7
49 changed files with 2201 additions and 1833 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
validator_client/slashing_protection/src/interchange.rs
View File

@@ -1,6 +1,6 @@
use serde_derive::{Deserialize, Serialize};
use std::collections::HashSet;
use types::{Epoch, Hash256, PublicKey, Slot};
use types::{Epoch, Hash256, PublicKeyBytes, Slot};
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
@@ -13,7 +13,7 @@ pub struct InterchangeMetadata {
#[derive(Debug, Clone, PartialEq, Eq, Hash, Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
pub struct InterchangeData {
pub pubkey: PublicKey,
pub pubkey: PublicKeyBytes,
pub signed_blocks: Vec<SignedBlock>,
pub signed_attestations: Vec<SignedAttestation>,
}

6
validator_client/slashing_protection/src/interchange_test.rs
View File

@@ -5,7 +5,7 @@ use crate::{
};
use serde_derive::{Deserialize, Serialize};
use tempfile::tempdir;
use types::{Epoch, Hash256, PublicKey, Slot};
use types::{Epoch, Hash256, PublicKeyBytes, Slot};
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct MultiTestCase {
@@ -25,7 +25,7 @@ pub struct TestCase {
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct TestBlock {
pub pubkey: PublicKey,
pub pubkey: PublicKeyBytes,
pub slot: Slot,
pub signing_root: Hash256,
pub should_succeed: bool,
@@ -33,7 +33,7 @@ pub struct TestBlock {
#[derive(Debug, Clone, Deserialize, Serialize)]
pub struct TestAttestation {
pub pubkey: PublicKey,
pub pubkey: PublicKeyBytes,
pub source_epoch: Epoch,
pub target_epoch: Epoch,
pub signing_root: Hash256,

4
validator_client/slashing_protection/src/lib.rs
View File

@@ -17,7 +17,7 @@ pub use crate::slashing_database::{
use rusqlite::Error as SQLError;
use std::io::{Error as IOError, ErrorKind};
use std::string::ToString;
use types::{Hash256, PublicKey};
use types::{Hash256, PublicKeyBytes};
/// The filename within the `validators` directory that contains the slashing protection DB.
pub const SLASHING_PROTECTION_FILENAME: &str = "slashing_protection.sqlite";
@@ -27,7 +27,7 @@ pub const SLASHING_PROTECTION_FILENAME: &str = "slashing_protection.sqlite";
/// This could be because it's slashable, or because an error occurred.
#[derive(PartialEq, Debug)]
pub enum NotSafe {
UnregisteredValidator(PublicKey),
UnregisteredValidator(PublicKeyBytes),
InvalidBlock(InvalidBlock),
InvalidAttestation(InvalidAttestation),
IOError(ErrorKind),

6
validator_client/slashing_protection/src/parallel_tests.rs
View File

@@ -16,7 +16,7 @@ fn block_same_slot() {
let pk = pubkey(0);
slashing_db.register_validator(&pk).unwrap();
slashing_db.register_validator(pk).unwrap();
// A stream of blocks all with the same slot.
let num_blocks = 10;
@@ -37,7 +37,7 @@ fn attestation_same_target() {
let pk = pubkey(0);
slashing_db.register_validator(&pk).unwrap();
slashing_db.register_validator(pk).unwrap();
// A stream of attestations all with the same target.
let num_attestations = 10;
@@ -64,7 +64,7 @@ fn attestation_surround_fest() {
let pk = pubkey(0);
slashing_db.register_validator(&pk).unwrap();
slashing_db.register_validator(pk).unwrap();
// A stream of attestations that all surround each other.
let num_attestations = 10;

58
validator_client/slashing_protection/src/slashing_database.rs
View File

@@ -10,7 +10,7 @@ use rusqlite::{params, OptionalExtension, Transaction, TransactionBehavior};
use std::fs::{File, OpenOptions};
use std::path::Path;
use std::time::Duration;
use types::{AttestationData, BeaconBlockHeader, Epoch, Hash256, PublicKey, SignedRoot, Slot};
use types::{AttestationData, BeaconBlockHeader, Epoch, Hash256, PublicKeyBytes, SignedRoot, Slot};
type Pool = r2d2::Pool<SqliteConnectionManager>;
@@ -147,14 +147,14 @@ impl SlashingDatabase {
///
/// This allows the validator to record their signatures in the database, and check
/// for slashings.
pub fn register_validator(&self, validator_pk: &PublicKey) -> Result<(), NotSafe> {
self.register_validators(std::iter::once(validator_pk))
pub fn register_validator(&self, validator_pk: PublicKeyBytes) -> Result<(), NotSafe> {
self.register_validators(std::iter::once(&validator_pk))
}
/// Register multiple validators with the slashing protection database.
pub fn register_validators<'a>(
&self,
public_keys: impl Iterator<Item = &'a PublicKey>,
public_keys: impl Iterator<Item = &'a PublicKeyBytes>,
) -> Result<(), NotSafe> {
let mut conn = self.conn_pool.get()?;
let txn = conn.transaction()?;
@@ -168,7 +168,7 @@ impl SlashingDatabase {
/// The caller must commit the transaction for the changes to be persisted.
pub fn register_validators_in_txn<'a>(
&self,
public_keys: impl Iterator<Item = &'a PublicKey>,
public_keys: impl Iterator<Item = &'a PublicKeyBytes>,
txn: &Transaction,
) -> Result<(), NotSafe> {
let mut stmt = txn.prepare("INSERT INTO validators (public_key) VALUES (?1)")?;
@@ -183,7 +183,7 @@ impl SlashingDatabase {
/// Check that all of the given validators are registered.
pub fn check_validator_registrations<'a>(
&self,
mut public_keys: impl Iterator<Item = &'a PublicKey>,
mut public_keys: impl Iterator<Item = &'a PublicKeyBytes>,
) -> Result<(), NotSafe> {
let mut conn = self.conn_pool.get()?;
let txn = conn.transaction()?;
@@ -195,7 +195,7 @@ impl SlashingDatabase {
///
/// This is NOT the same as a validator index, and depends on the ordering that validators
/// are registered with the slashing protection database (and may vary between machines).
pub fn get_validator_id(&self, public_key: &PublicKey) -> Result<i64, NotSafe> {
pub fn get_validator_id(&self, public_key: &PublicKeyBytes) -> Result<i64, NotSafe> {
let mut conn = self.conn_pool.get()?;
let txn = conn.transaction()?;
self.get_validator_id_in_txn(&txn, public_key)
@@ -204,17 +204,17 @@ impl SlashingDatabase {
fn get_validator_id_in_txn(
&self,
txn: &Transaction,
public_key: &PublicKey,
public_key: &PublicKeyBytes,
) -> Result<i64, NotSafe> {
self.get_validator_id_opt(txn, public_key)?
.ok_or_else(|| NotSafe::UnregisteredValidator(public_key.clone()))
.ok_or_else(|| NotSafe::UnregisteredValidator(*public_key))
}
/// Optional version of `get_validator_id`.
fn get_validator_id_opt(
&self,
txn: &Transaction,
public_key: &PublicKey,
public_key: &PublicKeyBytes,
) -> Result<Option<i64>, NotSafe> {
Ok(txn
.query_row(
@@ -229,7 +229,7 @@ impl SlashingDatabase {
fn check_block_proposal(
&self,
txn: &Transaction,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
slot: Slot,
signing_root: SigningRoot,
) -> Result<Safe, NotSafe> {
@@ -278,7 +278,7 @@ impl SlashingDatabase {
fn check_attestation(
&self,
txn: &Transaction,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
att_source_epoch: Epoch,
att_target_epoch: Epoch,
att_signing_root: SigningRoot,
@@ -408,7 +408,7 @@ impl SlashingDatabase {
fn insert_block_proposal(
&self,
txn: &Transaction,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
slot: Slot,
signing_root: SigningRoot,
) -> Result<(), NotSafe> {
@@ -429,7 +429,7 @@ impl SlashingDatabase {
fn insert_attestation(
&self,
txn: &Transaction,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
att_source_epoch: Epoch,
att_target_epoch: Epoch,
att_signing_root: SigningRoot,
@@ -457,7 +457,7 @@ impl SlashingDatabase {
/// This is the safe, externally-callable interface for checking block proposals.
pub fn check_and_insert_block_proposal(
&self,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
block_header: &BeaconBlockHeader,
domain: Hash256,
) -> Result<Safe, NotSafe> {
@@ -471,7 +471,7 @@ impl SlashingDatabase {
/// As for `check_and_insert_block_proposal` but without requiring the whole `BeaconBlockHeader`.
pub fn check_and_insert_block_signing_root(
&self,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
slot: Slot,
signing_root: SigningRoot,
) -> Result<Safe, NotSafe> {
@@ -490,7 +490,7 @@ impl SlashingDatabase {
/// Transactional variant of `check_and_insert_block_signing_root`.
pub fn check_and_insert_block_signing_root_txn(
&self,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
slot: Slot,
signing_root: SigningRoot,
txn: &Transaction,
@@ -511,7 +511,7 @@ impl SlashingDatabase {
/// This is the safe, externally-callable interface for checking attestations.
pub fn check_and_insert_attestation(
&self,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
attestation: &AttestationData,
domain: Hash256,
) -> Result<Safe, NotSafe> {
@@ -527,7 +527,7 @@ impl SlashingDatabase {
/// As for `check_and_insert_attestation` but without requiring the whole `AttestationData`.
pub fn check_and_insert_attestation_signing_root(
&self,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
att_source_epoch: Epoch,
att_target_epoch: Epoch,
att_signing_root: SigningRoot,
@@ -548,7 +548,7 @@ impl SlashingDatabase {
/// Transactional variant of `check_and_insert_attestation_signing_root`.
fn check_and_insert_attestation_signing_root_txn(
&self,
validator_pubkey: &PublicKey,
validator_pubkey: &PublicKeyBytes,
att_source_epoch: Epoch,
att_target_epoch: Epoch,
att_signing_root: SigningRoot,
@@ -600,7 +600,7 @@ impl SlashingDatabase {
let mut import_outcomes = vec![];
for record in interchange.data {
let pubkey = record.pubkey.clone();
let pubkey = record.pubkey;
let txn = conn.transaction()?;
match self.import_interchange_record(record, &txn) {
Ok(summary) => {
@@ -757,7 +757,7 @@ impl SlashingDatabase {
/// Remove all blocks for `public_key` with slots less than `new_min_slot`.
fn prune_signed_blocks(
&self,
public_key: &PublicKey,
public_key: &PublicKeyBytes,
new_min_slot: Slot,
txn: &Transaction,
) -> Result<(), NotSafe> {
@@ -780,7 +780,7 @@ impl SlashingDatabase {
/// Prune the signed blocks table for the given public keys.
pub fn prune_all_signed_blocks<'a>(
&self,
mut public_keys: impl Iterator<Item = &'a PublicKey>,
mut public_keys: impl Iterator<Item = &'a PublicKeyBytes>,
new_min_slot: Slot,
) -> Result<(), NotSafe> {
let mut conn = self.conn_pool.get()?;
@@ -803,7 +803,7 @@ impl SlashingDatabase {
/// attestations in the database.
fn prune_signed_attestations(
&self,
public_key: &PublicKey,
public_key: &PublicKeyBytes,
new_min_target: Epoch,
txn: &Transaction,
) -> Result<(), NotSafe> {
@@ -830,7 +830,7 @@ impl SlashingDatabase {
/// Prune the signed attestations table for the given validator keys.
pub fn prune_all_signed_attestations<'a>(
&self,
mut public_keys: impl Iterator<Item = &'a PublicKey>,
mut public_keys: impl Iterator<Item = &'a PublicKeyBytes>,
new_min_target: Epoch,
) -> Result<(), NotSafe> {
let mut conn = self.conn_pool.get()?;
@@ -853,7 +853,7 @@ impl SlashingDatabase {
/// Get a summary of a validator's slashing protection data for consumption by the user.
pub fn validator_summary(
&self,
public_key: &PublicKey,
public_key: &PublicKeyBytes,
txn: &Transaction,
) -> Result<ValidatorSummary, NotSafe> {
let validator_id = self.get_validator_id_in_txn(txn, public_key)?;
@@ -906,11 +906,11 @@ pub struct ValidatorSummary {
#[derive(Debug)]
pub enum InterchangeImportOutcome {
Success {
pubkey: PublicKey,
pubkey: PublicKeyBytes,
summary: ValidatorSummary,
},
Failure {
pubkey: PublicKey,
pubkey: PublicKeyBytes,
error: NotSafe,
},
}
@@ -981,7 +981,7 @@ mod tests {
let _db1 = SlashingDatabase::create(&file).unwrap();
let db2 = SlashingDatabase::open(&file).unwrap();
db2.register_validator(&pubkey(0)).unwrap_err();
db2.register_validator(pubkey(0)).unwrap_err();
}
// Attempting to create the same database twice should error.

15
validator_client/slashing_protection/src/test_utils.rs
View File

@@ -2,18 +2,19 @@ use crate::*;
use tempfile::{tempdir, TempDir};
use types::{
test_utils::generate_deterministic_keypair, AttestationData, BeaconBlockHeader, Hash256,
PublicKeyBytes,
};
pub const DEFAULT_VALIDATOR_INDEX: usize = 0;
pub const DEFAULT_DOMAIN: Hash256 = Hash256::zero();
pub const DEFAULT_GENESIS_VALIDATORS_ROOT: Hash256 = Hash256::zero();
pub fn pubkey(index: usize) -> PublicKey {
generate_deterministic_keypair(index).pk
pub fn pubkey(index: usize) -> PublicKeyBytes {
generate_deterministic_keypair(index).pk.compress()
}
pub struct Test<T> {
pubkey: PublicKey,
pubkey: PublicKeyBytes,
data: T,
domain: Hash256,
expected: Result<Safe, NotSafe>,
@@ -24,7 +25,7 @@ impl<T> Test<T> {
Self::with_pubkey(pubkey(DEFAULT_VALIDATOR_INDEX), data)
}
pub fn with_pubkey(pubkey: PublicKey, data: T) -> Self {
pub fn with_pubkey(pubkey: PublicKeyBytes, data: T) -> Self {
Self {
pubkey,
data,
@@ -58,7 +59,7 @@ impl<T> Test<T> {
pub struct StreamTest<T> {
/// Validators to register.
pub registered_validators: Vec<PublicKey>,
pub registered_validators: Vec<PublicKeyBytes>,
/// Vector of cases and the value expected when calling `check_and_insert_X`.
pub cases: Vec<Test<T>>,
}
@@ -89,7 +90,7 @@ impl StreamTest<AttestationData> {
let slashing_db = SlashingDatabase::create(&slashing_db_file).unwrap();
for pubkey in &self.registered_validators {
slashing_db.register_validator(pubkey).unwrap();
slashing_db.register_validator(*pubkey).unwrap();
}
for (i, test) in self.cases.iter().enumerate() {
@@ -112,7 +113,7 @@ impl StreamTest<BeaconBlockHeader> {
let slashing_db = SlashingDatabase::create(&slashing_db_file).unwrap();
for pubkey in &self.registered_validators {
slashing_db.register_validator(pubkey).unwrap();
slashing_db.register_validator(*pubkey).unwrap();
}
for (i, test) in self.cases.iter().enumerate() {