mirror of
https://github.com/sigp/lighthouse.git
synced 2026-03-09 03:31:45 +00:00
## Issue Addressed Closes #2052 ## Proposed Changes - Refactor the attester/proposer duties endpoints in the BN - Performance improvements - Fixes some potential inconsistencies with the dependent root fields. - Removes `http_api::beacon_proposer_cache` and just uses the one on the `BeaconChain` instead. - Move the code for the proposer/attester duties endpoints into separate files, for readability. - Refactor the `DutiesService` in the VC - Required to reduce the delay on broadcasting new blocks. - Gets rid of the `ValidatorDuty` shim struct that came about when we adopted the standard API. - Separate block/attestation duty tasks so that they don't block each other when one is slow. - In the VC, use `PublicKeyBytes` to represent validators instead of `PublicKey`. `PublicKey` is a legit crypto object whilst `PublicKeyBytes` is just a byte-array, it's much faster to clone/hash `PublicKeyBytes` and this change has had a significant impact on runtimes. - Unfortunately this has created lots of dust changes. - In the BN, store `PublicKeyBytes` in the `beacon_proposer_cache` and allow access to them. The HTTP API always sends `PublicKeyBytes` over the wire and the conversion from `PublicKey` -> `PublickeyBytes` is non-trivial, especially when queries have 100s/1000s of validators (like Pyrmont). - Add the `state_processing::state_advance` mod which dedups a lot of the "apply `n` skip slots to the state" code. - This also fixes a bug with some functions which were failing to include a state root as per [this comment](072695284f/consensus/state_processing/src/state_advance.rs (L69-L74)). I couldn't find any instance of this bug that resulted in anything more severe than keying a shuffling cache by the wrong block root. - Swap the VC block service to use `mpsc` from `tokio` instead of `futures`. This is consistent with the rest of the code base. ~~This PR *reduces* the size of the codebase 🎉~~ It *used* to reduce the size of the code base before I added more comments. ## Observations on Prymont - Proposer duties times down from peaks of 450ms to consistent <1ms. - Current epoch attester duties times down from >1s peaks to a consistent 20-30ms. - Block production down from +600ms to 100-200ms. ## Additional Info - ~~Blocked on #2241~~ - ~~Blocked on #2234~~ ## TODO - [x] ~~Refactor this into some smaller PRs?~~ Leaving this as-is for now. - [x] Address `per_slot_processing` roots. - [x] Investigate slow next epoch times. Not getting added to cache on block processing? - [x] Consider [this](072695284f/beacon_node/store/src/hot_cold_store.rs (L811-L812)) in the scenario of replacing the state roots Co-authored-by: pawan <pawandhananjay@gmail.com> Co-authored-by: Michael Sproul <michael@sigmaprime.io>
128 lines
4.5 KiB
Rust
128 lines
4.5 KiB
Rust
//! The `BeaconProposer` cache stores the proposer indices for some epoch.
|
|
//!
|
|
//! This cache is keyed by `(epoch, block_root)` where `block_root` is the block root at
|
|
//! `end_slot(epoch - 1)`. We make the assertion that the proposer shuffling is identical for all
|
|
//! blocks in `epoch` which share the common ancestor of `block_root`.
|
|
//!
|
|
//! The cache is a fairly unintelligent LRU cache that is not pruned after finality. This makes it
|
|
//! very simple to reason about, but it might store values that are useless due to finalization. The
|
|
//! values it stores are very small, so this should not be an issue.
|
|
|
|
use lru::LruCache;
|
|
use smallvec::SmallVec;
|
|
use types::{BeaconStateError, Epoch, EthSpec, Fork, Hash256, Slot, Unsigned};
|
|
|
|
/// The number of sets of proposer indices that should be cached.
|
|
const CACHE_SIZE: usize = 16;
|
|
|
|
/// This value is fairly unimportant, it's used to avoid heap allocations. The result of it being
|
|
/// incorrect is non-substantial from a consensus perspective (and probably also from a
|
|
/// performance perspective).
|
|
const TYPICAL_SLOTS_PER_EPOCH: usize = 32;
|
|
|
|
/// For some given slot, this contains the proposer index (`index`) and the `fork` that should be
|
|
/// used to verify their signature.
|
|
pub struct Proposer {
|
|
pub index: usize,
|
|
pub fork: Fork,
|
|
}
|
|
|
|
/// The list of proposers for some given `epoch`, alongside the `fork` that should be used to verify
|
|
/// their signatures.
|
|
pub struct EpochBlockProposers {
|
|
/// The epoch to which the proposers pertain.
|
|
epoch: Epoch,
|
|
/// The fork that should be used to verify proposer signatures.
|
|
fork: Fork,
|
|
/// A list of length `T::EthSpec::slots_per_epoch()`, representing the proposers for each slot
|
|
/// in that epoch.
|
|
///
|
|
/// E.g., if `self.epoch == 1`, then `self.proposers[0]` contains the proposer for slot `32`.
|
|
proposers: SmallVec<[usize; TYPICAL_SLOTS_PER_EPOCH]>,
|
|
}
|
|
|
|
/// A cache to store the proposers for some epoch.
|
|
///
|
|
/// See the module-level documentation for more information.
|
|
pub struct BeaconProposerCache {
|
|
cache: LruCache<(Epoch, Hash256), EpochBlockProposers>,
|
|
}
|
|
|
|
impl Default for BeaconProposerCache {
|
|
fn default() -> Self {
|
|
Self {
|
|
cache: LruCache::new(CACHE_SIZE),
|
|
}
|
|
}
|
|
}
|
|
|
|
impl BeaconProposerCache {
|
|
/// If it is cached, returns the proposer for the block at `slot` where the block has the
|
|
/// ancestor block root of `shuffling_decision_block` at `end_slot(slot.epoch() - 1)`.
|
|
pub fn get_slot<T: EthSpec>(
|
|
&mut self,
|
|
shuffling_decision_block: Hash256,
|
|
slot: Slot,
|
|
) -> Option<Proposer> {
|
|
let epoch = slot.epoch(T::slots_per_epoch());
|
|
let key = (epoch, shuffling_decision_block);
|
|
if let Some(cache) = self.cache.get(&key) {
|
|
// This `if` statement is likely unnecessary, but it feels like good practice.
|
|
if epoch == cache.epoch {
|
|
cache
|
|
.proposers
|
|
.get(slot.as_usize() % T::SlotsPerEpoch::to_usize())
|
|
.map(|&index| Proposer {
|
|
index,
|
|
fork: cache.fork,
|
|
})
|
|
} else {
|
|
None
|
|
}
|
|
} else {
|
|
None
|
|
}
|
|
}
|
|
|
|
/// As per `Self::get_slot`, but returns all proposers in all slots for the given `epoch`.
|
|
///
|
|
/// The nth slot in the returned `SmallVec` will be equal to the nth slot in the given `epoch`.
|
|
/// E.g., if `epoch == 1` then `smallvec[0]` refers to slot 32 (assuming `SLOTS_PER_EPOCH ==
|
|
/// 32`).
|
|
pub fn get_epoch<T: EthSpec>(
|
|
&mut self,
|
|
shuffling_decision_block: Hash256,
|
|
epoch: Epoch,
|
|
) -> Option<&SmallVec<[usize; TYPICAL_SLOTS_PER_EPOCH]>> {
|
|
let key = (epoch, shuffling_decision_block);
|
|
self.cache.get(&key).map(|cache| &cache.proposers)
|
|
}
|
|
|
|
/// Insert the proposers into the cache.
|
|
///
|
|
/// See `Self::get` for a description of `shuffling_decision_block`.
|
|
///
|
|
/// The `fork` value must be valid to verify proposer signatures in `epoch`.
|
|
pub fn insert(
|
|
&mut self,
|
|
epoch: Epoch,
|
|
shuffling_decision_block: Hash256,
|
|
proposers: Vec<usize>,
|
|
fork: Fork,
|
|
) -> Result<(), BeaconStateError> {
|
|
let key = (epoch, shuffling_decision_block);
|
|
if !self.cache.contains(&key) {
|
|
self.cache.put(
|
|
key,
|
|
EpochBlockProposers {
|
|
epoch,
|
|
fork,
|
|
proposers: proposers.into(),
|
|
},
|
|
);
|
|
}
|
|
|
|
Ok(())
|
|
}
|
|
}
|