mirror of
https://github.com/sigp/lighthouse.git
synced 2026-03-09 03:31:45 +00:00
## Issue Addressed Closes #2052 ## Proposed Changes - Refactor the attester/proposer duties endpoints in the BN - Performance improvements - Fixes some potential inconsistencies with the dependent root fields. - Removes `http_api::beacon_proposer_cache` and just uses the one on the `BeaconChain` instead. - Move the code for the proposer/attester duties endpoints into separate files, for readability. - Refactor the `DutiesService` in the VC - Required to reduce the delay on broadcasting new blocks. - Gets rid of the `ValidatorDuty` shim struct that came about when we adopted the standard API. - Separate block/attestation duty tasks so that they don't block each other when one is slow. - In the VC, use `PublicKeyBytes` to represent validators instead of `PublicKey`. `PublicKey` is a legit crypto object whilst `PublicKeyBytes` is just a byte-array, it's much faster to clone/hash `PublicKeyBytes` and this change has had a significant impact on runtimes. - Unfortunately this has created lots of dust changes. - In the BN, store `PublicKeyBytes` in the `beacon_proposer_cache` and allow access to them. The HTTP API always sends `PublicKeyBytes` over the wire and the conversion from `PublicKey` -> `PublickeyBytes` is non-trivial, especially when queries have 100s/1000s of validators (like Pyrmont). - Add the `state_processing::state_advance` mod which dedups a lot of the "apply `n` skip slots to the state" code. - This also fixes a bug with some functions which were failing to include a state root as per [this comment](072695284f/consensus/state_processing/src/state_advance.rs (L69-L74)). I couldn't find any instance of this bug that resulted in anything more severe than keying a shuffling cache by the wrong block root. - Swap the VC block service to use `mpsc` from `tokio` instead of `futures`. This is consistent with the rest of the code base. ~~This PR *reduces* the size of the codebase 🎉~~ It *used* to reduce the size of the code base before I added more comments. ## Observations on Prymont - Proposer duties times down from peaks of 450ms to consistent <1ms. - Current epoch attester duties times down from >1s peaks to a consistent 20-30ms. - Block production down from +600ms to 100-200ms. ## Additional Info - ~~Blocked on #2241~~ - ~~Blocked on #2234~~ ## TODO - [x] ~~Refactor this into some smaller PRs?~~ Leaving this as-is for now. - [x] Address `per_slot_processing` roots. - [x] Investigate slow next epoch times. Not getting added to cache on block processing? - [x] Consider [this](072695284f/beacon_node/store/src/hot_cold_store.rs (L811-L812)) in the scenario of replacing the state roots Co-authored-by: pawan <pawandhananjay@gmail.com> Co-authored-by: Michael Sproul <michael@sigmaprime.io>
77 lines
2.5 KiB
Rust
77 lines
2.5 KiB
Rust
use serde_derive::{Deserialize, Serialize};
|
|
use std::collections::HashSet;
|
|
use types::{Epoch, Hash256, PublicKeyBytes, Slot};
|
|
|
|
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize)]
|
|
#[serde(deny_unknown_fields)]
|
|
pub struct InterchangeMetadata {
|
|
#[serde(with = "serde_utils::quoted_u64::require_quotes")]
|
|
pub interchange_format_version: u64,
|
|
pub genesis_validators_root: Hash256,
|
|
}
|
|
|
|
#[derive(Debug, Clone, PartialEq, Eq, Hash, Deserialize, Serialize)]
|
|
#[serde(deny_unknown_fields)]
|
|
pub struct InterchangeData {
|
|
pub pubkey: PublicKeyBytes,
|
|
pub signed_blocks: Vec<SignedBlock>,
|
|
pub signed_attestations: Vec<SignedAttestation>,
|
|
}
|
|
|
|
#[derive(Debug, Clone, PartialEq, Eq, Hash, Deserialize, Serialize)]
|
|
#[serde(deny_unknown_fields)]
|
|
pub struct SignedBlock {
|
|
#[serde(with = "serde_utils::quoted_u64::require_quotes")]
|
|
pub slot: Slot,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub signing_root: Option<Hash256>,
|
|
}
|
|
|
|
#[derive(Debug, Clone, PartialEq, Eq, Hash, Deserialize, Serialize)]
|
|
#[serde(deny_unknown_fields)]
|
|
pub struct SignedAttestation {
|
|
#[serde(with = "serde_utils::quoted_u64::require_quotes")]
|
|
pub source_epoch: Epoch,
|
|
#[serde(with = "serde_utils::quoted_u64::require_quotes")]
|
|
pub target_epoch: Epoch,
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub signing_root: Option<Hash256>,
|
|
}
|
|
|
|
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize)]
|
|
pub struct Interchange {
|
|
pub metadata: InterchangeMetadata,
|
|
pub data: Vec<InterchangeData>,
|
|
}
|
|
|
|
impl Interchange {
|
|
pub fn from_json_str(json: &str) -> Result<Self, serde_json::Error> {
|
|
serde_json::from_str(json)
|
|
}
|
|
|
|
pub fn from_json_reader(reader: impl std::io::Read) -> Result<Self, serde_json::Error> {
|
|
serde_json::from_reader(reader)
|
|
}
|
|
|
|
pub fn write_to(&self, writer: impl std::io::Write) -> Result<(), serde_json::Error> {
|
|
serde_json::to_writer(writer, self)
|
|
}
|
|
|
|
/// Do these two `Interchange`s contain the same data (ignoring ordering)?
|
|
pub fn equiv(&self, other: &Self) -> bool {
|
|
let self_set = self.data.iter().collect::<HashSet<_>>();
|
|
let other_set = other.data.iter().collect::<HashSet<_>>();
|
|
self.metadata == other.metadata && self_set == other_set
|
|
}
|
|
|
|
/// The number of entries in `data`.
|
|
pub fn len(&self) -> usize {
|
|
self.data.len()
|
|
}
|
|
|
|
/// Is the `data` part of the interchange completely empty?
|
|
pub fn is_empty(&self) -> bool {
|
|
self.len() == 0
|
|
}
|
|
}
|