Optimize validator duties (#2243)

## Issue Addressed

Closes #2052

## Proposed Changes

- Refactor the attester/proposer duties endpoints in the BN
    - Performance improvements
    - Fixes some potential inconsistencies with the dependent root fields.
    - Removes `http_api::beacon_proposer_cache` and just uses the one on the `BeaconChain` instead.
    - Move the code for the proposer/attester duties endpoints into separate files, for readability.
- Refactor the `DutiesService` in the VC
    - Required to reduce the delay on broadcasting new blocks.
    - Gets rid of the `ValidatorDuty` shim struct that came about when we adopted the standard API.
    - Separate block/attestation duty tasks so that they don't block each other when one is slow.
- In the VC, use `PublicKeyBytes` to represent validators instead of `PublicKey`. `PublicKey` is a legit crypto object whilst `PublicKeyBytes` is just a byte-array, it's much faster to clone/hash `PublicKeyBytes` and this change has had a significant impact on runtimes.
    - Unfortunately this has created lots of dust changes.
 - In the BN, store `PublicKeyBytes` in the `beacon_proposer_cache` and allow access to them. The HTTP API always sends `PublicKeyBytes` over the wire and the conversion from `PublicKey` -> `PublickeyBytes` is non-trivial, especially when queries have 100s/1000s of validators (like Pyrmont).
 - Add the `state_processing::state_advance` mod which dedups a lot of the "apply `n` skip slots to the state" code.
    - This also fixes a bug with some functions which were failing to include a state root as per [this comment](072695284f/consensus/state_processing/src/state_advance.rs (L69-L74)). I couldn't find any instance of this bug that resulted in anything more severe than keying a shuffling cache by the wrong block root.
 - Swap the VC block service to use `mpsc` from `tokio` instead of `futures`. This is consistent with the rest of the code base.
    
~~This PR *reduces* the size of the codebase 🎉~~ It *used* to reduce the size of the code base before I added more comments. 

## Observations on Prymont

- Proposer duties times down from peaks of 450ms to consistent <1ms.
- Current epoch attester duties times down from >1s peaks to a consistent 20-30ms.
- Block production down from +600ms to 100-200ms.

## Additional Info

- ~~Blocked on #2241~~
- ~~Blocked on #2234~~

## TODO

- [x] ~~Refactor this into some smaller PRs?~~ Leaving this as-is for now.
- [x] Address `per_slot_processing` roots.
- [x] Investigate slow next epoch times. Not getting added to cache on block processing?
- [x] Consider [this](072695284f/beacon_node/store/src/hot_cold_store.rs (L811-L812)) in the scenario of replacing the state roots


Co-authored-by: pawan <pawandhananjay@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>

consensus/state_processing/src/lib.rs

@@ -9,6 +9,7 @@ pub mod genesis;
 pub mod per_block_processing;
 pub mod per_epoch_processing;
 pub mod per_slot_processing;
+pub mod state_advance;
 pub mod test_utils;
 pub mod verify_operation;
 

consensus/state_processing/src/state_advance.rs

@@ -0,0 +1,105 @@
+//! This module contains functions for advancing a `BeaconState` forward some number of slots
+//! without blocks (i.e., skip slots).
+//!
+//! These functions are not in the specification, however they're defined here to reduce code
+//! duplication and protect against some easy-to-make mistakes when performing state advances.
+
+use crate::*;
+use types::{BeaconState, ChainSpec, EthSpec, Hash256, Slot};
+
+#[derive(Debug, PartialEq)]
+pub enum Error {
+    BadTargetSlot { target_slot: Slot, state_slot: Slot },
+    PerSlotProcessing(per_slot_processing::Error),
+    StateRootNotProvided,
+}
+
+/// Advances the `state` to the given `target_slot`, assuming that there were no blocks between
+/// these slots.
+///
+/// ## Errors
+///
+/// - If `state.slot > target_slot`, an error will be returned.
+///
+/// ## Notes
+///
+/// This state advance method is "complete"; it outputs a perfectly valid `BeaconState` and doesn't
+/// do anything hacky like the "partial" method (see `partial_state_advance`).
+pub fn complete_state_advance<T: EthSpec>(
+    state: &mut BeaconState<T>,
+    mut state_root_opt: Option<Hash256>,
+    target_slot: Slot,
+    spec: &ChainSpec,
+) -> Result<(), Error> {
+    check_target_slot(state.slot, target_slot)?;
+
+    while state.slot < target_slot {
+        // Use the initial state root on the first iteration of the loop, then use `None`  for any
+        // future iterations.
+        let state_root_opt = state_root_opt.take();
+
+        per_slot_processing(state, state_root_opt, spec).map_err(Error::PerSlotProcessing)?;
+    }
+
+    Ok(())
+}
+
+/// Advances the `state` to the given `target_slot`, assuming that there were no blocks between
+/// these slots.
+///
+/// This is a "partial" state advance which outputs an **invalid** `BeaconState`. The state is
+/// invalid because the intermediate state roots are not computed. Avoiding computing state roots
+/// saves *a lot* of compute time and can be a useful optimization when a state only needs to be
+/// advanced to obtain proposer/attester shuffling as they are indifferent to state roots.
+///
+/// For clarity, **be careful with this function as it produces invalid states**.
+///
+/// ## Errors
+///
+/// - If `state.slot > target_slot`, an error will be returned.
+/// - If `state_root_opt.is_none()` but the latest block header requires a state root.
+pub fn partial_state_advance<T: EthSpec>(
+    state: &mut BeaconState<T>,
+    state_root_opt: Option<Hash256>,
+    target_slot: Slot,
+    spec: &ChainSpec,
+) -> Result<(), Error> {
+    check_target_slot(state.slot, target_slot)?;
+
+    // The only time that a state root is mandatory is if a block has been applied to the state
+    // without it yet being advanced another slot.
+    //
+    // Failing to provide a state root in this scenario would result in corrupting the
+    // `state.block_roots` array, since the `state.latest_block_header` would contain an invalid
+    // (all-zeros) state root.
+    let mut initial_state_root = Some(if state.slot > state.latest_block_header.slot {
+        state_root_opt.unwrap_or_else(Hash256::zero)
+    } else {
+        state_root_opt.ok_or(Error::StateRootNotProvided)?
+    });
+
+    while state.slot < target_slot {
+        // Use the initial state root on the first iteration of the loop, then use `[0; 32]` for any
+        // later iterations.
+        //
+        // Failing to provide the correct state root on the initial iteration may result in
+        // corrupting the `state.block_roots` array since the latest block header may not be updated
+        // with the correct state root.
+        let state_root = initial_state_root.take().unwrap_or_else(Hash256::zero);
+
+        per_slot_processing(state, Some(state_root), spec).map_err(Error::PerSlotProcessing)?;
+    }
+
+    Ok(())
+}
+
+fn check_target_slot(state_slot: Slot, target_slot: Slot) -> Result<(), Error> {
+    if state_slot > target_slot {
+        Err(Error::BadTargetSlot {
+            target_slot,
+            state_slot,
+        })
+    } else {
+        Ok(())
+    }
+}