Optimize validator duties (#2243)

## Issue Addressed

Closes #2052

## Proposed Changes

- Refactor the attester/proposer duties endpoints in the BN
    - Performance improvements
    - Fixes some potential inconsistencies with the dependent root fields.
    - Removes `http_api::beacon_proposer_cache` and just uses the one on the `BeaconChain` instead.
    - Move the code for the proposer/attester duties endpoints into separate files, for readability.
- Refactor the `DutiesService` in the VC
    - Required to reduce the delay on broadcasting new blocks.
    - Gets rid of the `ValidatorDuty` shim struct that came about when we adopted the standard API.
    - Separate block/attestation duty tasks so that they don't block each other when one is slow.
- In the VC, use `PublicKeyBytes` to represent validators instead of `PublicKey`. `PublicKey` is a legit crypto object whilst `PublicKeyBytes` is just a byte-array, it's much faster to clone/hash `PublicKeyBytes` and this change has had a significant impact on runtimes.
    - Unfortunately this has created lots of dust changes.
 - In the BN, store `PublicKeyBytes` in the `beacon_proposer_cache` and allow access to them. The HTTP API always sends `PublicKeyBytes` over the wire and the conversion from `PublicKey` -> `PublickeyBytes` is non-trivial, especially when queries have 100s/1000s of validators (like Pyrmont).
 - Add the `state_processing::state_advance` mod which dedups a lot of the "apply `n` skip slots to the state" code.
    - This also fixes a bug with some functions which were failing to include a state root as per [this comment](072695284f/consensus/state_processing/src/state_advance.rs (L69-L74)). I couldn't find any instance of this bug that resulted in anything more severe than keying a shuffling cache by the wrong block root.
 - Swap the VC block service to use `mpsc` from `tokio` instead of `futures`. This is consistent with the rest of the code base.
    
~~This PR *reduces* the size of the codebase 🎉~~ It *used* to reduce the size of the code base before I added more comments. 

## Observations on Prymont

- Proposer duties times down from peaks of 450ms to consistent <1ms.
- Current epoch attester duties times down from >1s peaks to a consistent 20-30ms.
- Block production down from +600ms to 100-200ms.

## Additional Info

- ~~Blocked on #2241~~
- ~~Blocked on #2234~~

## TODO

- [x] ~~Refactor this into some smaller PRs?~~ Leaving this as-is for now.
- [x] Address `per_slot_processing` roots.
- [x] Investigate slow next epoch times. Not getting added to cache on block processing?
- [x] Consider [this](072695284f/beacon_node/store/src/hot_cold_store.rs (L811-L812)) in the scenario of replacing the state roots


Co-authored-by: pawan <pawandhananjay@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>

validator_client/src/validator_store.rs

@@ -11,7 +11,7 @@ use std::sync::Arc;
 use tempfile::TempDir;
 use types::{
     graffiti::GraffitiString, Attestation, BeaconBlock, ChainSpec, Domain, Epoch, EthSpec, Fork,
-    Graffiti, Hash256, Keypair, PublicKey, SelectionProof, Signature, SignedAggregateAndProof,
+    Graffiti, Hash256, Keypair, PublicKeyBytes, SelectionProof, Signature, SignedAggregateAndProof,
     SignedBeaconBlock, SignedRoot, Slot,
 };
 use validator_dir::ValidatorDir;
@@ -106,7 +106,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
         .map_err(|e| format!("failed to create validator definitions: {:?}", e))?;
 
         self.slashing_protection
-            .register_validator(&validator_def.voting_public_key)
+            .register_validator(validator_def.voting_public_key.compress())
             .map_err(|e| format!("failed to register validator: {:?}", e))?;
 
         validator_def.enabled = enable;
@@ -120,7 +120,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
         Ok(validator_def)
     }
 
-    pub fn voting_pubkeys(&self) -> Vec<PublicKey> {
+    pub fn voting_pubkeys(&self) -> Vec<PublicKeyBytes> {
         self.validators
             .read()
             .iter_voting_pubkeys()
@@ -136,7 +136,11 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
         self.fork_service.fork()
     }
 
-    pub fn randao_reveal(&self, validator_pubkey: &PublicKey, epoch: Epoch) -> Option<Signature> {
+    pub fn randao_reveal(
+        &self,
+        validator_pubkey: &PublicKeyBytes,
+        epoch: Epoch,
+    ) -> Option<Signature> {
         self.validators
             .read()
             .voting_keypair(validator_pubkey)
@@ -153,13 +157,13 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
             })
     }
 
-    pub fn graffiti(&self, validator_pubkey: &PublicKey) -> Option<Graffiti> {
+    pub fn graffiti(&self, validator_pubkey: &PublicKeyBytes) -> Option<Graffiti> {
         self.validators.read().graffiti(validator_pubkey)
     }
 
     pub fn sign_block(
         &self,
-        validator_pubkey: &PublicKey,
+        validator_pubkey: &PublicKeyBytes,
         block: BeaconBlock<E>,
         current_slot: Slot,
     ) -> Option<SignedBeaconBlock<E>> {
@@ -236,7 +240,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
 
     pub fn sign_attestation(
         &self,
-        validator_pubkey: &PublicKey,
+        validator_pubkey: &PublicKeyBytes,
         validator_committee_position: usize,
         attestation: &mut Attestation<E>,
         current_epoch: Epoch,
@@ -334,7 +338,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
     /// modified by actors other than the signing validator.
     pub fn produce_signed_aggregate_and_proof(
         &self,
-        validator_pubkey: &PublicKey,
+        validator_pubkey: &PublicKeyBytes,
         validator_index: u64,
         aggregate: Attestation<E>,
         selection_proof: SelectionProof,
@@ -359,7 +363,7 @@ impl<T: SlotClock + 'static, E: EthSpec> ValidatorStore<T, E> {
     /// `validator_pubkey`.
     pub fn produce_selection_proof(
         &self,
-        validator_pubkey: &PublicKey,
+        validator_pubkey: &PublicKeyBytes,
         slot: Slot,
     ) -> Option<SelectionProof> {
         let validators = self.validators.read();