Optimize validator duties (#2243)

## Issue Addressed

Closes #2052

## Proposed Changes

- Refactor the attester/proposer duties endpoints in the BN
    - Performance improvements
    - Fixes some potential inconsistencies with the dependent root fields.
    - Removes `http_api::beacon_proposer_cache` and just uses the one on the `BeaconChain` instead.
    - Move the code for the proposer/attester duties endpoints into separate files, for readability.
- Refactor the `DutiesService` in the VC
    - Required to reduce the delay on broadcasting new blocks.
    - Gets rid of the `ValidatorDuty` shim struct that came about when we adopted the standard API.
    - Separate block/attestation duty tasks so that they don't block each other when one is slow.
- In the VC, use `PublicKeyBytes` to represent validators instead of `PublicKey`. `PublicKey` is a legit crypto object whilst `PublicKeyBytes` is just a byte-array, it's much faster to clone/hash `PublicKeyBytes` and this change has had a significant impact on runtimes.
    - Unfortunately this has created lots of dust changes.
 - In the BN, store `PublicKeyBytes` in the `beacon_proposer_cache` and allow access to them. The HTTP API always sends `PublicKeyBytes` over the wire and the conversion from `PublicKey` -> `PublickeyBytes` is non-trivial, especially when queries have 100s/1000s of validators (like Pyrmont).
 - Add the `state_processing::state_advance` mod which dedups a lot of the "apply `n` skip slots to the state" code.
    - This also fixes a bug with some functions which were failing to include a state root as per [this comment](072695284f/consensus/state_processing/src/state_advance.rs (L69-L74)). I couldn't find any instance of this bug that resulted in anything more severe than keying a shuffling cache by the wrong block root.
 - Swap the VC block service to use `mpsc` from `tokio` instead of `futures`. This is consistent with the rest of the code base.
    
~~This PR *reduces* the size of the codebase 🎉~~ It *used* to reduce the size of the code base before I added more comments. 

## Observations on Prymont

- Proposer duties times down from peaks of 450ms to consistent <1ms.
- Current epoch attester duties times down from >1s peaks to a consistent 20-30ms.
- Block production down from +600ms to 100-200ms.

## Additional Info

- ~~Blocked on #2241~~
- ~~Blocked on #2234~~

## TODO

- [x] ~~Refactor this into some smaller PRs?~~ Leaving this as-is for now.
- [x] Address `per_slot_processing` roots.
- [x] Investigate slow next epoch times. Not getting added to cache on block processing?
- [x] Consider [this](072695284f/beacon_node/store/src/hot_cold_store.rs (L811-L812)) in the scenario of replacing the state roots


Co-authored-by: pawan <pawandhananjay@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>

validator_client/src/graffiti_file.rs

@@ -5,7 +5,7 @@ use std::io::{prelude::*, BufReader};
 use std::path::PathBuf;
 use std::str::FromStr;
 
-use bls::blst_implementations::PublicKey;
+use bls::blst_implementations::PublicKeyBytes;
 use types::{graffiti::GraffitiString, Graffiti};
 
 #[derive(Debug)]
@@ -26,7 +26,7 @@ pub enum Error {
 #[derive(Debug, Clone, Serialize, Deserialize)]
 pub struct GraffitiFile {
     graffiti_path: PathBuf,
-    graffitis: HashMap<PublicKey, Graffiti>,
+    graffitis: HashMap<PublicKeyBytes, Graffiti>,
     default: Option<Graffiti>,
 }
 
@@ -44,7 +44,10 @@ impl GraffitiFile {
     /// default graffiti.
     ///
     /// Returns an error if loading from the graffiti file fails.
-    pub fn load_graffiti(&mut self, public_key: &PublicKey) -> Result<Option<Graffiti>, Error> {
+    pub fn load_graffiti(
+        &mut self,
+        public_key: &PublicKeyBytes,
+    ) -> Result<Option<Graffiti>, Error> {
         self.read_graffiti_file()?;
         Ok(self.graffitis.get(public_key).copied().or(self.default))
     }
@@ -78,7 +81,7 @@ impl GraffitiFile {
 /// `Ok((None, graffiti))` represents the graffiti for the default key.
 /// `Ok((Some(pk), graffiti))` represents graffiti for the public key `pk`.
 /// Returns an error if the line is in the wrong format or does not contain a valid public key or graffiti.
-fn read_line(line: &str) -> Result<(Option<PublicKey>, Graffiti), Error> {
+fn read_line(line: &str) -> Result<(Option<PublicKeyBytes>, Graffiti), Error> {
     if let Some(i) = line.find(':') {
         let (key, value) = line.split_at(i);
         // Note: `value.len() >=1` so `value[1..]` is safe
@@ -88,7 +91,7 @@ fn read_line(line: &str) -> Result<(Option<PublicKey>, Graffiti), Error> {
         if key == "default" {
             Ok((None, graffiti))
         } else {
-            let pk = PublicKey::from_str(&key).map_err(Error::InvalidPublicKey)?;
+            let pk = PublicKeyBytes::from_str(&key).map_err(Error::InvalidPublicKey)?;
             Ok((Some(pk), graffiti))
         }
     } else {
@@ -114,9 +117,9 @@ mod tests {
     // Create a graffiti file in the required format and return a path to the file.
     fn create_graffiti_file() -> PathBuf {
         let temp = TempDir::new().unwrap();
-        let pk1 = PublicKey::deserialize(&hex::decode(&PK1[2..]).unwrap()).unwrap();
-        let pk2 = PublicKey::deserialize(&hex::decode(&PK2[2..]).unwrap()).unwrap();
-        let pk3 = PublicKey::deserialize(&hex::decode(&PK3[2..]).unwrap()).unwrap();
+        let pk1 = PublicKeyBytes::deserialize(&hex::decode(&PK1[2..]).unwrap()).unwrap();
+        let pk2 = PublicKeyBytes::deserialize(&hex::decode(&PK2[2..]).unwrap()).unwrap();
+        let pk3 = PublicKeyBytes::deserialize(&hex::decode(&PK3[2..]).unwrap()).unwrap();
 
         let file_name = temp.into_path().join("graffiti.txt");
 
@@ -143,9 +146,9 @@ mod tests {
         let graffiti_file_path = create_graffiti_file();
         let mut gf = GraffitiFile::new(graffiti_file_path);
 
-        let pk1 = PublicKey::deserialize(&hex::decode(&PK1[2..]).unwrap()).unwrap();
-        let pk2 = PublicKey::deserialize(&hex::decode(&PK2[2..]).unwrap()).unwrap();
-        let pk3 = PublicKey::deserialize(&hex::decode(&PK3[2..]).unwrap()).unwrap();
+        let pk1 = PublicKeyBytes::deserialize(&hex::decode(&PK1[2..]).unwrap()).unwrap();
+        let pk2 = PublicKeyBytes::deserialize(&hex::decode(&PK2[2..]).unwrap()).unwrap();
+        let pk3 = PublicKeyBytes::deserialize(&hex::decode(&PK3[2..]).unwrap()).unwrap();
 
         // Read once
         gf.read_graffiti_file().unwrap();
@@ -165,7 +168,7 @@ mod tests {
         );
 
         // Random pk should return the default graffiti
-        let random_pk = Keypair::random().pk;
+        let random_pk = Keypair::random().pk.compress();
         assert_eq!(
             gf.load_graffiti(&random_pk).unwrap().unwrap(),
             GraffitiString::from_str(DEFAULT_GRAFFITI).unwrap().into()