Add configurable block replayer (#2863)

## Issue Addressed

Successor to #2431

## Proposed Changes

* Add a `BlockReplayer` struct to abstract over the intricacies of calling `per_slot_processing` and `per_block_processing` while avoiding unnecessary tree hashing.
* Add a variant of the forwards state root iterator that does not require an `end_state`.
* Use the `BlockReplayer` when reconstructing states in the database. Use the efficient forwards iterator for frozen states.
* Refactor the iterators to remove `Arc<HotColdDB>` (this seems to be neater than making _everything_ an `Arc<HotColdDB>` as I did in #2431).

Supplying the state roots allow us to avoid building a tree hash cache at all when reconstructing historic states, which saves around 1 second flat (regardless of `slots-per-restore-point`). This is a small percentage of worst-case state load times with 200K validators and SPRP=2048 (~15s vs ~16s) but a significant speed-up for more frequent restore points: state loads with SPRP=32 should be now consistently <500ms instead of 1.5s (a ~3x speedup).

## Additional Info

Required by https://github.com/sigp/lighthouse/pull/2628

beacon_node/store/src/chunked_iter.rs

@@ -1,27 +1,26 @@
 use crate::chunked_vector::{chunk_key, Chunk, Field};
 use crate::{HotColdDB, ItemStore};
 use slog::error;
-use std::sync::Arc;
 use types::{ChainSpec, EthSpec, Slot};
 
 /// Iterator over the values of a `BeaconState` vector field (like `block_roots`).
 ///
 /// Uses the freezer DB's separate table to load the values.
-pub struct ChunkedVectorIter<F, E, Hot, Cold>
+pub struct ChunkedVectorIter<'a, F, E, Hot, Cold>
 where
     F: Field<E>,
     E: EthSpec,
     Hot: ItemStore<E>,
     Cold: ItemStore<E>,
 {
-    pub(crate) store: Arc<HotColdDB<E, Hot, Cold>>,
+    pub(crate) store: &'a HotColdDB<E, Hot, Cold>,
     current_vindex: usize,
     pub(crate) end_vindex: usize,
     next_cindex: usize,
     current_chunk: Chunk<F::Value>,
 }
 
-impl<F, E, Hot, Cold> ChunkedVectorIter<F, E, Hot, Cold>
+impl<'a, F, E, Hot, Cold> ChunkedVectorIter<'a, F, E, Hot, Cold>
 where
     F: Field<E>,
     E: EthSpec,
@@ -35,7 +34,7 @@ where
     /// `HotColdDB::get_latest_restore_point_slot`. We pass it as a parameter so that the caller can
     /// maintain a stable view of the database (see `HybridForwardsBlockRootsIterator`).
     pub fn new(
-        store: Arc<HotColdDB<E, Hot, Cold>>,
+        store: &'a HotColdDB<E, Hot, Cold>,
         start_vindex: usize,
         last_restore_point_slot: Slot,
         spec: &ChainSpec,
@@ -57,7 +56,7 @@ where
     }
 }
 
-impl<F, E, Hot, Cold> Iterator for ChunkedVectorIter<F, E, Hot, Cold>
+impl<'a, F, E, Hot, Cold> Iterator for ChunkedVectorIter<'a, F, E, Hot, Cold>
 where
     F: Field<E>,
     E: EthSpec,