Add configurable block replayer (#2863)

## Issue Addressed

Successor to #2431

## Proposed Changes

* Add a `BlockReplayer` struct to abstract over the intricacies of calling `per_slot_processing` and `per_block_processing` while avoiding unnecessary tree hashing.
* Add a variant of the forwards state root iterator that does not require an `end_state`.
* Use the `BlockReplayer` when reconstructing states in the database. Use the efficient forwards iterator for frozen states.
* Refactor the iterators to remove `Arc<HotColdDB>` (this seems to be neater than making _everything_ an `Arc<HotColdDB>` as I did in #2431).

Supplying the state roots allow us to avoid building a tree hash cache at all when reconstructing historic states, which saves around 1 second flat (regardless of `slots-per-restore-point`). This is a small percentage of worst-case state load times with 200K validators and SPRP=2048 (~15s vs ~16s) but a significant speed-up for more frequent restore points: state loads with SPRP=32 should be now consistently <500ms instead of 1.5s (a ~3x speedup).

## Additional Info

Required by https://github.com/sigp/lighthouse/pull/2628

beacon_node/store/src/reconstruct.rs

@@ -3,7 +3,9 @@ use crate::hot_cold_store::{HotColdDB, HotColdDBError};
 use crate::{Error, ItemStore, KeyValueStore};
 use itertools::{process_results, Itertools};
 use slog::info;
-use state_processing::{per_block_processing, per_slot_processing, BlockSignatureStrategy};
+use state_processing::{
+    per_block_processing, per_slot_processing, BlockSignatureStrategy, VerifyBlockRoot,
+};
 use std::sync::Arc;
 use types::{EthSpec, Hash256};
 
@@ -48,8 +50,7 @@ where
         // Use a dummy root, as we never read the block for the upper limit state.
         let upper_limit_block_root = Hash256::repeat_byte(0xff);
 
-        let block_root_iter = Self::forwards_block_roots_iterator(
-            self.clone(),
+        let block_root_iter = self.forwards_block_roots_iterator(
             lower_limit_slot,
             upper_limit_state,
             upper_limit_block_root,
@@ -91,6 +92,7 @@ where
                         &block,
                         Some(block_root),
                         BlockSignatureStrategy::NoVerification,
+                        VerifyBlockRoot::True,
                         &self.spec,
                     )
                     .map_err(HotColdDBError::BlockReplayBlockError)?;