Initial work towards v0.2.0 (#924)

* Remove ping protocol

* Initial renaming of network services

* Correct rebasing relative to latest master

* Start updating types

* Adds HashMapDelay struct to utils

* Initial network restructure

* Network restructure. Adds new types for v0.2.0

* Removes build artefacts

* Shift validation to beacon chain

* Temporarily remove gossip validation

This is to be updated to match current optimisation efforts.

* Adds AggregateAndProof

* Begin rebuilding pubsub encoding/decoding

* Signature hacking

* Shift gossipsup decoding into eth2_libp2p

* Existing EF tests passing with fake_crypto

* Shifts block encoding/decoding into RPC

* Delete outdated API spec

* All release tests passing bar genesis state parsing

* Update and test YamlConfig

* Update to spec v0.10 compatible BLS

* Updates to BLS EF tests

* Add EF test for AggregateVerify

And delete unused hash2curve tests for uncompressed points

* Update EF tests to v0.10.1

* Use optional block root correctly in block proc

* Use genesis fork in deposit domain. All tests pass

* Fast aggregate verify test

* Update REST API docs

* Fix unused import

* Bump spec tags to v0.10.1

* Add `seconds_per_eth1_block` to chainspec

* Update to timestamp based eth1 voting scheme

* Return None from `get_votes_to_consider` if block cache is empty

* Handle overflows in `is_candidate_block`

* Revert to failing tests

* Fix eth1 data sets test

* Choose default vote according to spec

* Fix collect_valid_votes tests

* Fix `get_votes_to_consider` to choose all eligible blocks

* Uncomment winning_vote tests

* Add comments; remove unused code

* Reduce seconds_per_eth1_block for simulation

* Addressed review comments

* Add test for default vote case

* Fix logs

* Remove unused functions

* Meter default eth1 votes

* Fix comments

* Progress on attestation service

* Address review comments; remove unused dependency

* Initial work on removing libp2p lock

* Add LRU caches to store (rollup)

* Update attestation validation for DB changes (WIP)

* Initial version of should_forward_block

* Scaffold

* Progress on attestation validation

Also, consolidate prod+testing slot clocks so that they share much
of the same implementation and can both handle sub-slot time changes.

* Removes lock from libp2p service

* Completed network lock removal

* Finish(?) attestation processing

* Correct network termination future

* Add slot check to block check

* Correct fmt issues

* Remove Drop implementation for network service

* Add first attempt at attestation proc. re-write

* Add version 2 of attestation processing

* Minor fixes

* Add validator pubkey cache

* Make get_indexed_attestation take a committee

* Link signature processing into new attn verification

* First working version

* Ensure pubkey cache is updated

* Add more metrics, slight optimizations

* Clone committee cache during attestation processing

* Update shuffling cache during block processing

* Remove old commented-out code

* Fix shuffling cache insert bug

* Used indexed attestation in fork choice

* Restructure attn processing, add metrics

* Add more detailed metrics

* Tidy, fix failing tests

* Fix failing tests, tidy

* Address reviewers suggestions

* Disable/delete two outdated tests

* Modification of validator for subscriptions

* Add slot signing to validator client

* Further progress on validation subscription

* Adds necessary validator subscription functionality

* Add new Pubkeys struct to signature_sets

* Refactor with functional approach

* Update beacon chain

* Clean up validator <-> beacon node http types

* Add aggregator status to ValidatorDuty

* Impl Clone for manual slot clock

* Fix minor errors

* Further progress validator client subscription

* Initial subscription and aggregation handling

* Remove decompressed member from pubkey bytes

* Progress to modifying val client for attestation aggregation

* First draft of validator client upgrade for aggregate attestations

* Add hashmap for indices lookup

* Add state cache, remove store cache

* Only build the head committee cache

* Removes lock on a network channel

* Partially implement beacon node subscription http api

* Correct compilation issues

* Change `get_attesting_indices` to use Vec

* Fix failing test

* Partial implementation of timer

* Adds timer, removes exit_future, http api to op pool

* Partial multiple aggregate attestation handling

* Permits bulk messages accross gossipsub network channel

* Correct compile issues

* Improve gosispsub messaging and correct rest api helpers

* Added global gossipsub subscriptions

* Update validator subscriptions data structs

* Tidy

* Re-structure validator subscriptions

* Initial handling of subscriptions

* Re-structure network service

* Add pubkey cache persistence file

* Add more comments

* Integrate persistence file into builder

* Add pubkey cache tests

* Add HashSetDelay and introduce into attestation service

* Handles validator subscriptions

* Add data_dir to beacon chain builder

* Remove Option in pubkey cache persistence file

* Ensure consistency between datadir/data_dir

* Fix failing network test

* Peer subnet discovery gets queued for future subscriptions

* Reorganise attestation service functions

* Initial wiring of attestation service

* First draft of attestation service timing logic

* Correct minor typos

* Tidy

* Fix todos

* Improve tests

* Add PeerInfo to connected peers mapping

* Fix compile error

* Fix compile error from merge

* Split up block processing metrics

* Tidy

* Refactor get_pubkey_from_state

* Remove commented-out code

* Rename state_cache -> checkpoint_cache

* Rename Checkpoint -> Snapshot

* Tidy, add comments

* Tidy up find_head function

* Change some checkpoint -> snapshot

* Add tests

* Expose max_len

* Remove dead code

* Tidy

* Fix bug

* Add sync-speed metric

* Add first attempt at VerifiableBlock

* Start integrating into beacon chain

* Integrate VerifiableBlock

* Rename VerifableBlock -> PartialBlockVerification

* Add start of typed methods

* Add progress

* Add further progress

* Rename structs

* Add full block verification to block_processing.rs

* Further beacon chain integration

* Update checks for gossip

* Add todo

* Start adding segement verification

* Add passing chain segement test

* Initial integration with batch sync

* Minor changes

* Tidy, add more error checking

* Start adding chain_segment tests

* Finish invalid signature tests

* Include single and gossip verified blocks in tests

* Add gossip verification tests

* Start adding docs

* Finish adding comments to block_processing.rs

* Rename block_processing.rs -> block_verification

* Start removing old block processing code

* Fixes beacon_chain compilation

* Fix project-wide compile errors

* Remove old code

* Correct code to pass all tests

* Fix bug with beacon proposer index

* Fix shim for BlockProcessingError

* Only process one epoch at a time

* Fix loop in chain segment processing

* Correct tests from master merge

* Add caching for state.eth1_data_votes

* Add BeaconChain::validator_pubkey

* Revert "Add caching for state.eth1_data_votes"

This reverts commit cd73dcd643.

Co-authored-by: Grant Wuerker <gwuerker@gmail.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Michael Sproul <micsproul@gmail.com>
Co-authored-by: pawan <pawandhananjay@gmail.com>
Co-authored-by: Paul Hauner <paul@paulhauner.com>

beacon_node/beacon_chain/src/beacon_snapshot.rs

@@ -5,14 +5,14 @@ use types::{BeaconState, EthSpec, Hash256, SignedBeaconBlock};
 /// Represents some block and its associated state. Generally, this will be used for tracking the
 /// head, justified head and finalized head.
 #[derive(Clone, Serialize, PartialEq, Debug, Encode, Decode)]
-pub struct CheckPoint<E: EthSpec> {
+pub struct BeaconSnapshot<E: EthSpec> {
     pub beacon_block: SignedBeaconBlock<E>,
     pub beacon_block_root: Hash256,
     pub beacon_state: BeaconState<E>,
     pub beacon_state_root: Hash256,
 }
 
-impl<E: EthSpec> CheckPoint<E> {
+impl<E: EthSpec> BeaconSnapshot<E> {
     /// Create a new checkpoint.
     pub fn new(
         beacon_block: SignedBeaconBlock<E>,

beacon_node/beacon_chain/src/block_verification.rs

@@ -0,0 +1,801 @@
+//! Provides `SignedBeaconBlock` verification logic.
+//!
+//! Specifically, it provides the following:
+//!
+//! - Verification for gossip blocks (i.e., should we gossip some block from the network).
+//! - Verification for normal blocks (e.g., some block received on the RPC during a parent lookup).
+//! - Verification for chain segments (e.g., some chain of blocks received on the RPC during a
+//!    sync).
+//!
+//! The primary source of complexity here is that we wish to avoid doing duplicate work as a block
+//! moves through the verification process. For example, if some block is verified for gossip, we
+//! do not wish to re-verify the block proposal signature or re-hash the block. Or, if we've
+//! verified the signatures of a block during a chain segment import, we do not wish to verify each
+//! signature individually again.
+//!
+//! The incremental processing steps (e.g., signatures verified but not the state transition) is
+//! represented as a sequence of wrapper-types around the block. There is a linear progression of
+//! types, starting at a `SignedBeaconBlock` and finishing with a `Fully VerifiedBlock` (see
+//! diagram below).
+//!
+//! ```ignore
+//!           START
+//!             |
+//!             ▼
+//!     SignedBeaconBlock
+//!             |---------------
+//!             |              |
+//!             |              ▼
+//!             |      GossipVerifiedBlock
+//!             |              |
+//!             |---------------
+//!             |
+//!             ▼
+//!     SignatureVerifiedBlock
+//!             |
+//!             ▼
+//!      FullyVerifiedBlock
+//!             |
+//!             ▼
+//!            END
+//!
+//! ```
+use crate::validator_pubkey_cache::ValidatorPubkeyCache;
+use crate::{
+    beacon_chain::{BLOCK_PROCESSING_CACHE_LOCK_TIMEOUT, VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT},
+    metrics, BeaconChain, BeaconChainError, BeaconChainTypes, BeaconSnapshot,
+};
+use parking_lot::RwLockReadGuard;
+use state_processing::{
+    block_signature_verifier::{
+        BlockSignatureVerifier, Error as BlockSignatureVerifierError, G1Point,
+    },
+    per_block_processing, per_slot_processing, BlockProcessingError, BlockSignatureStrategy,
+    SlotProcessingError,
+};
+use std::borrow::Cow;
+use store::{Error as DBError, StateBatch};
+use types::{
+    BeaconBlock, BeaconState, BeaconStateError, ChainSpec, CloneConfig, EthSpec, Hash256,
+    RelativeEpoch, SignedBeaconBlock, Slot,
+};
+
+mod block_processing_outcome;
+
+pub use block_processing_outcome::BlockProcessingOutcome;
+
+/// Maximum block slot number. Block with slots bigger than this constant will NOT be processed.
+const MAXIMUM_BLOCK_SLOT_NUMBER: u64 = 4_294_967_296; // 2^32
+
+/// Returned when a block was not verified. A block is not verified for two reasons:
+///
+/// - The block is malformed/invalid (indicated by all results other than `BeaconChainError`.
+/// - We encountered an error whilst trying to verify the block (a `BeaconChainError`).
+#[derive(Debug, PartialEq)]
+pub enum BlockError {
+    /// The parent block was unknown.
+    ParentUnknown(Hash256),
+    /// The block slot is greater than the present slot.
+    FutureSlot {
+        present_slot: Slot,
+        block_slot: Slot,
+    },
+    /// The block state_root does not match the generated state.
+    StateRootMismatch { block: Hash256, local: Hash256 },
+    /// The block was a genesis block, these blocks cannot be re-imported.
+    GenesisBlock,
+    /// The slot is finalized, no need to import.
+    WouldRevertFinalizedSlot {
+        block_slot: Slot,
+        finalized_slot: Slot,
+    },
+    /// Block is already known, no need to re-import.
+    BlockIsAlreadyKnown,
+    /// The block slot exceeds the MAXIMUM_BLOCK_SLOT_NUMBER.
+    BlockSlotLimitReached,
+    /// The proposal signature in invalid.
+    ProposalSignatureInvalid,
+    /// A signature in the block is invalid (exactly which is unknown).
+    InvalidSignature,
+    /// The provided block is from an earlier slot than its parent.
+    BlockIsNotLaterThanParent { block_slot: Slot, state_slot: Slot },
+    /// At least one block in the chain segement did not have it's parent root set to the root of
+    /// the prior block.
+    NonLinearParentRoots,
+    /// The slots of the blocks in the chain segment were not strictly increasing. I.e., a child
+    /// had lower slot than a parent.
+    NonLinearSlots,
+    /// The block failed the specification's `per_block_processing` function, it is invalid.
+    PerBlockProcessingError(BlockProcessingError),
+    /// There was an error whilst processing the block. It is not necessarily invalid.
+    BeaconChainError(BeaconChainError),
+}
+
+impl From<BlockSignatureVerifierError> for BlockError {
+    fn from(e: BlockSignatureVerifierError) -> Self {
+        BlockError::BeaconChainError(BeaconChainError::BlockSignatureVerifierError(e))
+    }
+}
+
+impl From<BeaconChainError> for BlockError {
+    fn from(e: BeaconChainError) -> Self {
+        BlockError::BeaconChainError(e)
+    }
+}
+
+impl From<BeaconStateError> for BlockError {
+    fn from(e: BeaconStateError) -> Self {
+        BlockError::BeaconChainError(BeaconChainError::BeaconStateError(e))
+    }
+}
+
+impl From<SlotProcessingError> for BlockError {
+    fn from(e: SlotProcessingError) -> Self {
+        BlockError::BeaconChainError(BeaconChainError::SlotProcessingError(e))
+    }
+}
+
+impl From<DBError> for BlockError {
+    fn from(e: DBError) -> Self {
+        BlockError::BeaconChainError(BeaconChainError::DBError(e))
+    }
+}
+
+/// Verify all signatures (except deposit signatures) on all blocks in the `chain_segment`. If all
+/// signatures are valid, the `chain_segment` is mapped to a `Vec<SignatureVerifiedBlock>` that can
+/// later be transformed into a `FullyVerifiedBlock` without re-checking the signatures. If any
+/// signature in the block is invalid, an `Err` is returned (it is not possible to known _which_
+/// signature was invalid).
+///
+/// ## Errors
+///
+/// The given `chain_segement` must span no more than two epochs, otherwise an error will be
+/// returned.
+pub fn signature_verify_chain_segment<T: BeaconChainTypes>(
+    chain_segment: Vec<(Hash256, SignedBeaconBlock<T::EthSpec>)>,
+    chain: &BeaconChain<T>,
+) -> Result<Vec<SignatureVerifiedBlock<T>>, BlockError> {
+    let (mut parent, slot) = if let Some(block) = chain_segment.first().map(|(_, block)| block) {
+        let parent = load_parent(&block.message, chain)?;
+        (parent, block.slot())
+    } else {
+        return Ok(vec![]);
+    };
+
+    let highest_slot = chain_segment
+        .last()
+        .map(|(_, block)| block.slot())
+        .unwrap_or_else(|| slot);
+
+    let state = cheap_state_advance_to_obtain_committees(
+        &mut parent.beacon_state,
+        highest_slot,
+        &chain.spec,
+    )?;
+
+    let pubkey_cache = get_validator_pubkey_cache(chain)?;
+    let mut signature_verifier = get_signature_verifier(&state, &pubkey_cache, &chain.spec);
+
+    for (block_root, block) in &chain_segment {
+        signature_verifier.include_all_signatures(block, Some(*block_root))?;
+    }
+
+    if signature_verifier.verify().is_err() {
+        return Err(BlockError::InvalidSignature);
+    }
+
+    drop(pubkey_cache);
+
+    let mut signature_verified_blocks = chain_segment
+        .into_iter()
+        .map(|(block_root, block)| SignatureVerifiedBlock {
+            block,
+            block_root,
+            parent: None,
+        })
+        .collect::<Vec<_>>();
+
+    if let Some(signature_verified_block) = signature_verified_blocks.first_mut() {
+        signature_verified_block.parent = Some(parent);
+    }
+
+    Ok(signature_verified_blocks)
+}
+
+/// A wrapper around a `SignedBeaconBlock` that indicates it has been approved for re-gossiping on
+/// the p2p network.
+pub struct GossipVerifiedBlock<T: BeaconChainTypes> {
+    block: SignedBeaconBlock<T::EthSpec>,
+    block_root: Hash256,
+    parent: BeaconSnapshot<T::EthSpec>,
+}
+
+/// A wrapper around a `SignedBeaconBlock` that indicates that all signatures (except the deposit
+/// signatures) have been verified.
+pub struct SignatureVerifiedBlock<T: BeaconChainTypes> {
+    block: SignedBeaconBlock<T::EthSpec>,
+    block_root: Hash256,
+    parent: Option<BeaconSnapshot<T::EthSpec>>,
+}
+
+/// A wrapper around a `SignedBeaconBlock` that indicates that this block is fully verified and
+/// ready to import into the `BeaconChain`. The validation includes:
+///
+/// - Parent is known
+/// - Signatures
+/// - State root check
+/// - Per block processing
+///
+/// Note: a `FullyVerifiedBlock` is not _forever_ valid to be imported, it may later become invalid
+/// due to finality or some other event. A `FullyVerifiedBlock` should be imported into the
+/// `BeaconChain` immediately after it is instantiated.
+pub struct FullyVerifiedBlock<T: BeaconChainTypes> {
+    pub block: SignedBeaconBlock<T::EthSpec>,
+    pub block_root: Hash256,
+    pub state: BeaconState<T::EthSpec>,
+    pub parent_block: SignedBeaconBlock<T::EthSpec>,
+    pub intermediate_states: StateBatch<T::EthSpec>,
+}
+
+/// Implemented on types that can be converted into a `FullyVerifiedBlock`.
+///
+/// Used to allow functions to accept blocks at various stages of verification.
+pub trait IntoFullyVerifiedBlock<T: BeaconChainTypes> {
+    fn into_fully_verified_block(
+        self,
+        chain: &BeaconChain<T>,
+    ) -> Result<FullyVerifiedBlock<T>, BlockError>;
+
+    fn block(&self) -> &SignedBeaconBlock<T::EthSpec>;
+}
+
+impl<T: BeaconChainTypes> GossipVerifiedBlock<T> {
+    /// Instantiates `Self`, a wrapper that indicates the given `block` is safe to be re-gossiped
+    /// on the p2p network.
+    ///
+    /// Returns an error if the block is invalid, or if the block was unable to be verified.
+    pub fn new(
+        block: SignedBeaconBlock<T::EthSpec>,
+        chain: &BeaconChain<T>,
+    ) -> Result<Self, BlockError> {
+        // Do not gossip or process blocks from future slots.
+        //
+        // TODO: adjust this to allow for clock disparity tolerance.
+        let present_slot = chain.slot()?;
+        if block.slot() > present_slot {
+            return Err(BlockError::FutureSlot {
+                present_slot,
+                block_slot: block.slot(),
+            });
+        }
+
+        // Do not gossip a block from a finalized slot.
+        //
+        // TODO: adjust this to allow for clock disparity tolerance.
+        check_block_against_finalized_slot(&block.message, chain)?;
+
+        // TODO: add check for the `(block.proposer_index, block.slot)` tuple once we have v0.11.0
+
+        let mut parent = load_parent(&block.message, chain)?;
+        let block_root = get_block_root(&block);
+
+        let state = cheap_state_advance_to_obtain_committees(
+            &mut parent.beacon_state,
+            block.slot(),
+            &chain.spec,
+        )?;
+
+        let pubkey_cache = get_validator_pubkey_cache(chain)?;
+
+        let mut signature_verifier = get_signature_verifier(&state, &pubkey_cache, &chain.spec);
+
+        signature_verifier.include_block_proposal(&block, Some(block_root))?;
+
+        if signature_verifier.verify().is_ok() {
+            Ok(Self {
+                block,
+                block_root,
+                parent,
+            })
+        } else {
+            Err(BlockError::ProposalSignatureInvalid)
+        }
+    }
+}
+
+impl<T: BeaconChainTypes> IntoFullyVerifiedBlock<T> for GossipVerifiedBlock<T> {
+    /// Completes verification of the wrapped `block`.
+    fn into_fully_verified_block(
+        self,
+        chain: &BeaconChain<T>,
+    ) -> Result<FullyVerifiedBlock<T>, BlockError> {
+        let fully_verified = SignatureVerifiedBlock::from_gossip_verified_block(self, chain)?;
+        fully_verified.into_fully_verified_block(chain)
+    }
+
+    fn block(&self) -> &SignedBeaconBlock<T::EthSpec> {
+        &self.block
+    }
+}
+
+impl<T: BeaconChainTypes> SignatureVerifiedBlock<T> {
+    /// Instantiates `Self`, a wrapper that indicates that all signatures (except the deposit
+    /// signatures) are valid  (i.e., signed by the correct public keys).
+    ///
+    /// Returns an error if the block is invalid, or if the block was unable to be verified.
+    pub fn new(
+        block: SignedBeaconBlock<T::EthSpec>,
+        chain: &BeaconChain<T>,
+    ) -> Result<Self, BlockError> {
+        let mut parent = load_parent(&block.message, chain)?;
+        let block_root = get_block_root(&block);
+
+        let state = cheap_state_advance_to_obtain_committees(
+            &mut parent.beacon_state,
+            block.slot(),
+            &chain.spec,
+        )?;
+
+        let pubkey_cache = get_validator_pubkey_cache(chain)?;
+
+        let mut signature_verifier = get_signature_verifier(&state, &pubkey_cache, &chain.spec);
+
+        signature_verifier.include_all_signatures(&block, Some(block_root))?;
+
+        if signature_verifier.verify().is_ok() {
+            Ok(Self {
+                block,
+                block_root,
+                parent: Some(parent),
+            })
+        } else {
+            Err(BlockError::InvalidSignature)
+        }
+    }
+
+    /// Finishes signature verification on the provided `GossipVerifedBlock`. Does not re-verify
+    /// the proposer signature.
+    pub fn from_gossip_verified_block(
+        from: GossipVerifiedBlock<T>,
+        chain: &BeaconChain<T>,
+    ) -> Result<Self, BlockError> {
+        let mut parent = from.parent;
+        let block = from.block;
+
+        let state = cheap_state_advance_to_obtain_committees(
+            &mut parent.beacon_state,
+            block.slot(),
+            &chain.spec,
+        )?;
+
+        let pubkey_cache = get_validator_pubkey_cache(chain)?;
+
+        let mut signature_verifier = get_signature_verifier(&state, &pubkey_cache, &chain.spec);
+
+        signature_verifier.include_all_signatures_except_proposal(&block)?;
+
+        if signature_verifier.verify().is_ok() {
+            Ok(Self {
+                block,
+                block_root: from.block_root,
+                parent: Some(parent),
+            })
+        } else {
+            Err(BlockError::InvalidSignature)
+        }
+    }
+}
+
+impl<T: BeaconChainTypes> IntoFullyVerifiedBlock<T> for SignatureVerifiedBlock<T> {
+    /// Completes verification of the wrapped `block`.
+    fn into_fully_verified_block(
+        self,
+        chain: &BeaconChain<T>,
+    ) -> Result<FullyVerifiedBlock<T>, BlockError> {
+        let block = self.block;
+        let parent = self
+            .parent
+            .map(Result::Ok)
+            .unwrap_or_else(|| load_parent(&block.message, chain))?;
+
+        FullyVerifiedBlock::from_signature_verified_components(
+            block,
+            self.block_root,
+            parent,
+            chain,
+        )
+    }
+
+    fn block(&self) -> &SignedBeaconBlock<T::EthSpec> {
+        &self.block
+    }
+}
+
+impl<T: BeaconChainTypes> IntoFullyVerifiedBlock<T> for SignedBeaconBlock<T::EthSpec> {
+    /// Verifies the `SignedBeaconBlock` by first transforming it into a `SignatureVerifiedBlock`
+    /// and then using that implementation of `IntoFullyVerifiedBlock` to complete verification.
+    fn into_fully_verified_block(
+        self,
+        chain: &BeaconChain<T>,
+    ) -> Result<FullyVerifiedBlock<T>, BlockError> {
+        SignatureVerifiedBlock::new(self, chain)?.into_fully_verified_block(chain)
+    }
+
+    fn block(&self) -> &SignedBeaconBlock<T::EthSpec> {
+        &self
+    }
+}
+
+impl<T: BeaconChainTypes> FullyVerifiedBlock<T> {
+    /// Instantiates `Self`, a wrapper that indicates that the given `block` is fully valid. See
+    /// the struct-level documentation for more information.
+    ///
+    /// Note: this function does not verify block signatures, it assumes they are valid. Signature
+    /// verification must be done upstream (e.g., via a `SignatureVerifiedBlock`
+    ///
+    /// Returns an error if the block is invalid, or if the block was unable to be verified.
+    pub fn from_signature_verified_components(
+        block: SignedBeaconBlock<T::EthSpec>,
+        block_root: Hash256,
+        parent: BeaconSnapshot<T::EthSpec>,
+        chain: &BeaconChain<T>,
+    ) -> Result<Self, BlockError> {
+        // Reject any block if its parent is not known to fork choice.
+        //
+        // A block that is not in fork choice is either:
+        //
+        //  - Not yet imported: we should reject this block because we should only import a child
+        //  after its parent has been fully imported.
+        //  - Pre-finalized: if the parent block is _prior_ to finalization, we should ignore it
+        //  because it will revert finalization. Note that the finalized block is stored in fork
+        //  choice, so we will not reject any child of the finalized block (this is relevant during
+        //  genesis).
+        if !chain.fork_choice.contains_block(&block.parent_root()) {
+            return Err(BlockError::ParentUnknown(block.parent_root()));
+        }
+
+        /*
+         *  Perform cursory checks to see if the block is even worth processing.
+         */
+
+        check_block_relevancy(&block, Some(block_root), chain)?;
+
+        /*
+         * Advance the given `parent.beacon_state` to the slot of the given `block`.
+         */
+
+        let catchup_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_CATCHUP_STATE);
+
+        // Keep a batch of any states that were "skipped" (block-less) in between the parent state
+        // slot and the block slot. These will be stored in the database.
+        let mut intermediate_states = StateBatch::new();
+
+        // The block must have a higher slot than its parent.
+        if block.slot() <= parent.beacon_state.slot {
+            return Err(BlockError::BlockIsNotLaterThanParent {
+                block_slot: block.slot(),
+                state_slot: parent.beacon_state.slot,
+            });
+        }
+
+        // Transition the parent state to the block slot.
+        let mut state = parent.beacon_state;
+        let distance = block.slot().as_u64().saturating_sub(state.slot.as_u64());
+        for i in 0..distance {
+            let state_root = if i == 0 {
+                parent.beacon_block.state_root()
+            } else {
+                // This is a new state we've reached, so stage it for storage in the DB.
+                // Computing the state root here is time-equivalent to computing it during slot
+                // processing, but we get early access to it.
+                let state_root = state.update_tree_hash_cache()?;
+                intermediate_states.add_state(state_root, &state)?;
+                state_root
+            };
+
+            per_slot_processing(&mut state, Some(state_root), &chain.spec)?;
+        }
+
+        metrics::stop_timer(catchup_timer);
+
+        /*
+         * Build the committee caches on the state.
+         */
+
+        let committee_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_COMMITTEE);
+
+        state.build_committee_cache(RelativeEpoch::Previous, &chain.spec)?;
+        state.build_committee_cache(RelativeEpoch::Current, &chain.spec)?;
+
+        metrics::stop_timer(committee_timer);
+
+        /*
+         * Perform `per_block_processing` on the block and state, returning early if the block is
+         * invalid.
+         */
+
+        let core_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_CORE);
+
+        if let Err(err) = per_block_processing(
+            &mut state,
+            &block,
+            Some(block_root),
+            // Signatures were verified earlier in this function.
+            BlockSignatureStrategy::NoVerification,
+            &chain.spec,
+        ) {
+            match err {
+                // Capture `BeaconStateError` so that we can easily distinguish between a block
+                // that's invalid and one that caused an internal error.
+                BlockProcessingError::BeaconStateError(e) => return Err(e.into()),
+                other => return Err(BlockError::PerBlockProcessingError(other)),
+            }
+        };
+
+        metrics::stop_timer(core_timer);
+
+        /*
+         * Calculate the state root of the newly modified state
+         */
+
+        let state_root_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_STATE_ROOT);
+
+        let state_root = state.update_tree_hash_cache()?;
+
+        metrics::stop_timer(state_root_timer);
+
+        /*
+         * Check to ensure the state root on the block matches the one we have calculated.
+         */
+
+        if block.state_root() != state_root {
+            return Err(BlockError::StateRootMismatch {
+                block: block.state_root(),
+                local: state_root,
+            });
+        }
+
+        Ok(Self {
+            block,
+            block_root,
+            state,
+            parent_block: parent.beacon_block,
+            intermediate_states,
+        })
+    }
+}
+
+/// Returns `Ok(())` if the block is later than the finalized slot on `chain`.
+///
+/// Returns an error if the block is earlier or equal to the finalized slot, or there was an error
+/// verifying that condition.
+fn check_block_against_finalized_slot<T: BeaconChainTypes>(
+    block: &BeaconBlock<T::EthSpec>,
+    chain: &BeaconChain<T>,
+) -> Result<(), BlockError> {
+    let finalized_slot = chain
+        .head_info()?
+        .finalized_checkpoint
+        .epoch
+        .start_slot(T::EthSpec::slots_per_epoch());
+
+    if block.slot <= finalized_slot {
+        Err(BlockError::WouldRevertFinalizedSlot {
+            block_slot: block.slot,
+            finalized_slot,
+        })
+    } else {
+        Ok(())
+    }
+}
+
+/// Performs simple, cheap checks to ensure that the block is relevant to imported.
+///
+/// `Ok(block_root)` is returned if the block passes these checks and should progress with
+/// verification (viz., it is relevant).
+///
+/// Returns an error if the block fails one of these checks (viz., is not relevant) or an error is
+/// experienced whilst attempting to verify.
+pub fn check_block_relevancy<T: BeaconChainTypes>(
+    signed_block: &SignedBeaconBlock<T::EthSpec>,
+    block_root: Option<Hash256>,
+    chain: &BeaconChain<T>,
+) -> Result<Hash256, BlockError> {
+    let block = &signed_block.message;
+
+    // Do not process blocks from the future.
+    if block.slot > chain.slot()? {
+        return Err(BlockError::FutureSlot {
+            present_slot: chain.slot()?,
+            block_slot: block.slot,
+        });
+    }
+
+    // Do not re-process the genesis block.
+    if block.slot == 0 {
+        return Err(BlockError::GenesisBlock);
+    }
+
+    // This is an artificial (non-spec) restriction that provides some protection from overflow
+    // abuses.
+    if block.slot >= MAXIMUM_BLOCK_SLOT_NUMBER {
+        return Err(BlockError::BlockSlotLimitReached);
+    }
+
+    // Do not process a block from a finalized slot.
+    check_block_against_finalized_slot(block, chain)?;
+
+    let block_root = block_root.unwrap_or_else(|| get_block_root(&signed_block));
+
+    // Check if the block is already known. We know it is post-finalization, so it is
+    // sufficient to check the fork choice.
+    if chain.fork_choice.contains_block(&block_root) {
+        return Err(BlockError::BlockIsAlreadyKnown);
+    }
+
+    Ok(block_root)
+}
+
+/// Returns the canonical root of the given `block`.
+///
+/// Use this function to ensure that we report the block hashing time Prometheus metric.
+pub fn get_block_root<E: EthSpec>(block: &SignedBeaconBlock<E>) -> Hash256 {
+    let block_root_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_BLOCK_ROOT);
+
+    let block_root = block.canonical_root();
+
+    metrics::stop_timer(block_root_timer);
+
+    block_root
+}
+
+/// Load the parent snapshot (block and state) of the given `block`.
+///
+/// Returns `Err(BlockError::ParentUnknown)` if the parent is not found, or if an error occurs
+/// whilst attempting the operation.
+fn load_parent<T: BeaconChainTypes>(
+    block: &BeaconBlock<T::EthSpec>,
+    chain: &BeaconChain<T>,
+) -> Result<BeaconSnapshot<T::EthSpec>, BlockError> {
+    let db_read_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_DB_READ);
+
+    // Reject any block if its parent is not known to fork choice.
+    //
+    // A block that is not in fork choice is either:
+    //
+    //  - Not yet imported: we should reject this block because we should only import a child
+    //  after its parent has been fully imported.
+    //  - Pre-finalized: if the parent block is _prior_ to finalization, we should ignore it
+    //  because it will revert finalization. Note that the finalized block is stored in fork
+    //  choice, so we will not reject any child of the finalized block (this is relevant during
+    //  genesis).
+    if !chain.fork_choice.contains_block(&block.parent_root) {
+        return Err(BlockError::ParentUnknown(block.parent_root));
+    }
+
+    // Load the parent block and state from disk, returning early if it's not available.
+    let result = chain
+        .snapshot_cache
+        .try_write_for(BLOCK_PROCESSING_CACHE_LOCK_TIMEOUT)
+        .and_then(|mut snapshot_cache| snapshot_cache.try_remove(block.parent_root))
+        .map(|snapshot| Ok(Some(snapshot)))
+        .unwrap_or_else(|| {
+            // Load the blocks parent block from the database, returning invalid if that block is not
+            // found.
+            //
+            // We don't return a DBInconsistent error here since it's possible for a block to
+            // exist in fork choice but not in the database yet. In such a case we simply
+            // indicate that we don't yet know the parent.
+            let parent_block = if let Some(block) = chain.get_block(&block.parent_root)? {
+                block
+            } else {
+                return Ok(None);
+            };
+
+            // Load the parent blocks state from the database, returning an error if it is not found.
+            // It is an error because if we know the parent block we should also know the parent state.
+            let parent_state_root = parent_block.state_root();
+            let parent_state = chain
+                .get_state(&parent_state_root, Some(parent_block.slot()))?
+                .ok_or_else(|| {
+                    BeaconChainError::DBInconsistent(format!(
+                        "Missing state {:?}",
+                        parent_state_root
+                    ))
+                })?;
+
+            Ok(Some(BeaconSnapshot {
+                beacon_block: parent_block,
+                beacon_block_root: block.parent_root,
+                beacon_state: parent_state,
+                beacon_state_root: parent_state_root,
+            }))
+        })
+        .map_err(BlockError::BeaconChainError)?
+        .ok_or_else(|| BlockError::ParentUnknown(block.parent_root));
+
+    metrics::stop_timer(db_read_timer);
+
+    result
+}
+
+/// Performs a cheap (time-efficient) state advancement so the committees for `slot` can be
+/// obtained from `state`.
+///
+/// The state advancement is "cheap" since it does not generate state roots. As a result, the
+/// returned state might be holistically invalid but the committees will be correct (since they do
+/// not rely upon state roots).
+///
+/// If the given `state` can already serve the `slot`, the committees will be built on the `state`
+/// and `Cow::Borrowed(state)` will be returned. Otherwise, the state will be cloned, cheaply
+/// advanced and then returned as a `Cow::Owned`. The end result is that the given `state` is never
+/// mutated to be invalid (in fact, it is never changed beyond a simple committee cache build).
+fn cheap_state_advance_to_obtain_committees<'a, E: EthSpec>(
+    state: &'a mut BeaconState<E>,
+    block_slot: Slot,
+    spec: &ChainSpec,
+) -> Result<Cow<'a, BeaconState<E>>, BlockError> {
+    let block_epoch = block_slot.epoch(E::slots_per_epoch());
+
+    if state.current_epoch() == block_epoch {
+        state.build_committee_cache(RelativeEpoch::Current, spec)?;
+
+        Ok(Cow::Borrowed(state))
+    } else if state.slot > block_slot {
+        Err(BlockError::BlockIsNotLaterThanParent {
+            block_slot,
+            state_slot: state.slot,
+        })
+    } else {
+        let mut state = state.clone_with(CloneConfig::committee_caches_only());
+
+        while state.current_epoch() < block_epoch {
+            // Don't calculate state roots since they aren't required for calculating
+            // shuffling (achieved by providing Hash256::zero()).
+            per_slot_processing(&mut state, Some(Hash256::zero()), spec).map_err(|e| {
+                BlockError::BeaconChainError(BeaconChainError::SlotProcessingError(e))
+            })?;
+        }
+
+        state.build_committee_cache(RelativeEpoch::Current, spec)?;
+
+        Ok(Cow::Owned(state))
+    }
+}
+
+/// Obtains a read-locked `ValidatorPubkeyCache` from the `chain`.
+fn get_validator_pubkey_cache<T: BeaconChainTypes>(
+    chain: &BeaconChain<T>,
+) -> Result<RwLockReadGuard<ValidatorPubkeyCache>, BlockError> {
+    chain
+        .validator_pubkey_cache
+        .try_read_for(VALIDATOR_PUBKEY_CACHE_LOCK_TIMEOUT)
+        .ok_or_else(|| BeaconChainError::ValidatorPubkeyCacheLockTimeout)
+        .map_err(BlockError::BeaconChainError)
+}
+
+/// Produces an _empty_ `BlockSignatureVerifier`.
+///
+/// The signature verifier is empty because it does not yet have any of this block's signatures
+/// added to it. Use `Self::apply_to_signature_verifier` to apply the signatures.
+fn get_signature_verifier<'a, E: EthSpec>(
+    state: &'a BeaconState<E>,
+    validator_pubkey_cache: &'a ValidatorPubkeyCache,
+    spec: &'a ChainSpec,
+) -> BlockSignatureVerifier<'a, E, impl Fn(usize) -> Option<Cow<'a, G1Point>> + Clone> {
+    BlockSignatureVerifier::new(
+        state,
+        move |validator_index| {
+            // Disallow access to any validator pubkeys that are not in the current beacon
+            // state.
+            if validator_index < state.validators.len() {
+                validator_pubkey_cache
+                    .get(validator_index)
+                    .map(|pk| Cow::Borrowed(pk.as_point()))
+            } else {
+                None
+            }
+        },
+        spec,
+    )
+}

beacon_node/beacon_chain/src/block_verification/block_processing_outcome.rs

@@ -0,0 +1,105 @@
+use crate::{BeaconChainError, BlockError};
+use state_processing::BlockProcessingError;
+use types::{Hash256, Slot};
+
+/// This is a legacy object that is being kept around to reduce merge conflicts.
+///
+/// As soon as this is merged into master, it should be removed as soon as possible.
+#[derive(Debug, PartialEq)]
+pub enum BlockProcessingOutcome {
+    /// Block was valid and imported into the block graph.
+    Processed {
+        block_root: Hash256,
+    },
+    InvalidSignature,
+    /// The proposal signature in invalid.
+    ProposalSignatureInvalid,
+    /// The parent block was unknown.
+    ParentUnknown(Hash256),
+    /// The block slot is greater than the present slot.
+    FutureSlot {
+        present_slot: Slot,
+        block_slot: Slot,
+    },
+    /// The block state_root does not match the generated state.
+    StateRootMismatch {
+        block: Hash256,
+        local: Hash256,
+    },
+    /// The block was a genesis block, these blocks cannot be re-imported.
+    GenesisBlock,
+    /// The slot is finalized, no need to import.
+    WouldRevertFinalizedSlot {
+        block_slot: Slot,
+        finalized_slot: Slot,
+    },
+    /// Block is already known, no need to re-import.
+    BlockIsAlreadyKnown,
+    /// The block slot exceeds the MAXIMUM_BLOCK_SLOT_NUMBER.
+    BlockSlotLimitReached,
+    /// The provided block is from an earlier slot than its parent.
+    BlockIsNotLaterThanParent {
+        block_slot: Slot,
+        state_slot: Slot,
+    },
+    /// At least one block in the chain segement did not have it's parent root set to the root of
+    /// the prior block.
+    NonLinearParentRoots,
+    /// The slots of the blocks in the chain segment were not strictly increasing. I.e., a child
+    /// had lower slot than a parent.
+    NonLinearSlots,
+    /// The block could not be applied to the state, it is invalid.
+    PerBlockProcessingError(BlockProcessingError),
+}
+
+impl BlockProcessingOutcome {
+    pub fn shim(
+        result: Result<Hash256, BlockError>,
+    ) -> Result<BlockProcessingOutcome, BeaconChainError> {
+        match result {
+            Ok(block_root) => Ok(BlockProcessingOutcome::Processed { block_root }),
+            Err(BlockError::ParentUnknown(root)) => Ok(BlockProcessingOutcome::ParentUnknown(root)),
+            Err(BlockError::FutureSlot {
+                present_slot,
+                block_slot,
+            }) => Ok(BlockProcessingOutcome::FutureSlot {
+                present_slot,
+                block_slot,
+            }),
+            Err(BlockError::StateRootMismatch { block, local }) => {
+                Ok(BlockProcessingOutcome::StateRootMismatch { block, local })
+            }
+            Err(BlockError::GenesisBlock) => Ok(BlockProcessingOutcome::GenesisBlock),
+            Err(BlockError::WouldRevertFinalizedSlot {
+                block_slot,
+                finalized_slot,
+            }) => Ok(BlockProcessingOutcome::WouldRevertFinalizedSlot {
+                block_slot,
+                finalized_slot,
+            }),
+            Err(BlockError::BlockIsAlreadyKnown) => Ok(BlockProcessingOutcome::BlockIsAlreadyKnown),
+            Err(BlockError::BlockSlotLimitReached) => {
+                Ok(BlockProcessingOutcome::BlockSlotLimitReached)
+            }
+            Err(BlockError::ProposalSignatureInvalid) => {
+                Ok(BlockProcessingOutcome::ProposalSignatureInvalid)
+            }
+            Err(BlockError::InvalidSignature) => Ok(BlockProcessingOutcome::InvalidSignature),
+            Err(BlockError::BlockIsNotLaterThanParent {
+                block_slot,
+                state_slot,
+            }) => Ok(BlockProcessingOutcome::BlockIsNotLaterThanParent {
+                block_slot,
+                state_slot,
+            }),
+            Err(BlockError::NonLinearParentRoots) => {
+                Ok(BlockProcessingOutcome::NonLinearParentRoots)
+            }
+            Err(BlockError::NonLinearSlots) => Ok(BlockProcessingOutcome::NonLinearSlots),
+            Err(BlockError::PerBlockProcessingError(e)) => {
+                Ok(BlockProcessingOutcome::PerBlockProcessingError(e))
+            }
+            Err(BlockError::BeaconChainError(e)) => Err(e),
+        }
+    }
+}

beacon_node/beacon_chain/src/builder.rs

@@ -7,10 +7,11 @@ use crate::fork_choice::SszForkChoice;
 use crate::head_tracker::HeadTracker;
 use crate::persisted_beacon_chain::PersistedBeaconChain;
 use crate::shuffling_cache::ShufflingCache;
+use crate::snapshot_cache::{SnapshotCache, DEFAULT_SNAPSHOT_CACHE_SIZE};
 use crate::timeout_rw_lock::TimeoutRwLock;
 use crate::validator_pubkey_cache::ValidatorPubkeyCache;
 use crate::{
-    BeaconChain, BeaconChainTypes, CheckPoint, Eth1Chain, Eth1ChainBackend, EventHandler,
+    BeaconChain, BeaconChainTypes, BeaconSnapshot, Eth1Chain, Eth1ChainBackend, EventHandler,
     ForkChoice,
 };
 use eth1::Config as Eth1Config;
@@ -71,10 +72,10 @@ where
 pub struct BeaconChainBuilder<T: BeaconChainTypes> {
     store: Option<Arc<T::Store>>,
     store_migrator: Option<T::StoreMigrator>,
-    canonical_head: Option<CheckPoint<T::EthSpec>>,
+    canonical_head: Option<BeaconSnapshot<T::EthSpec>>,
     /// The finalized checkpoint to anchor the chain. May be genesis or a higher
     /// checkpoint.
-    pub finalized_checkpoint: Option<CheckPoint<T::EthSpec>>,
+    pub finalized_snapshot: Option<BeaconSnapshot<T::EthSpec>>,
     genesis_block_root: Option<Hash256>,
     op_pool: Option<OperationPool<T::EthSpec>>,
     fork_choice: Option<ForkChoice<T>>,
@@ -110,7 +111,7 @@ where
             store: None,
             store_migrator: None,
             canonical_head: None,
-            finalized_checkpoint: None,
+            finalized_snapshot: None,
             genesis_block_root: None,
             op_pool: None,
             fork_choice: None,
@@ -247,14 +248,14 @@ where
             .map_err(|e| format!("DB error when reading finalized state: {:?}", e))?
             .ok_or_else(|| "Finalized state not found in store".to_string())?;
 
-        self.finalized_checkpoint = Some(CheckPoint {
+        self.finalized_snapshot = Some(BeaconSnapshot {
             beacon_block_root: finalized_block_root,
             beacon_block: finalized_block,
             beacon_state_root: finalized_state_root,
             beacon_state: finalized_state,
         });
 
-        self.canonical_head = Some(CheckPoint {
+        self.canonical_head = Some(BeaconSnapshot {
             beacon_block_root: head_block_root,
             beacon_block: head_block,
             beacon_state_root: head_state_root,
@@ -291,7 +292,7 @@ where
         self.genesis_block_root = Some(beacon_block_root);
 
         store
-            .put_state(&beacon_state_root, beacon_state.clone())
+            .put_state(&beacon_state_root, &beacon_state)
             .map_err(|e| format!("Failed to store genesis state: {:?}", e))?;
         store
             .put(&beacon_block_root, &beacon_block)
@@ -305,7 +306,7 @@ where
             )
         })?;
 
-        self.finalized_checkpoint = Some(CheckPoint {
+        self.finalized_snapshot = Some(BeaconSnapshot {
             beacon_block_root,
             beacon_block,
             beacon_state_root,
@@ -367,7 +368,7 @@ where
         let mut canonical_head = if let Some(head) = self.canonical_head {
             head
         } else {
-            self.finalized_checkpoint
+            self.finalized_snapshot
                 .ok_or_else(|| "Cannot build without a state".to_string())?
         };
 
@@ -407,7 +408,7 @@ where
                 .op_pool
                 .ok_or_else(|| "Cannot build without op pool".to_string())?,
             eth1_chain: self.eth1_chain,
-            canonical_head: TimeoutRwLock::new(canonical_head),
+            canonical_head: TimeoutRwLock::new(canonical_head.clone()),
             genesis_block_root: self
                 .genesis_block_root
                 .ok_or_else(|| "Cannot build without a genesis block root".to_string())?,
@@ -418,6 +419,10 @@ where
                 .event_handler
                 .ok_or_else(|| "Cannot build without an event handler".to_string())?,
             head_tracker: self.head_tracker.unwrap_or_default(),
+            snapshot_cache: TimeoutRwLock::new(SnapshotCache::new(
+                DEFAULT_SNAPSHOT_CACHE_SIZE,
+                canonical_head,
+            )),
             shuffling_cache: TimeoutRwLock::new(ShufflingCache::new()),
             validator_pubkey_cache: TimeoutRwLock::new(validator_pubkey_cache),
             log: log.clone(),
@@ -469,30 +474,30 @@ where
             ForkChoice::from_ssz_container(persisted)
                 .map_err(|e| format!("Unable to read persisted fork choice from disk: {:?}", e))?
         } else {
-            let finalized_checkpoint = &self
-                .finalized_checkpoint
+            let finalized_snapshot = &self
+                .finalized_snapshot
                 .as_ref()
-                .ok_or_else(|| "fork_choice_backend requires a finalized_checkpoint")?;
+                .ok_or_else(|| "fork_choice_backend requires a finalized_snapshot")?;
             let genesis_block_root = self
                 .genesis_block_root
                 .ok_or_else(|| "fork_choice_backend requires a genesis_block_root")?;
 
             let backend = ProtoArrayForkChoice::new(
-                finalized_checkpoint.beacon_block.message.slot,
-                finalized_checkpoint.beacon_block.message.state_root,
+                finalized_snapshot.beacon_block.message.slot,
+                finalized_snapshot.beacon_block.message.state_root,
                 // Note: here we set the `justified_epoch` to be the same as the epoch of the
                 // finalized checkpoint. Whilst this finalized checkpoint may actually point to
                 // a _later_ justified checkpoint, that checkpoint won't yet exist in the fork
                 // choice.
-                finalized_checkpoint.beacon_state.current_epoch(),
-                finalized_checkpoint.beacon_state.current_epoch(),
-                finalized_checkpoint.beacon_block_root,
+                finalized_snapshot.beacon_state.current_epoch(),
+                finalized_snapshot.beacon_state.current_epoch(),
+                finalized_snapshot.beacon_block_root,
             )?;
 
             ForkChoice::new(
                 backend,
                 genesis_block_root,
-                &finalized_checkpoint.beacon_state,
+                &finalized_snapshot.beacon_state,
             )
         };
 
@@ -563,7 +568,7 @@ where
     /// Requires the state to be initialized.
     pub fn testing_slot_clock(self, slot_duration: Duration) -> Result<Self, String> {
         let genesis_time = self
-            .finalized_checkpoint
+            .finalized_snapshot
             .as_ref()
             .ok_or_else(|| "testing_slot_clock requires an initialized state")?
             .beacon_state
@@ -642,7 +647,7 @@ mod test {
 
     #[test]
     fn recent_genesis() {
-        let validator_count = 8;
+        let validator_count = 1;
         let genesis_time = 13_371_337;
 
         let log = get_logger();

beacon_node/beacon_chain/src/errors.rs

@@ -3,9 +3,11 @@ use crate::fork_choice::Error as ForkChoiceError;
 use operation_pool::OpPoolError;
 use ssz::DecodeError;
 use ssz_types::Error as SszTypesError;
-use state_processing::per_block_processing::errors::AttestationValidationError;
-use state_processing::BlockProcessingError;
-use state_processing::SlotProcessingError;
+use state_processing::{
+    block_signature_verifier::Error as BlockSignatureVerifierError,
+    per_block_processing::errors::AttestationValidationError,
+    signature_sets::Error as SignatureSetError, BlockProcessingError, SlotProcessingError,
+};
 use std::time::Duration;
 use types::*;
 
@@ -57,13 +59,18 @@ pub enum BeaconChainError {
     IncorrectStateForAttestation(RelativeEpochError),
     InvalidValidatorPubkeyBytes(DecodeError),
     ValidatorPubkeyCacheIncomplete(usize),
-    SignatureSetError(state_processing::signature_sets::Error),
+    SignatureSetError(SignatureSetError),
+    BlockSignatureVerifierError(state_processing::block_signature_verifier::Error),
+    DuplicateValidatorPublicKey,
     ValidatorPubkeyCacheFileError(String),
+    OpPoolError(OpPoolError),
 }
 
 easy_from_to!(SlotProcessingError, BeaconChainError);
 easy_from_to!(AttestationValidationError, BeaconChainError);
 easy_from_to!(SszTypesError, BeaconChainError);
+easy_from_to!(OpPoolError, BeaconChainError);
+easy_from_to!(BlockSignatureVerifierError, BeaconChainError);
 
 #[derive(Debug, PartialEq)]
 pub enum BlockProductionError {
@@ -84,3 +91,27 @@ easy_from_to!(BlockProcessingError, BlockProductionError);
 easy_from_to!(BeaconStateError, BlockProductionError);
 easy_from_to!(SlotProcessingError, BlockProductionError);
 easy_from_to!(Eth1ChainError, BlockProductionError);
+
+/// A reason for not propagating an attestation (single or aggregate).
+#[derive(Debug, PartialEq)]
+pub enum AttestationDropReason {
+    SlotClockError,
+    TooNew { attestation_slot: Slot, now: Slot },
+    TooOld { attestation_slot: Slot, now: Slot },
+    NoValidationState(BeaconChainError),
+    BlockUnknown(Hash256),
+    BadIndexedAttestation(AttestationValidationError),
+    AggregatorNotInAttestingIndices,
+    AggregatorNotSelected,
+    AggregatorSignatureInvalid,
+    SignatureInvalid,
+}
+
+/// A reason for not propagating a block.
+#[derive(Debug, PartialEq)]
+pub enum BlockDropReason {
+    SlotClockError,
+    TooNew { block_slot: Slot, now: Slot },
+    // FIXME(sproul): add detail here
+    ValidationFailure,
+}

beacon_node/beacon_chain/src/eth1_chain.rs

@@ -1,7 +1,6 @@
 use crate::metrics;
 use eth1::{Config as Eth1Config, Eth1Block, Service as HttpService};
 use eth2_hashing::hash;
-use exit_future::Exit;
 use futures::Future;
 use slog::{debug, error, trace, Logger};
 use ssz::{Decode, Encode};
@@ -279,7 +278,10 @@ impl<T: EthSpec, S: Store<T>> CachingEth1Backend<T, S> {
     }
 
     /// Starts the routine which connects to the external eth1 node and updates the caches.
-    pub fn start(&self, exit: Exit) -> impl Future<Item = (), Error = ()> {
+    pub fn start(
+        &self,
+        exit: tokio::sync::oneshot::Receiver<()>,
+    ) -> impl Future<Item = (), Error = ()> {
         self.core.auto_update(exit)
     }
 

beacon_node/beacon_chain/src/fork_choice/checkpoint_manager.rs

@@ -306,10 +306,7 @@ impl CheckpointManager {
                 .ok_or_else(|| Error::UnknownJustifiedBlock(block_root))?;
 
             let state = chain
-                .get_state_caching_only_with_committee_caches(
-                    &block.state_root(),
-                    Some(block.slot()),
-                )?
+                .get_state(&block.state_root(), Some(block.slot()))?
                 .ok_or_else(|| Error::UnknownJustifiedState(block.state_root()))?;
 
             Ok(get_effective_balances(&state))

beacon_node/beacon_chain/src/lib.rs

@@ -3,8 +3,9 @@
 extern crate lazy_static;
 
 mod beacon_chain;
+mod beacon_snapshot;
+mod block_verification;
 pub mod builder;
-mod checkpoint;
 mod errors;
 pub mod eth1_chain;
 pub mod events;
@@ -13,16 +14,17 @@ mod head_tracker;
 mod metrics;
 mod persisted_beacon_chain;
 mod shuffling_cache;
+mod snapshot_cache;
 pub mod test_utils;
 mod timeout_rw_lock;
 mod validator_pubkey_cache;
 
 pub use self::beacon_chain::{
-    AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, BlockProcessingOutcome,
-    StateSkipConfig,
+    AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, StateSkipConfig,
 };
-pub use self::checkpoint::CheckPoint;
+pub use self::beacon_snapshot::BeaconSnapshot;
 pub use self::errors::{BeaconChainError, BlockProductionError};
+pub use block_verification::{BlockError, BlockProcessingOutcome};
 pub use eth1_chain::{Eth1Chain, Eth1ChainBackend};
 pub use events::EventHandler;
 pub use fork_choice::ForkChoice;

beacon_node/beacon_chain/src/metrics.rs

@@ -32,6 +32,10 @@ lazy_static! {
         "beacon_block_processing_committee_building_seconds",
         "Time spent building/obtaining committees for block processing."
     );
+    pub static ref BLOCK_PROCESSING_SIGNATURE: Result<Histogram> = try_create_histogram(
+        "beacon_block_processing_signature_seconds",
+        "Time spent doing signature verification for a block."
+    );
     pub static ref BLOCK_PROCESSING_CORE: Result<Histogram> = try_create_histogram(
         "beacon_block_processing_core_seconds",
         "Time spent doing the core per_block_processing state processing."

beacon_node/beacon_chain/src/snapshot_cache.rs

@@ -0,0 +1,217 @@
+use crate::BeaconSnapshot;
+use std::cmp;
+use types::{Epoch, EthSpec, Hash256};
+
+/// The default size of the cache.
+pub const DEFAULT_SNAPSHOT_CACHE_SIZE: usize = 4;
+
+/// Provides a cache of `BeaconSnapshot` that is intended primarily for block processing.
+///
+/// ## Cache Queuing
+///
+/// The cache has a non-standard queue mechanism (specifically, it is not LRU).
+///
+/// The cache has a max number of elements (`max_len`). Until `max_len` is achieved, all snapshots
+/// are simply added to the queue. Once `max_len` is achieved, adding a new snapshot will cause an
+/// existing snapshot to be ejected. The ejected snapshot will:
+///
+/// - Never be the `head_block_root`.
+/// - Be the snapshot with the lowest `state.slot` (ties broken arbitrarily).
+pub struct SnapshotCache<T: EthSpec> {
+    max_len: usize,
+    head_block_root: Hash256,
+    snapshots: Vec<BeaconSnapshot<T>>,
+}
+
+impl<T: EthSpec> SnapshotCache<T> {
+    /// Instantiate a new cache which contains the `head` snapshot.
+    ///
+    /// Setting `max_len = 0` is equivalent to setting `max_len = 1`.
+    pub fn new(max_len: usize, head: BeaconSnapshot<T>) -> Self {
+        Self {
+            max_len: cmp::max(max_len, 1),
+            head_block_root: head.beacon_block_root,
+            snapshots: vec![head],
+        }
+    }
+
+    /// Insert a snapshot, potentially removing an existing snapshot if `self` is at capacity (see
+    /// struct-level documentation for more info).
+    pub fn insert(&mut self, snapshot: BeaconSnapshot<T>) {
+        if self.snapshots.len() < self.max_len {
+            self.snapshots.push(snapshot);
+        } else {
+            let insert_at = self
+                .snapshots
+                .iter()
+                .enumerate()
+                .filter_map(|(i, snapshot)| {
+                    if snapshot.beacon_block_root != self.head_block_root {
+                        Some((i, snapshot.beacon_state.slot))
+                    } else {
+                        None
+                    }
+                })
+                .min_by_key(|(_i, slot)| *slot)
+                .map(|(i, _slot)| i);
+
+            if let Some(i) = insert_at {
+                self.snapshots[i] = snapshot;
+            }
+        }
+    }
+
+    /// If there is a snapshot with `block_root`, remove and return it.
+    pub fn try_remove(&mut self, block_root: Hash256) -> Option<BeaconSnapshot<T>> {
+        self.snapshots
+            .iter()
+            .position(|snapshot| snapshot.beacon_block_root == block_root)
+            .map(|i| self.snapshots.remove(i))
+    }
+
+    /// If there is a snapshot with `block_root`, clone it (with only the committee caches) and
+    /// return the clone.
+    pub fn get_cloned(&self, block_root: Hash256) -> Option<BeaconSnapshot<T>> {
+        self.snapshots
+            .iter()
+            .find(|snapshot| snapshot.beacon_block_root == block_root)
+            .map(|snapshot| snapshot.clone_with_only_committee_caches())
+    }
+
+    /// Removes all snapshots from the queue that are less than or equal to the finalized epoch.
+    pub fn prune(&mut self, finalized_epoch: Epoch) {
+        self.snapshots.retain(|snapshot| {
+            snapshot.beacon_state.slot > finalized_epoch.start_slot(T::slots_per_epoch())
+        })
+    }
+
+    /// Inform the cache that the head of the beacon chain has changed.
+    ///
+    /// The snapshot that matches this `head_block_root` will never be ejected from the cache
+    /// during `Self::insert`.
+    pub fn update_head(&mut self, head_block_root: Hash256) {
+        self.head_block_root = head_block_root
+    }
+}
+
+#[cfg(test)]
+mod test {
+    use super::*;
+    use types::{
+        test_utils::{generate_deterministic_keypair, TestingBeaconStateBuilder},
+        BeaconBlock, Epoch, MainnetEthSpec, Signature, SignedBeaconBlock, Slot,
+    };
+
+    const CACHE_SIZE: usize = 4;
+
+    fn get_snapshot(i: u64) -> BeaconSnapshot<MainnetEthSpec> {
+        let spec = MainnetEthSpec::default_spec();
+
+        let state_builder = TestingBeaconStateBuilder::from_deterministic_keypairs(1, &spec);
+        let (beacon_state, _keypairs) = state_builder.build();
+
+        BeaconSnapshot {
+            beacon_state,
+            beacon_state_root: Hash256::from_low_u64_be(i),
+            beacon_block: SignedBeaconBlock {
+                message: BeaconBlock::empty(&spec),
+                signature: Signature::new(&[42], &generate_deterministic_keypair(0).sk),
+            },
+            beacon_block_root: Hash256::from_low_u64_be(i),
+        }
+    }
+
+    #[test]
+    fn insert_get_prune_update() {
+        let mut cache = SnapshotCache::new(CACHE_SIZE, get_snapshot(0));
+
+        // Insert a bunch of entries in the cache. It should look like this:
+        //
+        // Index    Root
+        // 0        0     <--head
+        // 1        1
+        // 2        2
+        // 3        3
+        for i in 1..CACHE_SIZE as u64 {
+            let mut snapshot = get_snapshot(i);
+
+            // Each snapshot should be one slot into an epoch, with each snapshot one epoch apart.
+            snapshot.beacon_state.slot = Slot::from(i * MainnetEthSpec::slots_per_epoch() + 1);
+
+            cache.insert(snapshot);
+
+            assert_eq!(
+                cache.snapshots.len(),
+                i as usize + 1,
+                "cache length should be as expected"
+            );
+            assert_eq!(cache.head_block_root, Hash256::from_low_u64_be(0));
+        }
+
+        // Insert a new value in the cache. Afterwards it should look like:
+        //
+        // Index    Root
+        // 0        0     <--head
+        // 1        42
+        // 2        2
+        // 3        3
+        assert_eq!(cache.snapshots.len(), CACHE_SIZE);
+        cache.insert(get_snapshot(42));
+        assert_eq!(cache.snapshots.len(), CACHE_SIZE);
+
+        assert!(
+            cache.try_remove(Hash256::from_low_u64_be(1)).is_none(),
+            "the snapshot with the lowest slot should have been removed during the insert function"
+        );
+        assert!(cache.get_cloned(Hash256::from_low_u64_be(1)).is_none());
+
+        assert!(
+            cache
+                .get_cloned(Hash256::from_low_u64_be(0))
+                .expect("the head should still be in the cache")
+                .beacon_block_root
+                == Hash256::from_low_u64_be(0),
+            "get_cloned should get the correct snapshot"
+        );
+        assert!(
+            cache
+                .try_remove(Hash256::from_low_u64_be(0))
+                .expect("the head should still be in the cache")
+                .beacon_block_root
+                == Hash256::from_low_u64_be(0),
+            "try_remove should get the correct snapshot"
+        );
+
+        assert_eq!(
+            cache.snapshots.len(),
+            CACHE_SIZE - 1,
+            "try_remove should shorten the cache"
+        );
+
+        // Prune the cache. Afterwards it should look like:
+        //
+        // Index    Root
+        // 0        2
+        // 1        3
+        cache.prune(Epoch::new(2));
+
+        assert_eq!(cache.snapshots.len(), 2);
+
+        cache.update_head(Hash256::from_low_u64_be(2));
+
+        // Over-fill the cache so it needs to eject some old values on insert.
+        for i in 0..CACHE_SIZE as u64 {
+            cache.insert(get_snapshot(u64::max_value() - i));
+        }
+
+        // Ensure that the new head value was not removed from the cache.
+        assert!(
+            cache
+                .try_remove(Hash256::from_low_u64_be(2))
+                .expect("the new head should still be in the cache")
+                .beacon_block_root
+                == Hash256::from_low_u64_be(2),
+            "try_remove should get the correct snapshot"
+        );
+    }
+}

beacon_node/beacon_chain/src/test_utils.rs

@@ -6,8 +6,7 @@ use crate::{
     builder::{BeaconChainBuilder, Witness},
     eth1_chain::CachingEth1Backend,
     events::NullEventHandler,
-    AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, BlockProcessingOutcome,
-    StateSkipConfig,
+    AttestationProcessingOutcome, BeaconChain, BeaconChainTypes, StateSkipConfig,
 };
 use genesis::interop_genesis_state;
 use rayon::prelude::*;
@@ -256,20 +255,15 @@ where
 
             let (block, new_state) = self.build_block(state.clone(), slot, block_strategy);
 
-            let outcome = self
+            let block_root = self
                 .chain
                 .process_block(block)
                 .expect("should not error during block processing");
 
             self.chain.fork_choice().expect("should find head");
 
-            if let BlockProcessingOutcome::Processed { block_root } = outcome {
-                head_block_root = Some(block_root);
-
-                self.add_free_attestations(&attestation_strategy, &new_state, block_root, slot);
-            } else {
-                panic!("block should be successfully processed: {:?}", outcome);
-            }
+            head_block_root = Some(block_root);
+            self.add_free_attestations(&attestation_strategy, &new_state, block_root, slot);
 
             state = new_state;
             slot += 1;
@@ -348,7 +342,7 @@ where
         .for_each(|attestation| {
             match self
                 .chain
-                .process_attestation(attestation)
+                .process_attestation(attestation, Some(false))
                 .expect("should not error during attestation processing")
             {
                 AttestationProcessingOutcome::Processed => (),

beacon_node/beacon_chain/src/validator_pubkey_cache.rs

@@ -1,10 +1,11 @@
 use crate::errors::BeaconChainError;
 use ssz::{Decode, DecodeError, Encode};
+use std::collections::HashMap;
 use std::convert::TryInto;
 use std::fs::{File, OpenOptions};
 use std::io::{self, Read, Write};
 use std::path::Path;
-use types::{BeaconState, EthSpec, PublicKey, PublicKeyBytes};
+use types::{BeaconState, EthSpec, PublicKey, PublicKeyBytes, Validator};
 
 /// Provides a mapping of `validator_index -> validator_publickey`.
 ///
@@ -19,6 +20,7 @@ use types::{BeaconState, EthSpec, PublicKey, PublicKeyBytes};
 /// copy of itself. This allows it to be restored between process invocations.
 pub struct ValidatorPubkeyCache {
     pubkeys: Vec<PublicKey>,
+    indices: HashMap<PublicKeyBytes, usize>,
     persitence_file: ValidatorPubkeyCacheFile,
 }
 
@@ -47,6 +49,7 @@ impl ValidatorPubkeyCache {
         let mut cache = Self {
             persitence_file: ValidatorPubkeyCacheFile::create(persistence_path)?,
             pubkeys: vec![],
+            indices: HashMap::new(),
         };
 
         cache.import_new_pubkeys(state)?;
@@ -61,38 +64,57 @@ impl ValidatorPubkeyCache {
         &mut self,
         state: &BeaconState<T>,
     ) -> Result<(), BeaconChainError> {
-        state
-            .validators
-            .iter()
-            .skip(self.pubkeys.len())
-            .try_for_each(|v| {
-                let i = self.pubkeys.len();
+        if state.validators.len() > self.pubkeys.len() {
+            self.import(&state.validators[self.pubkeys.len()..])
+        } else {
+            Ok(())
+        }
+    }
 
-                // The item is written to disk (the persistence file) _before_ it is written into
-                // the local struct.
-                //
-                // This means that a pubkey cache read from disk will always be equivalent to or
-                // _later than_ the cache that was running in the previous instance of Lighthouse.
-                //
-                // The motivation behind this ordering is that we do not want to have states that
-                // reference a pubkey that is not in our cache. However, it's fine to have pubkeys
-                // that are never referenced in a state.
-                self.persitence_file.append(i, &v.pubkey)?;
+    /// Adds zero or more validators to `self`.
+    fn import(&mut self, validators: &[Validator]) -> Result<(), BeaconChainError> {
+        self.pubkeys.reserve(validators.len());
+        self.indices.reserve(validators.len());
 
-                self.pubkeys.push(
-                    (&v.pubkey)
-                        .try_into()
-                        .map_err(BeaconChainError::InvalidValidatorPubkeyBytes)?,
-                );
+        for v in validators.iter() {
+            let i = self.pubkeys.len();
 
-                Ok(())
-            })
+            if self.indices.contains_key(&v.pubkey) {
+                return Err(BeaconChainError::DuplicateValidatorPublicKey);
+            }
+
+            // The item is written to disk (the persistence file) _before_ it is written into
+            // the local struct.
+            //
+            // This means that a pubkey cache read from disk will always be equivalent to or
+            // _later than_ the cache that was running in the previous instance of Lighthouse.
+            //
+            // The motivation behind this ordering is that we do not want to have states that
+            // reference a pubkey that is not in our cache. However, it's fine to have pubkeys
+            // that are never referenced in a state.
+            self.persitence_file.append(i, &v.pubkey)?;
+
+            self.pubkeys.push(
+                (&v.pubkey)
+                    .try_into()
+                    .map_err(BeaconChainError::InvalidValidatorPubkeyBytes)?,
+            );
+
+            self.indices.insert(v.pubkey.clone(), i);
+        }
+
+        Ok(())
     }
 
     /// Get the public key for a validator with index `i`.
     pub fn get(&self, i: usize) -> Option<&PublicKey> {
         self.pubkeys.get(i)
     }
+
+    /// Get the index of a validator with `pubkey`.
+    pub fn get_index(&self, pubkey: &PublicKeyBytes) -> Option<usize> {
+        self.indices.get(pubkey).copied()
+    }
 }
 
 /// Allows for maintaining an on-disk copy of the `ValidatorPubkeyCache`. The file is raw SSZ bytes
@@ -168,12 +190,14 @@ impl ValidatorPubkeyCacheFile {
 
         let mut last = None;
         let mut pubkeys = Vec::with_capacity(list.len());
+        let mut indices = HashMap::new();
 
         for (index, pubkey) in list {
             let expected = last.map(|n| n + 1);
             if expected.map_or(true, |expected| index == expected) {
                 last = Some(index);
                 pubkeys.push((&pubkey).try_into().map_err(Error::SszError)?);
+                indices.insert(pubkey, index);
             } else {
                 return Err(Error::InconsistentIndex {
                     expected,
@@ -184,6 +208,7 @@ impl ValidatorPubkeyCacheFile {
 
         Ok(ValidatorPubkeyCache {
             pubkeys,
+            indices,
             persitence_file: self,
         })
     }
@@ -221,6 +246,16 @@ mod test {
             if i < validator_count {
                 let pubkey = cache.get(i).expect("pubkey should be present");
                 assert_eq!(pubkey, &keypairs[i].pk, "pubkey should match cache");
+
+                let pubkey_bytes: PublicKeyBytes = pubkey.clone().into();
+
+                assert_eq!(
+                    i,
+                    cache
+                        .get_index(&pubkey_bytes)
+                        .expect("should resolve index"),
+                    "index should match cache"
+                );
             } else {
                 assert_eq!(
                     cache.get(i),