Initial work towards v0.2.0 (#924)

* Remove ping protocol * Initial renaming of network services * Correct rebasing relative to latest master * Start updating types * Adds HashMapDelay struct to utils * Initial network restructure * Network restructure. Adds new types for v0.2.0 * Removes build artefacts * Shift validation to beacon chain * Temporarily remove gossip validation This is to be updated to match current optimisation efforts. * Adds AggregateAndProof * Begin rebuilding pubsub encoding/decoding * Signature hacking * Shift gossipsup decoding into eth2_libp2p * Existing EF tests passing with fake_crypto * Shifts block encoding/decoding into RPC * Delete outdated API spec * All release tests passing bar genesis state parsing * Update and test YamlConfig * Update to spec v0.10 compatible BLS * Updates to BLS EF tests * Add EF test for AggregateVerify And delete unused hash2curve tests for uncompressed points * Update EF tests to v0.10.1 * Use optional block root correctly in block proc * Use genesis fork in deposit domain. All tests pass * Fast aggregate verify test * Update REST API docs * Fix unused import * Bump spec tags to v0.10.1 * Add `seconds_per_eth1_block` to chainspec * Update to timestamp based eth1 voting scheme * Return None from `get_votes_to_consider` if block cache is empty * Handle overflows in `is_candidate_block` * Revert to failing tests * Fix eth1 data sets test * Choose default vote according to spec * Fix collect_valid_votes tests * Fix `get_votes_to_consider` to choose all eligible blocks * Uncomment winning_vote tests * Add comments; remove unused code * Reduce seconds_per_eth1_block for simulation * Addressed review comments * Add test for default vote case * Fix logs * Remove unused functions * Meter default eth1 votes * Fix comments * Progress on attestation service * Address review comments; remove unused dependency * Initial work on removing libp2p lock * Add LRU caches to store (rollup) * Update attestation validation for DB changes (WIP) * Initial version of should_forward_block * Scaffold * Progress on attestation validation Also, consolidate prod+testing slot clocks so that they share much of the same implementation and can both handle sub-slot time changes. * Removes lock from libp2p service * Completed network lock removal * Finish(?) attestation processing * Correct network termination future * Add slot check to block check * Correct fmt issues * Remove Drop implementation for network service * Add first attempt at attestation proc. re-write * Add version 2 of attestation processing * Minor fixes * Add validator pubkey cache * Make get_indexed_attestation take a committee * Link signature processing into new attn verification * First working version * Ensure pubkey cache is updated * Add more metrics, slight optimizations * Clone committee cache during attestation processing * Update shuffling cache during block processing * Remove old commented-out code * Fix shuffling cache insert bug * Used indexed attestation in fork choice * Restructure attn processing, add metrics * Add more detailed metrics * Tidy, fix failing tests * Fix failing tests, tidy * Address reviewers suggestions * Disable/delete two outdated tests * Modification of validator for subscriptions * Add slot signing to validator client * Further progress on validation subscription * Adds necessary validator subscription functionality * Add new Pubkeys struct to signature_sets * Refactor with functional approach * Update beacon chain * Clean up validator <-> beacon node http types * Add aggregator status to ValidatorDuty * Impl Clone for manual slot clock * Fix minor errors * Further progress validator client subscription * Initial subscription and aggregation handling * Remove decompressed member from pubkey bytes * Progress to modifying val client for attestation aggregation * First draft of validator client upgrade for aggregate attestations * Add hashmap for indices lookup * Add state cache, remove store cache * Only build the head committee cache * Removes lock on a network channel * Partially implement beacon node subscription http api * Correct compilation issues * Change `get_attesting_indices` to use Vec * Fix failing test * Partial implementation of timer * Adds timer, removes exit_future, http api to op pool * Partial multiple aggregate attestation handling * Permits bulk messages accross gossipsub network channel * Correct compile issues * Improve gosispsub messaging and correct rest api helpers * Added global gossipsub subscriptions * Update validator subscriptions data structs * Tidy * Re-structure validator subscriptions * Initial handling of subscriptions * Re-structure network service * Add pubkey cache persistence file * Add more comments * Integrate persistence file into builder * Add pubkey cache tests * Add HashSetDelay and introduce into attestation service * Handles validator subscriptions * Add data_dir to beacon chain builder * Remove Option in pubkey cache persistence file * Ensure consistency between datadir/data_dir * Fix failing network test * Peer subnet discovery gets queued for future subscriptions * Reorganise attestation service functions * Initial wiring of attestation service * First draft of attestation service timing logic * Correct minor typos * Tidy * Fix todos * Improve tests * Add PeerInfo to connected peers mapping * Fix compile error * Fix compile error from merge * Split up block processing metrics * Tidy * Refactor get_pubkey_from_state * Remove commented-out code * Rename state_cache -> checkpoint_cache * Rename Checkpoint -> Snapshot * Tidy, add comments * Tidy up find_head function * Change some checkpoint -> snapshot * Add tests * Expose max_len * Remove dead code * Tidy * Fix bug * Add sync-speed metric * Add first attempt at VerifiableBlock * Start integrating into beacon chain * Integrate VerifiableBlock * Rename VerifableBlock -> PartialBlockVerification * Add start of typed methods * Add progress * Add further progress * Rename structs * Add full block verification to block_processing.rs * Further beacon chain integration * Update checks for gossip * Add todo * Start adding segement verification * Add passing chain segement test * Initial integration with batch sync * Minor changes * Tidy, add more error checking * Start adding chain_segment tests * Finish invalid signature tests * Include single and gossip verified blocks in tests * Add gossip verification tests * Start adding docs * Finish adding comments to block_processing.rs * Rename block_processing.rs -> block_verification * Start removing old block processing code * Fixes beacon_chain compilation * Fix project-wide compile errors * Remove old code * Correct code to pass all tests * Fix bug with beacon proposer index * Fix shim for BlockProcessingError * Only process one epoch at a time * Fix loop in chain segment processing * Correct tests from master merge * Add caching for state.eth1_data_votes * Add BeaconChain::validator_pubkey * Revert "Add caching for state.eth1_data_votes" This reverts commit cd73dcd643. Co-authored-by: Grant Wuerker <gwuerker@gmail.com> Co-authored-by: Michael Sproul <michael@sigmaprime.io> Co-authored-by: Michael Sproul <micsproul@gmail.com> Co-authored-by: pawan <pawandhananjay@gmail.com> Co-authored-by: Paul Hauner <paul@paulhauner.com>
2026-04-18 21:38:31 +00:00 · 2020-03-17 17:24:44 +11:00
parent c198bddf9e
commit 95c8e476bc
161 changed files with 9771 additions and 5266 deletions
--- a/eth2/operation_pool/Cargo.toml
+++ b/eth2/operation_pool/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "operation_pool"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Michael Sproul <michael@sigmaprime.io>"]
 edition = "2018"

--- a/eth2/operation_pool/src/lib.rs
+++ b/eth2/operation_pool/src/lib.rs
@@ -10,8 +10,8 @@ use attestation_id::AttestationId;
 use max_cover::maximum_cover;
 use parking_lot::RwLock;
 use state_processing::per_block_processing::errors::{
-    AttestationValidationError, AttesterSlashingValidationError, ExitValidationError,
-    ProposerSlashingValidationError,
+    AttestationInvalid, AttestationValidationError, AttesterSlashingValidationError,
+    ExitValidationError, ProposerSlashingValidationError,
 };
 use state_processing::per_block_processing::{
    get_slashable_indices_modular, verify_attestation_for_block_inclusion,
@@ -22,25 +22,43 @@ use std::collections::{hash_map, HashMap, HashSet};
 use std::marker::PhantomData;
 use types::{
    typenum::Unsigned, Attestation, AttesterSlashing, BeaconState, BeaconStateError, ChainSpec,
-    EthSpec, Fork, ProposerSlashing, RelativeEpoch, SignedVoluntaryExit, Validator,
+    CommitteeIndex, Epoch, EthSpec, Fork, ProposerSlashing, RelativeEpoch, SignedVoluntaryExit,
+    Slot, Validator,
 };

+/// The number of slots we keep shard subnet attestations in the operation pool for. A value of 0
+/// means we remove the attestation pool as soon as the slot ends.
+const ATTESTATION_SUBNET_SLOT_DURATION: u64 = 1;
+
 #[derive(Default, Debug)]
 pub struct OperationPool<T: EthSpec + Default> {
-    /// Map from attestation ID (see below) to vectors of attestations.
-    attestations: RwLock<HashMap<AttestationId, Vec<Attestation<T>>>>,
+    /// Map from attestation ID (see `attestation_id`) to vectors of attestations.
+    ///
+    /// These are collected from the aggregate channel. They should already be aggregated but we
+    /// check for disjoint attestations in the unlikely event we receive disjoint attestations.
+    aggregate_attestations: RwLock<HashMap<AttestationId, Vec<Attestation<T>>>>,
+    /// A collection of aggregated attestations for a particular slot and committee index.
+    ///
+    /// Un-aggregated attestations are collected on a shard subnet and if a connected validator is
+    /// required to aggregate these attestations they are aggregated and stored here until the
+    /// validator is required to publish the aggregate attestation.
+    /// This segregates attestations into (slot,committee_index) then by `AttestationId`.
+    committee_attestations:
+        RwLock<HashMap<(Slot, CommitteeIndex), HashMap<AttestationId, Attestation<T>>>>,
    /// Map from two attestation IDs to a slashing for those IDs.
    attester_slashings: RwLock<HashMap<(AttestationId, AttestationId), AttesterSlashing<T>>>,
    /// Map from proposer index to slashing.
    proposer_slashings: RwLock<HashMap<u64, ProposerSlashing>>,
    /// Map from exiting validator to their exit data.
    voluntary_exits: RwLock<HashMap<u64, SignedVoluntaryExit>>,
+    /// Marker to pin the generics.
    _phantom: PhantomData<T>,
 }

 #[derive(Debug, PartialEq)]
 pub enum OpPoolError {
    GetAttestationsTotalBalanceError(BeaconStateError),
+    NoAttestationsForSlotCommittee,
 }

 impl<T: EthSpec> OperationPool<T> {
@@ -49,12 +67,13 @@ impl<T: EthSpec> OperationPool<T> {
        Self::default()
    }

-    /// Insert an attestation into the pool, aggregating it with existing attestations if possible.
+    /// Insert an attestation from the aggregate channel into the pool, checking if the
+    /// aggregate can be further aggregated
    ///
    /// ## Note
    ///
    /// This function assumes the given `attestation` is valid.
-    pub fn insert_attestation(
+    pub fn insert_aggregate_attestation(
        &self,
        attestation: Attestation<T>,
        fork: &Fork,
@@ -63,7 +82,7 @@ impl<T: EthSpec> OperationPool<T> {
        let id = AttestationId::from_data(&attestation.data, fork, spec);

        // Take a write lock on the attestations map.
-        let mut attestations = self.attestations.write();
+        let mut attestations = self.aggregate_attestations.write();

        let existing_attestations = match attestations.entry(id) {
            hash_map::Entry::Vacant(entry) => {
@@ -90,9 +109,90 @@ impl<T: EthSpec> OperationPool<T> {
        Ok(())
    }

-    /// Total number of attestations in the pool, including attestations for the same data.
+    /// Insert a raw un-aggregated attestation into the pool, for a given (slot, committee_index).
+    ///
+    /// ## Note
+    ///
+    /// It would be a fair assumption that all attestations here are unaggregated and we
+    /// therefore do not need to check if `signers_disjoint_form`. However the cost of doing
+    /// so is low, so we perform this check for added safety.
+    pub fn insert_raw_attestation(
+        &self,
+        attestation: Attestation<T>,
+        fork: &Fork,
+        spec: &ChainSpec,
+    ) -> Result<(), AttestationValidationError> {
+        let id = AttestationId::from_data(&attestation.data, fork, spec);
+
+        let slot = attestation.data.slot.clone();
+        let committee_index = attestation.data.index.clone();
+
+        // Take a write lock on the attestations map.
+        let mut attestations = self.committee_attestations.write();
+
+        let slot_index_map = attestations
+            .entry((slot, committee_index))
+            .or_insert_with(|| HashMap::new());
+
+        let existing_attestation = match slot_index_map.entry(id) {
+            hash_map::Entry::Vacant(entry) => {
+                entry.insert(attestation);
+                return Ok(());
+            }
+            hash_map::Entry::Occupied(entry) => entry.into_mut(),
+        };
+
+        if existing_attestation.signers_disjoint_from(&attestation) {
+            existing_attestation.aggregate(&attestation);
+        } else if *existing_attestation != attestation {
+            return Err(AttestationValidationError::Invalid(
+                AttestationInvalid::NotDisjoint,
+            ));
+        }
+
+        Ok(())
+    }
+
+    /// Total number of aggregate attestations in the pool from the aggregate channel, including attestations for the same data.
    pub fn num_attestations(&self) -> usize {
-        self.attestations.read().values().map(Vec::len).sum()
+        self.aggregate_attestations
+            .read()
+            .values()
+            .map(Vec::len)
+            .sum()
+    }
+
+    /// Total number of attestations in the pool, including attestations for the same data.
+    pub fn total_num_attestations(&self) -> usize {
+        self.num_attestations().saturating_add(
+            self.committee_attestations
+                .read()
+                .values()
+                .map(|map| map.values().len())
+                .sum(),
+        )
+    }
+
+    /// Get the aggregated raw attestations for a (slot, committee)
+    //TODO: Check this logic and optimize
+    pub fn get_raw_aggregated_attestations(
+        &self,
+        slot: &Slot,
+        index: &CommitteeIndex,
+        state: &BeaconState<T>,
+        spec: &ChainSpec,
+    ) -> Result<Attestation<T>, OpPoolError> {
+        let curr_domain_bytes =
+            AttestationId::compute_domain_bytes(state.current_epoch(), &state.fork, spec);
+        self.committee_attestations
+            .read()
+            .get(&(*slot, *index))
+            .ok_or_else(|| OpPoolError::NoAttestationsForSlotCommittee)?
+            .iter()
+            .filter(|(key, _)| key.domain_bytes_match(&curr_domain_bytes))
+            .next()
+            .map(|(_key, attestation)| attestation.clone())
+            .ok_or_else(|| OpPoolError::NoAttestationsForSlotCommittee)
    }

    /// Get a list of attestations for inclusion in a block.
@@ -109,7 +209,7 @@ impl<T: EthSpec> OperationPool<T> {
        let prev_domain_bytes = AttestationId::compute_domain_bytes(prev_epoch, &state.fork, spec);
        let curr_domain_bytes =
            AttestationId::compute_domain_bytes(current_epoch, &state.fork, spec);
-        let reader = self.attestations.read();
+        let reader = self.aggregate_attestations.read();
        let active_indices = state
            .get_cached_active_validator_indices(RelativeEpoch::Current)
            .map_err(OpPoolError::GetAttestationsTotalBalanceError)?;
@@ -141,19 +241,38 @@ impl<T: EthSpec> OperationPool<T> {
        ))
    }

-    /// Remove attestations which are too old to be included in a block.
-    pub fn prune_attestations(&self, finalized_state: &BeaconState<T>) {
+    /// Removes aggregate attestations which are too old to be included in a block.
+    ///
+    /// This leaves the committee_attestations intact. The committee attestations have their own
+    /// prune function as these are not for block inclusion and can be pruned more frequently.
+    /// See `prune_committee_attestations`.
+    //TODO: Michael to check this before merge
+    pub fn prune_attestations(&self, current_epoch: &Epoch) {
        // We know we can include an attestation if:
        // state.slot <= attestation_slot + SLOTS_PER_EPOCH
        // We approximate this check using the attestation's epoch, to avoid computing
        // the slot or relying on the committee cache of the finalized state.
-        self.attestations.write().retain(|_, attestations| {
-            // All the attestations in this bucket have the same data, so we only need to
-            // check the first one.
-            attestations.first().map_or(false, |att| {
-                finalized_state.current_epoch() <= att.data.target.epoch + 1
-            })
-        });
+        self.aggregate_attestations
+            .write()
+            .retain(|_, attestations| {
+                // All the attestations in this bucket have the same data, so we only need to
+                // check the first one.
+                attestations
+                    .first()
+                    .map_or(false, |att| *current_epoch <= att.data.target.epoch + 1)
+            });
+    }
+
+    /// Removes old committee attestations. These should be used in the slot that they are
+    /// collected. We keep these around for one extra slot (i.e current_slot + 1) to account for
+    /// potential delays.
+    ///
+    /// The beacon chain should call this function every slot with the current slot as the
+    /// parameter.
+    pub fn prune_committee_attestations(&self, current_slot: &Slot) {
+        self.committee_attestations
+            .write()
+            .retain(|(slot, _), _| *slot + ATTESTATION_SUBNET_SLOT_DURATION >= *current_slot)
    }

    /// Insert a proposer slashing into the pool.
@@ -332,8 +451,8 @@ impl<T: EthSpec> OperationPool<T> {
    }

    /// Prune all types of transactions given the latest finalized state.
+    // TODO: Michael - Can we shift these to per-epoch?
    pub fn prune_all(&self, finalized_state: &BeaconState<T>, spec: &ChainSpec) {
-        self.prune_attestations(finalized_state);
        self.prune_proposer_slashings(finalized_state);
        self.prune_attester_slashings(finalized_state, spec);
        self.prune_voluntary_exits(finalized_state);
@@ -383,7 +502,8 @@ fn prune_validator_hash_map<T, F, E: EthSpec>(
 /// Compare two operation pools.
 impl<T: EthSpec + Default> PartialEq for OperationPool<T> {
    fn eq(&self, other: &Self) -> bool {
-        *self.attestations.read() == *other.attestations.read()
+        *self.aggregate_attestations.read() == *other.aggregate_attestations.read()
+            && *self.committee_attestations.read() == *other.committee_attestations.read()
            && *self.attester_slashings.read() == *other.attester_slashings.read()
            && *self.proposer_slashings.read() == *other.proposer_slashings.read()
            && *self.voluntary_exits.read() == *other.voluntary_exits.read()
@@ -397,6 +517,7 @@ mod release_tests {
    use super::*;
    use state_processing::common::{get_attesting_indices, get_base_reward};
    use std::collections::BTreeSet;
+    use std::iter::FromIterator;
    use types::test_utils::*;
    use types::*;

@@ -820,11 +941,15 @@ mod release_tests {
            let committee = state
                .get_beacon_committee(att.data.slot, att.data.index)
                .expect("should get beacon committee");
-            let att_indices = get_attesting_indices::<MainnetEthSpec>(
-                committee.committee,
-                &fresh_validators_bitlist,
-            )
-            .unwrap();
+
+            let att_indices = BTreeSet::from_iter(
+                get_attesting_indices::<MainnetEthSpec>(
+                    committee.committee,
+                    &fresh_validators_bitlist,
+                )
+                .unwrap(),
+            );
+
            let fresh_indices = &att_indices - &seen_indices;

            let rewards = fresh_indices
--- a/eth2/operation_pool/src/persistence.rs
+++ b/eth2/operation_pool/src/persistence.rs
@@ -17,7 +17,9 @@ pub struct PersistedOperationPool<T: EthSpec> {
    /// Mapping from attestation ID to attestation mappings.
    // We could save space by not storing the attestation ID, but it might
    // be difficult to make that roundtrip due to eager aggregation.
-    attestations: Vec<(AttestationId, Vec<Attestation<T>>)>,
+    // Note: That we don't store the committee attestations as these are short lived and not worth
+    // persisting
+    aggregate_attestations: Vec<(AttestationId, Vec<Attestation<T>>)>,
    /// Attester slashings.
    attester_slashings: Vec<AttesterSlashing<T>>,
    /// Proposer slashings.
@@ -29,8 +31,8 @@ pub struct PersistedOperationPool<T: EthSpec> {
 impl<T: EthSpec> PersistedOperationPool<T> {
    /// Convert an `OperationPool` into serializable form.
    pub fn from_operation_pool(operation_pool: &OperationPool<T>) -> Self {
-        let attestations = operation_pool
-            .attestations
+        let aggregate_attestations = operation_pool
+            .aggregate_attestations
            .read()
            .iter()
            .map(|(att_id, att)| (att_id.clone(), att.clone()))
@@ -58,7 +60,7 @@ impl<T: EthSpec> PersistedOperationPool<T> {
            .collect();

        Self {
-            attestations,
+            aggregate_attestations,
            attester_slashings,
            proposer_slashings,
            voluntary_exits,
@@ -67,7 +69,7 @@ impl<T: EthSpec> PersistedOperationPool<T> {

    /// Reconstruct an `OperationPool`.
    pub fn into_operation_pool(self, state: &BeaconState<T>, spec: &ChainSpec) -> OperationPool<T> {
-        let attestations = RwLock::new(self.attestations.into_iter().collect());
+        let aggregate_attestations = RwLock::new(self.aggregate_attestations.into_iter().collect());
        let attester_slashings = RwLock::new(
            self.attester_slashings
                .into_iter()
@@ -93,7 +95,8 @@ impl<T: EthSpec> PersistedOperationPool<T> {
        );

        OperationPool {
-            attestations,
+            aggregate_attestations,
+            committee_attestations: Default::default(),
            attester_slashings,
            proposer_slashings,
            voluntary_exits,
--- a/eth2/proto_array_fork_choice/Cargo.toml
+++ b/eth2/proto_array_fork_choice/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "proto_array_fork_choice"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@sigmaprime.io>"]
 edition = "2018"

--- a/eth2/state_processing/Cargo.toml
+++ b/eth2/state_processing/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "state_processing"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

@@ -15,7 +15,6 @@ serde = "1.0.102"
 serde_derive = "1.0.102"
 lazy_static = "1.4.0"
 serde_yaml = "0.8.11"
-eth2_ssz = "0.1.2"
 beacon_chain = { path = "../../beacon_node/beacon_chain" }
 store = { path = "../../beacon_node/store" }

@@ -24,6 +23,7 @@ store = { path = "../../beacon_node/store" }
 bls = { path = "../utils/bls" }
 integer-sqrt = "0.1.2"
 itertools = "0.8.1"
+eth2_ssz = "0.1.2"
 eth2_ssz_types = { path = "../utils/ssz_types" }
 merkle_proof = { path = "../utils/merkle_proof" }
 log = "0.4.8"
--- a/eth2/state_processing/src/common/get_attesting_indices.rs
+++ b/eth2/state_processing/src/common/get_attesting_indices.rs
@@ -1,4 +1,3 @@
-use std::collections::BTreeSet;
 use types::*;

 /// Returns validator indices which participated in the attestation, sorted by increasing index.
@@ -7,17 +6,20 @@ use types::*;
 pub fn get_attesting_indices<T: EthSpec>(
    committee: &[usize],
    bitlist: &BitList<T::MaxValidatorsPerCommittee>,
-) -> Result<BTreeSet<usize>, BeaconStateError> {
+) -> Result<Vec<usize>, BeaconStateError> {
    if bitlist.len() != committee.len() {
        return Err(BeaconStateError::InvalidBitfield);
    }

-    Ok(committee
-        .iter()
-        .enumerate()
-        .filter_map(|(i, validator_index)| match bitlist.get(i) {
-            Ok(true) => Some(*validator_index),
-            _ => None,
-        })
-        .collect())
+    let mut indices = Vec::with_capacity(bitlist.num_set_bits());
+
+    for (i, validator_index) in committee.iter().enumerate() {
+        if let Ok(true) = bitlist.get(i) {
+            indices.push(*validator_index)
+        }
+    }
+
+    indices.sort_unstable();
+
+    Ok(indices)
 }
--- a/eth2/state_processing/src/common/initiate_validator_exit.rs
+++ b/eth2/state_processing/src/common/initiate_validator_exit.rs
@@ -10,7 +10,7 @@ pub fn initiate_validator_exit<T: EthSpec>(
    spec: &ChainSpec,
 ) -> Result<(), Error> {
    if index >= state.validators.len() {
-        return Err(Error::UnknownValidator);
+        return Err(Error::UnknownValidator(index as u64));
    }

    // Return if the validator already initiated exit
--- a/eth2/state_processing/src/common/slash_validator.rs
+++ b/eth2/state_processing/src/common/slash_validator.rs
@@ -12,7 +12,7 @@ pub fn slash_validator<T: EthSpec>(
    spec: &ChainSpec,
 ) -> Result<(), Error> {
    if slashed_index >= state.validators.len() || slashed_index >= state.balances.len() {
-        return Err(BeaconStateError::UnknownValidator);
+        return Err(BeaconStateError::UnknownValidator(slashed_index as u64));
    }

    let epoch = state.current_epoch();
--- a/eth2/state_processing/src/lib.rs
+++ b/eth2/state_processing/src/lib.rs
@@ -10,8 +10,8 @@ pub mod test_utils;

 pub use genesis::{initialize_beacon_state_from_eth1, is_valid_genesis_state, process_activations};
 pub use per_block_processing::{
-    errors::BlockProcessingError, per_block_processing, signature_sets, BlockSignatureStrategy,
-    VerifySignatures,
+    block_signature_verifier, errors::BlockProcessingError, per_block_processing, signature_sets,
+    BlockSignatureStrategy, BlockSignatureVerifier, VerifySignatures,
 };
 pub use per_epoch_processing::{errors::EpochProcessingError, per_epoch_processing};
 pub use per_slot_processing::{per_slot_processing, Error as SlotProcessingError};
--- a/eth2/state_processing/src/per_block_processing.rs
+++ b/eth2/state_processing/src/per_block_processing.rs
@@ -1,7 +1,7 @@
 use crate::common::{initiate_validator_exit, slash_validator};
 use errors::{BlockOperationError, BlockProcessingError, HeaderInvalid, IntoWithIndex};
 use rayon::prelude::*;
-use signature_sets::{block_proposal_signature_set, randao_signature_set};
+use signature_sets::{block_proposal_signature_set, get_pubkey_from_state, randao_signature_set};
 use std::convert::TryInto;
 use tree_hash::TreeHash;
 use types::*;
@@ -21,7 +21,7 @@ pub use verify_deposit::{
 pub use verify_exit::{verify_exit, verify_exit_time_independent_only};

 pub mod block_processing_builder;
-mod block_signature_verifier;
+pub mod block_signature_verifier;
 pub mod errors;
 mod is_valid_indexed_attestation;
 pub mod signature_sets;
@@ -83,8 +83,14 @@ pub fn per_block_processing<T: EthSpec>(
        BlockSignatureStrategy::VerifyBulk => {
            // Verify all signatures in the block at once.
            block_verify!(
-                BlockSignatureVerifier::verify_entire_block(state, signed_block, block_root, spec)
-                    .is_ok(),
+                BlockSignatureVerifier::verify_entire_block(
+                    state,
+                    |i| get_pubkey_from_state(state, i),
+                    signed_block,
+                    block_root,
+                    spec
+                )
+                .is_ok(),
                BlockProcessingError::BulkSignatureVerificationFailed
            );
            VerifySignatures::False
@@ -176,7 +182,14 @@ pub fn verify_block_signature<T: EthSpec>(
    spec: &ChainSpec,
 ) -> Result<(), BlockOperationError<HeaderInvalid>> {
    verify!(
-        block_proposal_signature_set(state, block, block_root, spec)?.is_valid(),
+        block_proposal_signature_set(
+            state,
+            |i| get_pubkey_from_state(state, i),
+            block,
+            block_root,
+            spec
+        )?
+        .is_valid(),
        HeaderInvalid::ProposalSignatureInvalid
    );

@@ -196,7 +209,8 @@ pub fn process_randao<T: EthSpec>(
    if verify_signatures.is_true() {
        // Verify RANDAO reveal signature.
        block_verify!(
-            randao_signature_set(state, block, spec)?.is_valid(),
+            randao_signature_set(state, |i| get_pubkey_from_state(state, i), block, spec)?
+                .is_valid(),
            BlockProcessingError::RandaoSignatureInvalid
        );
    }
--- a/eth2/state_processing/src/per_block_processing/block_signature_verifier.rs
+++ b/eth2/state_processing/src/per_block_processing/block_signature_verifier.rs
@@ -1,14 +1,16 @@
 use super::signature_sets::{Error as SignatureSetError, Result as SignatureSetResult, *};
-
 use crate::common::get_indexed_attestation;
 use crate::per_block_processing::errors::{AttestationInvalid, BlockOperationError};
 use bls::{verify_signature_sets, SignatureSet};
 use rayon::prelude::*;
+use std::borrow::Cow;
 use types::{
    BeaconState, BeaconStateError, ChainSpec, EthSpec, Hash256, IndexedAttestation,
    SignedBeaconBlock,
 };

+pub use bls::G1Point;
+
 pub type Result<T> = std::result::Result<T, Error>;

 #[derive(Debug, PartialEq)]
@@ -46,23 +48,27 @@ impl From<BlockOperationError<AttestationInvalid>> for Error {
 ///
 /// This allows for optimizations related to batch BLS operations (see the
 /// `Self::verify_entire_block(..)` function).
-pub struct BlockSignatureVerifier<'a, T: EthSpec> {
-    block: &'a SignedBeaconBlock<T>,
+pub struct BlockSignatureVerifier<'a, T, F>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>> + Clone,
+{
+    get_pubkey: Box<F>,
    state: &'a BeaconState<T>,
    spec: &'a ChainSpec,
    sets: Vec<SignatureSet<'a>>,
 }

-impl<'a, T: EthSpec> BlockSignatureVerifier<'a, T> {
+impl<'a, T, F> BlockSignatureVerifier<'a, T, F>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>> + Clone,
+{
    /// Create a new verifier without any included signatures. See the `include...` functions to
    /// add signatures, and the `verify`
-    pub fn new(
-        state: &'a BeaconState<T>,
-        block: &'a SignedBeaconBlock<T>,
-        spec: &'a ChainSpec,
-    ) -> Self {
+    pub fn new(state: &'a BeaconState<T>, get_pubkey: F, spec: &'a ChainSpec) -> Self {
        Self {
-            block,
+            get_pubkey: Box::new(get_pubkey),
            state,
            spec,
            sets: vec![],
@@ -78,22 +84,13 @@ impl<'a, T: EthSpec> BlockSignatureVerifier<'a, T> {
    /// See `Self::verify` for more detail.
    pub fn verify_entire_block(
        state: &'a BeaconState<T>,
+        get_pubkey: F,
        block: &'a SignedBeaconBlock<T>,
        block_root: Option<Hash256>,
        spec: &'a ChainSpec,
    ) -> Result<()> {
-        let mut verifier = Self::new(state, block, spec);
-
-        verifier.include_block_proposal(block_root)?;
-        verifier.include_randao_reveal()?;
-        verifier.include_proposer_slashings()?;
-        verifier.include_attester_slashings()?;
-        verifier.include_attestations()?;
-        /*
-         * Deposits are not included because they can legally have invalid signatures.
-         */
-        verifier.include_exits()?;
-
+        let mut verifier = Self::new(state, get_pubkey, spec);
+        verifier.include_all_signatures(block, block_root)?;
        verifier.verify()
    }

@@ -127,31 +124,82 @@ impl<'a, T: EthSpec> BlockSignatureVerifier<'a, T> {
        }
    }

+    /// Includes all signatures on the block (except the deposit signatures) for verification.
+    pub fn include_all_signatures(
+        &mut self,
+        block: &'a SignedBeaconBlock<T>,
+        block_root: Option<Hash256>,
+    ) -> Result<()> {
+        self.include_block_proposal(block, block_root)?;
+        self.include_randao_reveal(block)?;
+        self.include_proposer_slashings(block)?;
+        self.include_attester_slashings(block)?;
+        self.include_attestations(block)?;
+        // Deposits are not included because they can legally have invalid signatures.
+        self.include_exits(block)?;
+
+        Ok(())
+    }
+
+    /// Includes all signatures on the block (except the deposit signatures and the proposal
+    /// signature) for verification.
+    pub fn include_all_signatures_except_proposal(
+        &mut self,
+        block: &'a SignedBeaconBlock<T>,
+    ) -> Result<()> {
+        self.include_randao_reveal(block)?;
+        self.include_proposer_slashings(block)?;
+        self.include_attester_slashings(block)?;
+        self.include_attestations(block)?;
+        // Deposits are not included because they can legally have invalid signatures.
+        self.include_exits(block)?;
+
+        Ok(())
+    }
+
    /// Includes the block signature for `self.block` for verification.
-    fn include_block_proposal(&mut self, block_root: Option<Hash256>) -> Result<()> {
-        let set = block_proposal_signature_set(self.state, self.block, block_root, self.spec)?;
+    pub fn include_block_proposal(
+        &mut self,
+        block: &'a SignedBeaconBlock<T>,
+        block_root: Option<Hash256>,
+    ) -> Result<()> {
+        let set = block_proposal_signature_set(
+            self.state,
+            self.get_pubkey.clone(),
+            block,
+            block_root,
+            self.spec,
+        )?;
        self.sets.push(set);
        Ok(())
    }

    /// Includes the randao signature for `self.block` for verification.
-    fn include_randao_reveal(&mut self) -> Result<()> {
-        let set = randao_signature_set(self.state, &self.block.message, self.spec)?;
+    pub fn include_randao_reveal(&mut self, block: &'a SignedBeaconBlock<T>) -> Result<()> {
+        let set = randao_signature_set(
+            self.state,
+            self.get_pubkey.clone(),
+            &block.message,
+            self.spec,
+        )?;
        self.sets.push(set);
        Ok(())
    }

    /// Includes all signatures in `self.block.body.proposer_slashings` for verification.
-    fn include_proposer_slashings(&mut self) -> Result<()> {
-        let mut sets: Vec<SignatureSet> = self
-            .block
+    pub fn include_proposer_slashings(&mut self, block: &'a SignedBeaconBlock<T>) -> Result<()> {
+        let mut sets: Vec<SignatureSet> = block
            .message
            .body
            .proposer_slashings
            .iter()
            .map(|proposer_slashing| {
-                let (set_1, set_2) =
-                    proposer_slashing_signature_set(self.state, proposer_slashing, self.spec)?;
+                let (set_1, set_2) = proposer_slashing_signature_set(
+                    self.state,
+                    self.get_pubkey.clone(),
+                    proposer_slashing,
+                    self.spec,
+                )?;
                Ok(vec![set_1, set_2])
            })
            .collect::<SignatureSetResult<Vec<Vec<SignatureSet>>>>()?
@@ -164,15 +212,19 @@ impl<'a, T: EthSpec> BlockSignatureVerifier<'a, T> {
    }

    /// Includes all signatures in `self.block.body.attester_slashings` for verification.
-    fn include_attester_slashings(&mut self) -> Result<()> {
-        self.block
+    pub fn include_attester_slashings(&mut self, block: &'a SignedBeaconBlock<T>) -> Result<()> {
+        block
            .message
            .body
            .attester_slashings
            .iter()
            .try_for_each(|attester_slashing| {
-                let (set_1, set_2) =
-                    attester_slashing_signature_sets(&self.state, attester_slashing, &self.spec)?;
+                let (set_1, set_2) = attester_slashing_signature_sets(
+                    &self.state,
+                    self.get_pubkey.clone(),
+                    attester_slashing,
+                    &self.spec,
+                )?;

                self.sets.push(set_1);
                self.sets.push(set_2);
@@ -182,8 +234,11 @@ impl<'a, T: EthSpec> BlockSignatureVerifier<'a, T> {
    }

    /// Includes all signatures in `self.block.body.attestations` for verification.
-    fn include_attestations(&mut self) -> Result<Vec<IndexedAttestation<T>>> {
-        self.block
+    pub fn include_attestations(
+        &mut self,
+        block: &'a SignedBeaconBlock<T>,
+    ) -> Result<Vec<IndexedAttestation<T>>> {
+        block
            .message
            .body
            .attestations
@@ -197,6 +252,7 @@ impl<'a, T: EthSpec> BlockSignatureVerifier<'a, T> {

                self.sets.push(indexed_attestation_signature_set(
                    &self.state,
+                    self.get_pubkey.clone(),
                    &attestation.signature,
                    &indexed_attestation,
                    &self.spec,
@@ -209,14 +265,13 @@ impl<'a, T: EthSpec> BlockSignatureVerifier<'a, T> {
    }

    /// Includes all signatures in `self.block.body.voluntary_exits` for verification.
-    fn include_exits(&mut self) -> Result<()> {
-        let mut sets = self
-            .block
+    pub fn include_exits(&mut self, block: &'a SignedBeaconBlock<T>) -> Result<()> {
+        let mut sets = block
            .message
            .body
            .voluntary_exits
            .iter()
-            .map(|exit| exit_signature_set(&self.state, exit, &self.spec))
+            .map(|exit| exit_signature_set(&self.state, self.get_pubkey.clone(), exit, &self.spec))
            .collect::<SignatureSetResult<_>>()?;

        self.sets.append(&mut sets);
--- a/eth2/state_processing/src/per_block_processing/errors.rs
+++ b/eth2/state_processing/src/per_block_processing/errors.rs
@@ -235,6 +235,8 @@ pub enum AttestationInvalid {
        committee_len: usize,
        bitfield_len: usize,
    },
+    /// The attestation was not disjoint compared to already seen attestations.
+    NotDisjoint,
    /// The validator index was unknown.
    UnknownValidator(u64),
    /// The attestation signature verification failed.
--- a/eth2/state_processing/src/per_block_processing/is_valid_indexed_attestation.rs
+++ b/eth2/state_processing/src/per_block_processing/is_valid_indexed_attestation.rs
@@ -1,5 +1,5 @@
 use super::errors::{BlockOperationError, IndexedAttestationInvalid as Invalid};
-use super::signature_sets::indexed_attestation_signature_set;
+use super::signature_sets::{get_pubkey_from_state, indexed_attestation_signature_set};
 use crate::VerifySignatures;
 use types::*;

@@ -43,6 +43,7 @@ pub fn is_valid_indexed_attestation<T: EthSpec>(
        verify!(
            indexed_attestation_signature_set(
                state,
+                |i| get_pubkey_from_state(state, i),
                &indexed_attestation.signature,
                &indexed_attestation,
                spec
--- a/eth2/state_processing/src/per_block_processing/signature_sets.rs
+++ b/eth2/state_processing/src/per_block_processing/signature_sets.rs
@@ -3,6 +3,7 @@
 //!
 //! This module exposes one function to extract each type of `SignatureSet` from a `BeaconBlock`.
 use bls::{G1Point, G1Ref, SignatureSet, SignedMessage};
+use ssz::DecodeError;
 use std::borrow::Cow;
 use std::convert::TryInto;
 use tree_hash::TreeHash;
@@ -18,7 +19,7 @@ pub type Result<T> = std::result::Result<T, Error>;
 #[derive(Debug, PartialEq, Clone)]
 pub enum Error {
    /// Signature verification failed. The block is invalid.
-    SignatureInvalid,
+    SignatureInvalid(DecodeError),
    /// There was an error attempting to read from a `BeaconState`. Block
    /// validity was not determined.
    BeaconStateError(BeaconStateError),
@@ -39,13 +40,36 @@ impl From<BeaconStateError> for Error {
    }
 }

-/// A signature set that is valid if a block was signed by the expected block producer.
-pub fn block_proposal_signature_set<'a, T: EthSpec>(
+/// Helper function to get a public key from a `state`.
+pub fn get_pubkey_from_state<'a, T>(
    state: &'a BeaconState<T>,
+    validator_index: usize,
+) -> Option<Cow<'a, G1Point>>
+where
+    T: EthSpec,
+{
+    state
+        .validators
+        .get(validator_index)
+        .and_then(|v| {
+            let pk: Option<PublicKey> = (&v.pubkey).try_into().ok();
+            pk
+        })
+        .map(|pk| Cow::Owned(pk.into_point()))
+}
+
+/// A signature set that is valid if a block was signed by the expected block producer.
+pub fn block_proposal_signature_set<'a, T, F>(
+    state: &'a BeaconState<T>,
+    get_pubkey: F,
    signed_block: &'a SignedBeaconBlock<T>,
    block_root: Option<Hash256>,
    spec: &'a ChainSpec,
-) -> Result<SignatureSet<'a>> {
+) -> Result<SignatureSet<'a>>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>>,
+{
    let block = &signed_block.message;
    let proposer_index = state.get_beacon_proposer_index(block.slot, spec)?;

@@ -67,17 +91,22 @@ pub fn block_proposal_signature_set<'a, T: EthSpec>(

    Ok(SignatureSet::single(
        &signed_block.signature,
-        validator_pubkey(state, proposer_index)?,
+        get_pubkey(proposer_index).ok_or_else(|| Error::ValidatorUnknown(proposer_index as u64))?,
        message.as_bytes().to_vec(),
    ))
 }

 /// A signature set that is valid if the block proposers randao reveal signature is correct.
-pub fn randao_signature_set<'a, T: EthSpec>(
+pub fn randao_signature_set<'a, T, F>(
    state: &'a BeaconState<T>,
+    get_pubkey: F,
    block: &'a BeaconBlock<T>,
    spec: &'a ChainSpec,
-) -> Result<SignatureSet<'a>> {
+) -> Result<SignatureSet<'a>>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>>,
+{
    let proposer_index = state.get_beacon_proposer_index(block.slot, spec)?;

    let domain = spec.get_domain(
@@ -86,34 +115,41 @@ pub fn randao_signature_set<'a, T: EthSpec>(
        &state.fork,
    );

-    let message = state.current_epoch().signing_root(domain);
+    let message = block.slot.epoch(T::slots_per_epoch()).signing_root(domain);

    Ok(SignatureSet::single(
        &block.body.randao_reveal,
-        validator_pubkey(state, proposer_index)?,
+        get_pubkey(proposer_index).ok_or_else(|| Error::ValidatorUnknown(proposer_index as u64))?,
        message.as_bytes().to_vec(),
    ))
 }

 /// Returns two signature sets, one for each `BlockHeader` included in the `ProposerSlashing`.
-pub fn proposer_slashing_signature_set<'a, T: EthSpec>(
+pub fn proposer_slashing_signature_set<'a, T, F>(
    state: &'a BeaconState<T>,
+    get_pubkey: F,
    proposer_slashing: &'a ProposerSlashing,
    spec: &'a ChainSpec,
-) -> Result<(SignatureSet<'a>, SignatureSet<'a>)> {
+) -> Result<(SignatureSet<'a>, SignatureSet<'a>)>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>>,
+{
    let proposer_index = proposer_slashing.proposer_index as usize;

    Ok((
        block_header_signature_set(
            state,
            &proposer_slashing.signed_header_1,
-            validator_pubkey(state, proposer_index)?,
+            get_pubkey(proposer_index)
+                .ok_or_else(|| Error::ValidatorUnknown(proposer_index as u64))?,
            spec,
        )?,
        block_header_signature_set(
            state,
            &proposer_slashing.signed_header_2,
-            validator_pubkey(state, proposer_index)?,
+            get_pubkey(proposer_index)
+                .ok_or_else(|| Error::ValidatorUnknown(proposer_index as u64))?,
            spec,
        )?,
    ))
@@ -146,16 +182,24 @@ fn block_header_signature_set<'a, T: EthSpec>(
 }

 /// Returns the signature set for the given `indexed_attestation`.
-pub fn indexed_attestation_signature_set<'a, 'b, T: EthSpec>(
+pub fn indexed_attestation_signature_set<'a, 'b, T, F>(
    state: &'a BeaconState<T>,
+    get_pubkey: F,
    signature: &'a AggregateSignature,
    indexed_attestation: &'b IndexedAttestation<T>,
    spec: &'a ChainSpec,
-) -> Result<SignatureSet<'a>> {
+) -> Result<SignatureSet<'a>>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>>,
+{
    let pubkeys = indexed_attestation
        .attesting_indices
        .into_iter()
-        .map(|&validator_idx| Ok(validator_pubkey(state, validator_idx as usize)?))
+        .map(|&validator_idx| {
+            Ok(get_pubkey(validator_idx as usize)
+                .ok_or_else(|| Error::ValidatorUnknown(validator_idx))?)
+        })
        .collect::<Result<_>>()?;

    let domain = spec.get_domain(
@@ -172,17 +216,25 @@ pub fn indexed_attestation_signature_set<'a, 'b, T: EthSpec>(

 /// Returns the signature set for the given `indexed_attestation` but pubkeys are supplied directly
 /// instead of from the state.
-pub fn indexed_attestation_signature_set_from_pubkeys<'a, 'b, T: EthSpec>(
-    pubkeys: Vec<&'a PublicKey>,
+pub fn indexed_attestation_signature_set_from_pubkeys<'a, 'b, T, F>(
+    get_pubkey: F,
    signature: &'a AggregateSignature,
    indexed_attestation: &'b IndexedAttestation<T>,
    fork: &Fork,
    spec: &'a ChainSpec,
-) -> Result<SignatureSet<'a>> {
-    let pubkeys = pubkeys
+) -> Result<SignatureSet<'a>>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>>,
+{
+    let pubkeys = indexed_attestation
+        .attesting_indices
        .into_iter()
-        .map(|pubkey| Cow::Borrowed(&pubkey.as_raw().point))
-        .collect();
+        .map(|&validator_idx| {
+            Ok(get_pubkey(validator_idx as usize)
+                .ok_or_else(|| Error::ValidatorUnknown(validator_idx))?)
+        })
+        .collect::<Result<_>>()?;

    let domain = spec.get_domain(
        indexed_attestation.data.target.epoch,
@@ -197,20 +249,27 @@ pub fn indexed_attestation_signature_set_from_pubkeys<'a, 'b, T: EthSpec>(
 }

 /// Returns the signature set for the given `attester_slashing` and corresponding `pubkeys`.
-pub fn attester_slashing_signature_sets<'a, T: EthSpec>(
+pub fn attester_slashing_signature_sets<'a, T, F>(
    state: &'a BeaconState<T>,
+    get_pubkey: F,
    attester_slashing: &'a AttesterSlashing<T>,
    spec: &'a ChainSpec,
-) -> Result<(SignatureSet<'a>, SignatureSet<'a>)> {
+) -> Result<(SignatureSet<'a>, SignatureSet<'a>)>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>> + Clone,
+{
    Ok((
        indexed_attestation_signature_set(
            state,
+            get_pubkey.clone(),
            &attester_slashing.attestation_1.signature,
            &attester_slashing.attestation_1,
            spec,
        )?,
        indexed_attestation_signature_set(
            state,
+            get_pubkey,
            &attester_slashing.attestation_2.signature,
            &attester_slashing.attestation_2,
            spec,
@@ -250,11 +309,16 @@ pub fn deposit_signature_set<'a>(

 /// Returns a signature set that is valid if the `SignedVoluntaryExit` was signed by the indicated
 /// validator.
-pub fn exit_signature_set<'a, T: EthSpec>(
+pub fn exit_signature_set<'a, T, F>(
    state: &'a BeaconState<T>,
+    get_pubkey: F,
    signed_exit: &'a SignedVoluntaryExit,
    spec: &'a ChainSpec,
-) -> Result<SignatureSet<'a>> {
+) -> Result<SignatureSet<'a>>
+where
+    T: EthSpec,
+    F: Fn(usize) -> Option<Cow<'a, G1Point>>,
+{
    let exit = &signed_exit.message;
    let proposer_index = exit.validator_index as usize;

@@ -264,30 +328,7 @@ pub fn exit_signature_set<'a, T: EthSpec>(

    Ok(SignatureSet::single(
        &signed_exit.signature,
-        validator_pubkey(state, proposer_index)?,
+        get_pubkey(proposer_index).ok_or_else(|| Error::ValidatorUnknown(proposer_index as u64))?,
        message,
    ))
 }
-
-/// Maps a validator index to a `PublicKey`.
-pub fn validator_pubkey<'a, T: EthSpec>(
-    state: &'a BeaconState<T>,
-    validator_index: usize,
-) -> Result<Cow<'a, G1Point>> {
-    let pubkey_bytes = &state
-        .validators
-        .get(validator_index)
-        .ok_or_else(|| Error::ValidatorUnknown(validator_index as u64))?
-        .pubkey;
-
-    if let Some(pubkey) = pubkey_bytes.decompressed() {
-        Ok(Cow::Borrowed(&pubkey.as_raw().point))
-    } else {
-        pubkey_bytes
-            .try_into()
-            .map(|pubkey: PublicKey| Cow::Owned(pubkey.as_raw().point.clone()))
-            .map_err(|_| Error::BadBlsBytes {
-                validator_index: validator_index as u64,
-            })
-    }
-}
--- a/eth2/state_processing/src/per_block_processing/verify_exit.rs
+++ b/eth2/state_processing/src/per_block_processing/verify_exit.rs
@@ -1,5 +1,8 @@
 use super::errors::{BlockOperationError, ExitInvalid};
-use crate::per_block_processing::{signature_sets::exit_signature_set, VerifySignatures};
+use crate::per_block_processing::{
+    signature_sets::{exit_signature_set, get_pubkey_from_state},
+    VerifySignatures,
+};
 use types::*;

 type Result<T> = std::result::Result<T, BlockOperationError<ExitInvalid>>;
@@ -84,7 +87,13 @@ fn verify_exit_parametric<T: EthSpec>(

    if verify_signatures.is_true() {
        verify!(
-            exit_signature_set(state, signed_exit, spec)?.is_valid(),
+            exit_signature_set(
+                state,
+                |i| get_pubkey_from_state(state, i),
+                signed_exit,
+                spec
+            )?
+            .is_valid(),
            ExitInvalid::BadSignature
        );
    }
--- a/eth2/state_processing/src/per_block_processing/verify_proposer_slashing.rs
+++ b/eth2/state_processing/src/per_block_processing/verify_proposer_slashing.rs
@@ -1,5 +1,5 @@
 use super::errors::{BlockOperationError, ProposerSlashingInvalid as Invalid};
-use super::signature_sets::proposer_slashing_signature_set;
+use super::signature_sets::{get_pubkey_from_state, proposer_slashing_signature_set};
 use crate::VerifySignatures;
 use types::*;

@@ -49,8 +49,12 @@ pub fn verify_proposer_slashing<T: EthSpec>(
    );

    if verify_signatures.is_true() {
-        let (signature_set_1, signature_set_2) =
-            proposer_slashing_signature_set(state, proposer_slashing, spec)?;
+        let (signature_set_1, signature_set_2) = proposer_slashing_signature_set(
+            state,
+            |i| get_pubkey_from_state(state, i),
+            proposer_slashing,
+            spec,
+        )?;
        verify!(signature_set_1.is_valid(), Invalid::BadProposal1Signature);
        verify!(signature_set_2.is_valid(), Invalid::BadProposal2Signature);
    }
--- a/eth2/types/Cargo.toml
+++ b/eth2/types/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "types"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>", "Age Manning <Age@AgeManning.com>"]
 edition = "2018"

--- a/eth2/types/src/aggregate_and_proof.rs
+++ b/eth2/types/src/aggregate_and_proof.rs
@@ -0,0 +1,80 @@
+use super::{Attestation, Domain, EthSpec, Fork, PublicKey, SecretKey, Signature, SignedRoot};
+use crate::test_utils::TestRandom;
+use serde_derive::{Deserialize, Serialize};
+use ssz_derive::{Decode, Encode};
+use test_random_derive::TestRandom;
+use tree_hash_derive::TreeHash;
+
+/// A Validators aggregate attestation and selection proof.
+///
+/// Spec v0.10.1
+#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Encode, Decode, TestRandom, TreeHash)]
+#[serde(bound = "T: EthSpec")]
+pub struct AggregateAndProof<T: EthSpec> {
+    /// The index of the validator that created the attestation.
+    pub aggregator_index: u64,
+    /// The aggregate attestation.
+    pub aggregate: Attestation<T>,
+    /// A proof provided by the validator that permits them to publish on the
+    /// `beacon_aggregate_and_proof` gossipsub topic.
+    pub selection_proof: Signature,
+}
+
+impl<T: EthSpec> AggregateAndProof<T> {
+    pub fn is_valid_selection_proof(&self, validator_pubkey: &PublicKey, fork: &Fork) -> bool {
+        let target_epoch = self.aggregate.data.slot.epoch(T::slots_per_epoch());
+        let domain = T::default_spec().get_domain(target_epoch, Domain::SelectionProof, fork);
+        let message = self.aggregate.data.slot.signing_root(domain);
+        self.selection_proof
+            .verify(message.as_bytes(), validator_pubkey)
+    }
+
+    /// Converts Self into a SignedAggregateAndProof.
+    pub fn into_signed(self, secret_key: &SecretKey, fork: &Fork) -> SignedAggregateAndProof<T> {
+        let target_epoch = self.aggregate.data.slot.epoch(T::slots_per_epoch());
+        let domain = T::default_spec().get_domain(target_epoch, Domain::AggregateAndProof, fork);
+        let sign_message = self.signing_root(domain);
+        let signature = Signature::new(sign_message.as_bytes(), &secret_key);
+
+        SignedAggregateAndProof {
+            message: self,
+            signature,
+        }
+    }
+}
+
+impl<T: EthSpec> SignedRoot for AggregateAndProof<T> {}
+
+/// A Validators signed aggregate proof to publish on the `beacon_aggregate_and_proof`
+/// gossipsub topic.
+///
+/// Spec v0.10.1
+#[derive(Debug, Clone, PartialEq, Serialize, Deserialize, Encode, Decode, TestRandom, TreeHash)]
+#[serde(bound = "T: EthSpec")]
+pub struct SignedAggregateAndProof<T: EthSpec> {
+    /// The `AggregateAndProof` that was signed.
+    pub message: AggregateAndProof<T>,
+    /// The aggregate attestation.
+    pub signature: Signature,
+}
+
+impl<T: EthSpec> SignedRoot for SignedAggregateAndProof<T> {}
+
+impl<T: EthSpec> SignedAggregateAndProof<T> {
+    /// Verifies the signature of the `AggregateAndProof`
+    pub fn is_valid_signature(&self, validator_pubkey: &PublicKey, fork: &Fork) -> bool {
+        let target_epoch = self.message.aggregate.data.slot.epoch(T::slots_per_epoch());
+        let domain = T::default_spec().get_domain(target_epoch, Domain::AggregateAndProof, fork);
+        let message = self.signing_root(domain);
+        self.signature.verify(message.as_bytes(), validator_pubkey)
+    }
+
+    /// Verifies the signature of the `AggregateAndProof` as well the underlying selection_proof in
+    /// the contained `AggregateAndProof`.
+    pub fn is_valid(&self, validator_pubkey: &PublicKey, fork: &Fork) -> bool {
+        self.is_valid_signature(validator_pubkey, fork)
+            && self
+                .message
+                .is_valid_selection_proof(validator_pubkey, fork)
+    }
+}
--- a/eth2/types/src/attestation.rs
+++ b/eth2/types/src/attestation.rs
@@ -1,6 +1,6 @@
 use super::{
    AggregateSignature, AttestationData, BitList, ChainSpec, Domain, EthSpec, Fork, SecretKey,
-    Signature, SignedRoot,
+    Signature, SignedRoot, SubnetId,
 };
 use crate::test_utils::TestRandom;

@@ -75,6 +75,14 @@ impl<T: EthSpec> Attestation<T> {
            Ok(())
        }
    }
+
+    /// Returns the subnet id associated with the attestation.
+    ///
+    /// Note, this will return the subnet id for an aggregated attestation. This is done
+    /// to avoid checking aggregate bits every time we wish to get an id.
+    pub fn subnet_id(&self) -> SubnetId {
+        SubnetId::new(self.data.index % T::default_spec().attestation_subnet_count)
+    }
 }

 #[cfg(test)]
--- a/eth2/types/src/beacon_state.rs
+++ b/eth2/types/src/beacon_state.rs
@@ -11,6 +11,7 @@ use serde_derive::{Deserialize, Serialize};
 use ssz::ssz_encode;
 use ssz_derive::{Decode, Encode};
 use ssz_types::{typenum::Unsigned, BitVector, FixedVector};
+use std::convert::TryInto;
 use swap_or_not_shuffle::compute_shuffled_index;
 use test_random_derive::TestRandom;
 use tree_hash::TreeHash;
@@ -36,7 +37,7 @@ const MAX_RANDOM_BYTE: u64 = (1 << 8) - 1;
 pub enum Error {
    EpochOutOfBounds,
    SlotOutOfBounds,
-    UnknownValidator,
+    UnknownValidator(u64),
    UnableToDetermineProducer,
    InvalidBitfield,
    ValidatorIsWithdrawable,
@@ -332,6 +333,7 @@ impl<T: EthSpec> BeaconState<T> {
    ///
    /// Spec v0.10.1
    pub fn get_active_validator_indices(&self, epoch: Epoch) -> Vec<usize> {
+        // FIXME(sproul): put a bounds check on here based on the maximum lookahead
        get_active_validator_indices(&self.validators, epoch)
    }

@@ -427,6 +429,30 @@ impl<T: EthSpec> BeaconState<T> {
        }
    }

+    /// Return `true` if the validator who produced `slot_signature` is eligible to aggregate.
+    ///
+    /// Spec v0.10.1
+    pub fn is_aggregator(
+        &self,
+        slot: Slot,
+        index: CommitteeIndex,
+        slot_signature: &Signature,
+        spec: &ChainSpec,
+    ) -> Result<bool, Error> {
+        let committee = self.get_beacon_committee(slot, index)?;
+        let modulo = std::cmp::max(
+            1,
+            committee.committee.len() as u64 / spec.target_aggregators_per_committee,
+        );
+        let signature_hash = hash(&slot_signature.as_bytes());
+        let signature_hash_int = u64::from_le_bytes(
+            signature_hash[0..8]
+                .try_into()
+                .expect("first 8 bytes of signature should always convert to fixed array"),
+        );
+        Ok(signature_hash_int % modulo == 0)
+    }
+
    /// Returns the beacon proposer index for the `slot` in the given `relative_epoch`.
    ///
    /// Spec v0.10.1
@@ -710,7 +736,7 @@ impl<T: EthSpec> BeaconState<T> {
        self.validators
            .get(validator_index)
            .map(|v| v.effective_balance)
-            .ok_or_else(|| Error::UnknownValidator)
+            .ok_or_else(|| Error::UnknownValidator(validator_index as u64))
    }

    ///  Return the epoch at which an activation or exit triggered in ``epoch`` takes effect.
@@ -734,7 +760,7 @@ impl<T: EthSpec> BeaconState<T> {
        ))
    }

-    /// Returns the `slot`, `index` and `committee_position` for which a validator must produce an
+    /// Returns the `slot`, `index`, `committee_position` and `committee_len` for which a validator must produce an
    /// attestation.
    ///
    /// Note: Utilizes the cache and will fail if the appropriate cache is not initialized.
@@ -770,7 +796,6 @@ impl<T: EthSpec> BeaconState<T> {
        self.update_pubkey_cache()?;
        self.build_tree_hash_cache()?;
        self.exit_cache.build(&self.validators, spec)?;
-        self.decompress_validator_pubkeys()?;

        Ok(())
    }
@@ -938,23 +963,6 @@ impl<T: EthSpec> BeaconState<T> {
        self.tree_hash_cache = None;
    }

-    /// Iterate through all validators and decompress their public key, unless it has already been
-    /// decompressed.
-    ///
-    /// Does not check the validity of already decompressed keys.
-    pub fn decompress_validator_pubkeys(&mut self) -> Result<(), Error> {
-        self.validators.iter_mut().try_for_each(|validator| {
-            if validator.pubkey.decompressed().is_none() {
-                validator
-                    .pubkey
-                    .decompress()
-                    .map_err(Error::InvalidValidatorPubkey)
-            } else {
-                Ok(())
-            }
-        })
-    }
-
    /// Clone the state whilst preserving only the selected caches.
    pub fn clone_with(&self, config: CloneConfig) -> Self {
        BeaconState {
--- a/eth2/types/src/chain_spec.rs
+++ b/eth2/types/src/chain_spec.rs
@@ -17,6 +17,8 @@ pub enum Domain {
    Randao,
    Deposit,
    VoluntaryExit,
+    SelectionProof,
+    AggregateAndProof,
 }

 /// Holds all the "constants" for a BeaconChain.
@@ -93,6 +95,8 @@ pub struct ChainSpec {
    domain_randao: u32,
    domain_deposit: u32,
    domain_voluntary_exit: u32,
+    domain_selection_proof: u32,
+    domain_aggregate_and_proof: u32,

    /*
     * Fork choice
@@ -105,8 +109,17 @@ pub struct ChainSpec {
    pub eth1_follow_distance: u64,
    pub seconds_per_eth1_block: u64,

+    /*
+     * Networking
+     */
    pub boot_nodes: Vec<String>,
    pub network_id: u8,
+    pub attestation_propagation_slot_range: u64,
+    pub maximum_gossip_clock_disparity_millis: u64,
+    pub target_aggregators_per_committee: u64,
+    pub attestation_subnet_count: u64,
+    pub random_subnets_per_validator: u64,
+    pub epochs_per_random_subnet_subscription: u64,
 }

 impl ChainSpec {
@@ -120,6 +133,8 @@ impl ChainSpec {
            Domain::Randao => self.domain_randao,
            Domain::Deposit => self.domain_deposit,
            Domain::VoluntaryExit => self.domain_voluntary_exit,
+            Domain::SelectionProof => self.domain_selection_proof,
+            Domain::AggregateAndProof => self.domain_aggregate_and_proof,
        }
    }

@@ -223,6 +238,8 @@ impl ChainSpec {
            domain_randao: 2,
            domain_deposit: 3,
            domain_voluntary_exit: 4,
+            domain_selection_proof: 5,
+            domain_aggregate_and_proof: 6,

            /*
             * Fork choice
@@ -240,6 +257,12 @@ impl ChainSpec {
             */
            boot_nodes: vec![],
            network_id: 1, // mainnet network id
+            attestation_propagation_slot_range: 32,
+            attestation_subnet_count: 64,
+            random_subnets_per_validator: 1,
+            maximum_gossip_clock_disparity_millis: 500,
+            target_aggregators_per_committee: 16,
+            epochs_per_random_subnet_subscription: 256,
        }
    }

--- a/eth2/types/src/eth_spec.rs
+++ b/eth2/types/src/eth_spec.rs
@@ -12,6 +12,7 @@ pub trait EthSpec: 'static + Default + Sync + Send + Clone + Debug + PartialEq {
     */
    type GenesisEpoch: Unsigned + Clone + Sync + Send + Debug + PartialEq;
    type JustificationBitsLength: Unsigned + Clone + Sync + Send + Debug + PartialEq + Default;
+    type SubnetBitfieldLength: Unsigned + Clone + Sync + Send + Debug + PartialEq + Default;
    /*
     * Misc
     */
@@ -125,6 +126,7 @@ pub struct MainnetEthSpec;

 impl EthSpec for MainnetEthSpec {
    type JustificationBitsLength = U4;
+    type SubnetBitfieldLength = U64;
    type MaxValidatorsPerCommittee = U2048;
    type GenesisEpoch = U0;
    type SlotsPerEpoch = U32;
@@ -164,6 +166,7 @@ impl EthSpec for MinimalEthSpec {

    params_from_eth_spec!(MainnetEthSpec {
        JustificationBitsLength,
+        SubnetBitfieldLength,
        MaxValidatorsPerCommittee,
        GenesisEpoch,
        HistoricalRootsLimit,
@@ -196,6 +199,7 @@ impl EthSpec for InteropEthSpec {

    params_from_eth_spec!(MainnetEthSpec {
        JustificationBitsLength,
+        SubnetBitfieldLength,
        MaxValidatorsPerCommittee,
        GenesisEpoch,
        HistoricalRootsLimit,
--- a/eth2/types/src/lib.rs
+++ b/eth2/types/src/lib.rs
@@ -6,6 +6,7 @@
 #[macro_use]
 pub mod test_utils;

+pub mod aggregate_and_proof;
 pub mod attestation;
 pub mod attestation_data;
 pub mod attestation_duty;
@@ -39,10 +40,12 @@ pub mod voluntary_exit;
 #[macro_use]
 pub mod slot_epoch_macros;
 pub mod slot_epoch;
+pub mod subnet_id;
 mod tree_hash_impls;

 use ethereum_types::{H160, H256};

+pub use crate::aggregate_and_proof::{AggregateAndProof, SignedAggregateAndProof};
 pub use crate::attestation::{Attestation, Error as AttestationError};
 pub use crate::attestation_data::AttestationData;
 pub use crate::attestation_duty::AttestationDuty;
@@ -70,6 +73,7 @@ pub use crate::signed_beacon_block_header::SignedBeaconBlockHeader;
 pub use crate::signed_voluntary_exit::SignedVoluntaryExit;
 pub use crate::signing_root::{SignedRoot, SigningRoot};
 pub use crate::slot_epoch::{Epoch, Slot};
+pub use crate::subnet_id::SubnetId;
 pub use crate::validator::Validator;
 pub use crate::voluntary_exit::VoluntaryExit;

--- a/eth2/types/src/slot_epoch.rs
+++ b/eth2/types/src/slot_epoch.rs
@@ -94,6 +94,7 @@ impl Epoch {
 }

 impl SignedRoot for Epoch {}
+impl SignedRoot for Slot {}

 pub struct SlotIter<'a> {
    current_iteration: u64,
--- a/eth2/types/src/subnet_id.rs
+++ b/eth2/types/src/subnet_id.rs
@@ -0,0 +1,26 @@
+//! Identifies each shard by an integer identifier.
+use serde_derive::{Deserialize, Serialize};
+use std::ops::{Deref, DerefMut};
+
+#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
+pub struct SubnetId(u64);
+
+impl SubnetId {
+    pub fn new(id: u64) -> Self {
+        SubnetId(id)
+    }
+}
+
+impl Deref for SubnetId {
+    type Target = u64;
+
+    fn deref(&self) -> &Self::Target {
+        &self.0
+    }
+}
+
+impl DerefMut for SubnetId {
+    fn deref_mut(&mut self) -> &mut Self::Target {
+        &mut self.0
+    }
+}
--- a/eth2/utils/bls/Cargo.toml
+++ b/eth2/utils/bls/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "bls"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/bls/src/fake_public_key.rs
+++ b/eth2/utils/bls/src/fake_public_key.rs
@@ -85,6 +85,14 @@ impl FakePublicKey {
    pub fn as_raw(&self) -> &Self {
        self
    }
+
+    pub fn as_point(&self) -> &G1Point {
+        &self.point
+    }
+
+    pub fn into_point(self) -> G1Point {
+        self.point
+    }
 }

 impl fmt::Display for FakePublicKey {
--- a/eth2/utils/bls/src/macros.rs
+++ b/eth2/utils/bls/src/macros.rs
@@ -92,7 +92,6 @@ macro_rules! bytes_struct {
        #[derive(Clone)]
        pub struct $name {
            bytes: [u8; $byte_size],
-            decompressed: Option<$type>
        }
    };
    ($name: ident, $type: ty, $byte_size: expr, $small_name: expr) => {
@@ -103,14 +102,12 @@ macro_rules! bytes_struct {
            pub fn from_bytes(bytes: &[u8]) -> Result<Self, ssz::DecodeError> {
                Ok(Self {
                    bytes: Self::get_bytes(bytes)?,
-                    decompressed: None
                })
            }

            pub fn empty() -> Self {
                Self {
                    bytes: [0; $byte_size],
-                    decompressed: None
                }
            }

@@ -134,15 +131,6 @@ macro_rules! bytes_struct {
                    Ok(result)
                }
            }
-
-            pub fn decompress(&mut self) -> Result<(), ssz::DecodeError> {
-                self.decompressed = Some(<&Self as std::convert::TryInto<$type>>::try_into(self)?);
-                Ok(())
-            }
-
-            pub fn decompressed(&self) -> &Option<$type> {
-                &self.decompressed
-            }
        }

        impl std::fmt::Debug for $name {
@@ -180,7 +168,44 @@ macro_rules! bytes_struct {
            }
        }

-        impl_ssz!($name, $byte_size, "$type");
+        impl ssz::Encode for $name {
+            fn is_ssz_fixed_len() -> bool {
+                true
+            }
+
+            fn ssz_fixed_len() -> usize {
+                $byte_size
+            }
+
+            fn ssz_bytes_len(&self) -> usize {
+                $byte_size
+            }
+
+            fn ssz_append(&self, buf: &mut Vec<u8>) {
+                buf.extend_from_slice(&self.bytes)
+            }
+        }
+
+        impl ssz::Decode for $name {
+            fn is_ssz_fixed_len() -> bool {
+                true
+            }
+
+            fn ssz_fixed_len() -> usize {
+                $byte_size
+            }
+
+            fn from_ssz_bytes(bytes: &[u8]) -> Result<Self, DecodeError> {
+                let len = bytes.len();
+                let expected = <Self as ssz::Decode>::ssz_fixed_len();
+
+                if len != expected {
+                    Err(ssz::DecodeError::InvalidByteLength { len, expected })
+                } else {
+                    Self::from_bytes(bytes)
+                }
+            }
+        }

        impl tree_hash::TreeHash for $name {
            fn tree_hash_type() -> tree_hash::TreeHashType {
--- a/eth2/utils/bls/src/public_key.rs
+++ b/eth2/utils/bls/src/public_key.rs
@@ -1,5 +1,5 @@
 use super::{SecretKey, BLS_PUBLIC_KEY_BYTE_SIZE};
-use milagro_bls::PublicKey as RawPublicKey;
+use milagro_bls::{G1Point, PublicKey as RawPublicKey};
 use serde::de::{Deserialize, Deserializer};
 use serde::ser::{Serialize, Serializer};
 use serde_hex::{encode as hex_encode, PrefixedHexVisitor};
@@ -24,11 +24,21 @@ impl PublicKey {
        Self(raw)
    }

-    /// Returns the underlying signature.
+    /// Returns a reference to the underlying signature.
    pub fn as_raw(&self) -> &RawPublicKey {
        &self.0
    }

+    /// Consumes self and returns the underlying signature.
+    pub fn as_point(&self) -> &G1Point {
+        &self.0.point
+    }
+
+    /// Consumes self and returns the underlying signature.
+    pub fn into_point(self) -> G1Point {
+        self.0.point
+    }
+
    /// Returns the underlying point as compressed bytes.
    ///
    /// Identical to `self.as_uncompressed_bytes()`.
--- a/eth2/utils/compare_fields/Cargo.toml
+++ b/eth2/utils/compare_fields/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "compare_fields"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/compare_fields_derive/Cargo.toml
+++ b/eth2/utils/compare_fields_derive/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "compare_fields_derive"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/deposit_contract/Cargo.toml
+++ b/eth2/utils/deposit_contract/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "deposit_contract"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/eth2_config/Cargo.toml
+++ b/eth2/utils/eth2_config/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "eth2_config"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/eth2_interop_keypairs/Cargo.toml
+++ b/eth2/utils/eth2_interop_keypairs/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "eth2_interop_keypairs"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/eth2_testnet_config/Cargo.toml
+++ b/eth2/utils/eth2_testnet_config/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "eth2_testnet_config"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/hashmap_delay/Cargo.toml
+++ b/eth2/utils/hashmap_delay/Cargo.toml
@@ -0,0 +1,9 @@
+[package]
+name = "hashmap_delay"
+version = "0.2.0"
+authors = ["Age Manning <Age@AgeManning.com>"]
+edition = "2018"
+
+[dependencies]
+tokio-timer = "0.2.12"
+futures = "0.1.29"
--- a/eth2/utils/hashmap_delay/src/hashmap_delay.rs
+++ b/eth2/utils/hashmap_delay/src/hashmap_delay.rs
@@ -0,0 +1,161 @@
+//! A simple hashmap object coupled with a `delay_queue` which has entries that expire after a
+//! fixed time.
+//!
+//! A `HashMapDelay` implements `Stream` which removes expired items from the map.
+
+/// The default delay for entries, in seconds. This is only used when `insert()` is used to add
+/// entries.
+const DEFAULT_DELAY: u64 = 30;
+
+use futures::prelude::*;
+use std::collections::HashMap;
+use std::time::Duration;
+use tokio_timer::delay_queue::{self, DelayQueue};
+
+pub struct HashMapDelay<K, V>
+where
+    K: std::cmp::Eq + std::hash::Hash + std::clone::Clone,
+{
+    /// The given entries.
+    entries: HashMap<K, MapEntry<V>>,
+    /// A queue holding the timeouts of each entry.
+    expirations: DelayQueue<K>,
+    /// The default expiration timeout of an entry.
+    default_entry_timeout: Duration,
+}
+
+/// A wrapping around entries that adds the link to the entry's expiration, via a `delay_queue` key.
+struct MapEntry<V> {
+    /// The expiration key for the entry.
+    key: delay_queue::Key,
+    /// The actual entry.
+    value: V,
+}
+
+impl<K, V> Default for HashMapDelay<K, V>
+where
+    K: std::cmp::Eq + std::hash::Hash + std::clone::Clone,
+{
+    fn default() -> Self {
+        HashMapDelay::new(Duration::from_secs(DEFAULT_DELAY))
+    }
+}
+
+impl<K, V> HashMapDelay<K, V>
+where
+    K: std::cmp::Eq + std::hash::Hash + std::clone::Clone,
+{
+    /// Creates a new instance of `HashMapDelay`.
+    pub fn new(default_entry_timeout: Duration) -> Self {
+        HashMapDelay {
+            entries: HashMap::new(),
+            expirations: DelayQueue::new(),
+            default_entry_timeout,
+        }
+    }
+
+    /// Insert an entry into the mapping. Entries will expire after the `default_entry_timeout`.
+    pub fn insert(&mut self, key: K, value: V) {
+        self.insert_at(key, value, self.default_entry_timeout);
+    }
+
+    /// Inserts an entry that will expire at a given instant.
+    pub fn insert_at(&mut self, key: K, value: V, entry_duration: Duration) {
+        let delay_key = self.expirations.insert(key.clone(), entry_duration);
+        let entry = MapEntry {
+            key: delay_key,
+            value,
+        };
+        self.entries.insert(key, entry);
+    }
+
+    /// Gets a reference to an entry if it exists.
+    ///
+    /// Returns None if the entry does not exist.
+    pub fn get(&self, key: &K) -> Option<&V> {
+        self.entries.get(key).map(|entry| &entry.value)
+    }
+
+    /// Gets a mutable reference to an entry if it exists.
+    ///
+    /// Returns None if the entry does not exist.
+    pub fn get_mut(&mut self, key: &K) -> Option<&mut V> {
+        self.entries.get_mut(key).map(|entry| &mut entry.value)
+    }
+
+    /// Returns true if the key exists, false otherwise.
+    pub fn contains_key(&self, key: &K) -> bool {
+        self.entries.contains_key(key)
+    }
+
+    /// Returns the length of the mapping.
+    pub fn len(&self) -> usize {
+        self.entries.len()
+    }
+
+    /// Updates the timeout for a given key. Returns true if the key existed, false otherwise.
+    ///
+    /// Panics if the duration is too far in the future.
+    pub fn update_timeout(&mut self, key: &K, timeout: Duration) -> bool {
+        if let Some(entry) = self.entries.get(key) {
+            self.expirations.reset(&entry.key, timeout);
+            true
+        } else {
+            false
+        }
+    }
+
+    /// Removes a key from the map returning the value associated with the key that was in the map.
+    ///
+    /// Return None if the key was not in the map.
+    pub fn remove(&mut self, key: &K) -> Option<V> {
+        if let Some(entry) = self.entries.remove(key) {
+            self.expirations.remove(&entry.key);
+            return Some(entry.value);
+        }
+        return None;
+    }
+
+    /// Retains only the elements specified by the predicate.
+    ///
+    /// In other words, remove all pairs `(k, v)` such that `f(&k,&mut v)` returns false.
+    pub fn retain<F: FnMut(&K, &mut V) -> bool>(&mut self, mut f: F) {
+        let expiration = &mut self.expirations;
+        self.entries.retain(|key, entry| {
+            let result = f(key, &mut entry.value);
+            if !result {
+                expiration.remove(&entry.key);
+            }
+            result
+        })
+    }
+
+    /// Removes all entries from the map.
+    pub fn clear(&mut self) {
+        self.entries.clear();
+        self.expirations.clear();
+    }
+}
+
+impl<K, V> Stream for HashMapDelay<K, V>
+where
+    K: std::cmp::Eq + std::hash::Hash + std::clone::Clone,
+{
+    type Item = (K, V);
+    type Error = &'static str;
+
+    fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
+        match self.expirations.poll() {
+            Ok(Async::Ready(Some(key))) => {
+                let key = key.into_inner();
+                match self.entries.remove(&key) {
+                    Some(entry) => Ok(Async::Ready(Some((key, entry.value)))),
+                    None => Err("Value no longer exists in expirations"),
+                }
+            }
+            Ok(Async::Ready(None)) => Ok(Async::Ready(None)),
+            Ok(Async::NotReady) => Ok(Async::NotReady),
+            Err(_) => Err("Error polling HashMapDelay"),
+        }
+    }
+}
--- a/eth2/utils/hashmap_delay/src/hashset_delay.rs
+++ b/eth2/utils/hashmap_delay/src/hashset_delay.rs
@@ -0,0 +1,157 @@
+//NOTE: This is just a specific case of a HashMapDelay.
+// The code has been copied to make unique `insert` and `insert_at` functions.
+
+/// The default delay for entries, in seconds. This is only used when `insert()` is used to add
+/// entries.
+const DEFAULT_DELAY: u64 = 30;
+
+use futures::prelude::*;
+use std::collections::HashMap;
+use std::time::{Duration, Instant};
+use tokio_timer::delay_queue::{self, DelayQueue};
+
+pub struct HashSetDelay<K>
+where
+    K: std::cmp::Eq + std::hash::Hash + std::clone::Clone,
+{
+    /// The given entries.
+    entries: HashMap<K, MapEntry>,
+    /// A queue holding the timeouts of each entry.
+    expirations: DelayQueue<K>,
+    /// The default expiration timeout of an entry.
+    default_entry_timeout: Duration,
+}
+
+/// A wrapping around entries that adds the link to the entry's expiration, via a `delay_queue` key.
+struct MapEntry {
+    /// The expiration key for the entry.
+    key: delay_queue::Key,
+    /// The actual entry.
+    value: Instant,
+}
+
+impl<K> Default for HashSetDelay<K>
+where
+    K: std::cmp::Eq + std::hash::Hash + std::clone::Clone,
+{
+    fn default() -> Self {
+        HashSetDelay::new(Duration::from_secs(DEFAULT_DELAY))
+    }
+}
+
+impl<K> HashSetDelay<K>
+where
+    K: std::cmp::Eq + std::hash::Hash + std::clone::Clone,
+{
+    /// Creates a new instance of `HashSetDelay`.
+    pub fn new(default_entry_timeout: Duration) -> Self {
+        HashSetDelay {
+            entries: HashMap::new(),
+            expirations: DelayQueue::new(),
+            default_entry_timeout,
+        }
+    }
+
+    /// Insert an entry into the mapping. Entries will expire after the `default_entry_timeout`.
+    pub fn insert(&mut self, key: K) {
+        self.insert_at(key, self.default_entry_timeout);
+    }
+
+    /// Inserts an entry that will expire at a given instant.
+    pub fn insert_at(&mut self, key: K, entry_duration: Duration) {
+        let delay_key = self.expirations.insert(key.clone(), entry_duration.clone());
+        let entry = MapEntry {
+            key: delay_key,
+            value: Instant::now() + entry_duration,
+        };
+        self.entries.insert(key, entry);
+    }
+
+    /// Gets a reference to an entry if it exists.
+    ///
+    /// Returns None if the entry does not exist.
+    pub fn get(&self, key: &K) -> Option<&Instant> {
+        self.entries.get(key).map(|entry| &entry.value)
+    }
+
+    /// Returns true if the key exists, false otherwise.
+    pub fn contains(&self, key: &K) -> bool {
+        self.entries.contains_key(key)
+    }
+
+    /// Returns the length of the mapping.
+    pub fn len(&self) -> usize {
+        self.entries.len()
+    }
+
+    /// Updates the timeout for a given key. Returns true if the key existed, false otherwise.
+    ///
+    /// Panics if the duration is too far in the future.
+    pub fn update_timeout(&mut self, key: &K, timeout: Duration) -> bool {
+        if let Some(entry) = self.entries.get(key) {
+            self.expirations.reset(&entry.key, timeout);
+            true
+        } else {
+            false
+        }
+    }
+
+    /// Removes a key from the map returning the value associated with the key that was in the map.
+    ///
+    /// Return false if the key was not in the map.
+    pub fn remove(&mut self, key: &K) -> bool {
+        if let Some(entry) = self.entries.remove(key) {
+            self.expirations.remove(&entry.key);
+            return true;
+        }
+        return false;
+    }
+
+    /// Retains only the elements specified by the predicate.
+    ///
+    /// In other words, remove all pairs `(k, v)` such that `f(&k,&mut v)` returns false.
+    pub fn retain<F: FnMut(&K) -> bool>(&mut self, mut f: F) {
+        let expiration = &mut self.expirations;
+        self.entries.retain(|key, entry| {
+            let result = f(key);
+            if !result {
+                expiration.remove(&entry.key);
+            }
+            result
+        })
+    }
+
+    /// Removes all entries from the map.
+    pub fn clear(&mut self) {
+        self.entries.clear();
+        self.expirations.clear();
+    }
+
+    /// Returns a vector of referencing all keys in the map.
+    pub fn keys_vec(&self) -> Vec<&K> {
+        self.entries.keys().collect()
+    }
+}
+
+impl<K> Stream for HashSetDelay<K>
+where
+    K: std::cmp::Eq + std::hash::Hash + std::clone::Clone,
+{
+    type Item = K;
+    type Error = &'static str;
+
+    fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
+        match self.expirations.poll() {
+            Ok(Async::Ready(Some(key))) => {
+                let key = key.into_inner();
+                match self.entries.remove(&key) {
+                    Some(_) => Ok(Async::Ready(Some(key))),
+                    None => Err("Value no longer exists in expirations"),
+                }
+            }
+            Ok(Async::Ready(None)) => Ok(Async::Ready(None)),
+            Ok(Async::NotReady) => Ok(Async::NotReady),
+            Err(_) => Err("Error polling HashSetDelay"),
+        }
+    }
+}
--- a/eth2/utils/hashmap_delay/src/lib.rs
+++ b/eth2/utils/hashmap_delay/src/lib.rs
@@ -0,0 +1,21 @@
+//! This crate provides two objects:
+//! - `HashMapDelay`
+//! - `HashSetDelay`
+//!
+//! # HashMapDelay
+//!
+//! This provides a `HashMap` coupled with a `DelayQueue`. Objects that are inserted into
+//! the map are inserted with an expiry. `Stream` is implemented on the `HashMapDelay`
+//! which return objects that have expired. These objects are removed from the mapping.
+//!
+//! # HashSetDelay
+//!
+//! This is similar to a `HashMapDelay` except the mapping maps to the expiry time. This
+//! allows users to add objects and check their expiry deadlines before the `Stream`
+//! consumes them.
+
+mod hashmap_delay;
+mod hashset_delay;
+
+pub use crate::hashmap_delay::HashMapDelay;
+pub use crate::hashset_delay::HashSetDelay;
--- a/eth2/utils/int_to_bytes/Cargo.toml
+++ b/eth2/utils/int_to_bytes/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "int_to_bytes"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/lighthouse_bootstrap/Cargo.toml
+++ b/eth2/utils/lighthouse_bootstrap/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "lighthouse_bootstrap"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/lighthouse_metrics/Cargo.toml
+++ b/eth2/utils/lighthouse_metrics/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "lighthouse_metrics"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/logging/Cargo.toml
+++ b/eth2/utils/logging/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "logging"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["blacktemplar <blacktemplar@a1.net>"]
 edition = "2018"

--- a/eth2/utils/merkle_proof/Cargo.toml
+++ b/eth2/utils/merkle_proof/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "merkle_proof"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Michael Sproul <michael@sigmaprime.io>"]
 edition = "2018"

--- a/eth2/utils/remote_beacon_node/Cargo.toml
+++ b/eth2/utils/remote_beacon_node/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "remote_beacon_node"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

@@ -12,7 +12,7 @@ url = "1.2"
 serde = "1.0"
 futures = "0.1.25"
 types = { path = "../../../eth2/types" }
-rest_api = { path = "../../../beacon_node/rest_api" }
+rest_types = { path = "../rest_types" }
 hex = "0.3"
 eth2_ssz = { path = "../../../eth2/utils/ssz" }
 serde_json = "^1.0"
--- a/eth2/utils/remote_beacon_node/src/lib.rs
+++ b/eth2/utils/remote_beacon_node/src/lib.rs
@@ -15,15 +15,16 @@ use std::marker::PhantomData;
 use std::time::Duration;
 use types::{
    Attestation, AttesterSlashing, BeaconBlock, BeaconState, CommitteeIndex, Epoch, EthSpec, Fork,
-    Hash256, ProposerSlashing, PublicKey, Signature, SignedBeaconBlock, Slot,
+    Hash256, ProposerSlashing, PublicKey, Signature, SignedAggregateAndProof, SignedBeaconBlock,
+    Slot,
 };
 use url::Url;

 pub use operation_pool::PersistedOperationPool;
 pub use proto_array_fork_choice::core::ProtoArray;
-pub use rest_api::{
-    CanonicalHeadResponse, Committee, HeadBeaconBlock, ValidatorDutiesRequest, ValidatorDuty,
-    ValidatorRequest, ValidatorResponse,
+pub use rest_types::{
+    CanonicalHeadResponse, Committee, HeadBeaconBlock, ValidatorDutiesRequest, ValidatorDutyBytes,
+    ValidatorRequest, ValidatorResponse, ValidatorSubscription,
 };

 // Setting a long timeout for debug ensures that crypto-heavy operations can still succeed.
@@ -63,6 +64,8 @@ pub enum Error {
    SerdeJsonError(serde_json::Error),
    /// The server responded to the request, however it did not return a 200-type success code.
    DidNotSucceed { status: StatusCode, body: String },
+    /// The request input was invalid.
+    InvalidInput,
 }

 #[derive(Clone)]
@@ -166,7 +169,7 @@ pub enum PublishStatus {
    Valid,
    /// The object was not valid and may or may not have been published to the network.
    Invalid(String),
-    /// The server responsed with an unknown status code. The object may or may not have been
+    /// The server responded with an unknown status code. The object may or may not have been
    /// published to the network.
    Unknown,
 }
@@ -178,7 +181,7 @@ impl PublishStatus {
    }
 }

-/// Provides the functions on the `/beacon` endpoint of the node.
+/// Provides the functions on the `/validator` endpoint of the node.
 #[derive(Clone)]
 pub struct Validator<E>(HttpClient<E>);

@@ -207,13 +210,30 @@ impl<E: EthSpec> Validator<E> {
            .and_then(move |url| client.json_get(url, query_params))
    }

-    /// Posts an attestation to the beacon node, expecting it to verify it and publish it to the network.
-    pub fn publish_attestation(
+    /// Produces an aggregate attestation.
+    pub fn produce_aggregate_attestation(
        &self,
-        attestation: Attestation<E>,
+        slot: Slot,
+        committee_index: CommitteeIndex,
+    ) -> impl Future<Item = Attestation<E>, Error = Error> {
+        let query_params = vec![
+            ("slot".into(), format!("{}", slot)),
+            ("committee_index".into(), format!("{}", committee_index)),
+        ];
+
+        let client = self.0.clone();
+        self.url("aggregate_attestation")
+            .into_future()
+            .and_then(move |url| client.json_get(url, query_params))
+    }
+
+    /// Posts a list of attestations to the beacon node, expecting it to verify it and publish it to the network.
+    pub fn publish_attestations(
+        &self,
+        attestation: Vec<Attestation<E>>,
    ) -> impl Future<Item = PublishStatus, Error = Error> {
        let client = self.0.clone();
-        self.url("attestation")
+        self.url("attestations")
            .into_future()
            .and_then(move |url| client.json_post::<_>(url, attestation))
            .and_then(|mut response| {
@@ -232,12 +252,37 @@ impl<E: EthSpec> Validator<E> {
            })
    }

+    /// Posts a list of signed aggregates and proofs to the beacon node, expecting it to verify it and publish it to the network.
+    pub fn publish_aggregate_and_proof(
+        &self,
+        signed_aggregate_and_proofs: Vec<SignedAggregateAndProof<E>>,
+    ) -> impl Future<Item = PublishStatus, Error = Error> {
+        let client = self.0.clone();
+        self.url("aggregate_and_proofs")
+            .into_future()
+            .and_then(move |url| client.json_post::<_>(url, signed_aggregate_and_proofs))
+            .and_then(|mut response| {
+                response
+                    .text()
+                    .map(|text| (response, text))
+                    .map_err(Error::from)
+            })
+            .and_then(|(response, text)| match response.status() {
+                StatusCode::OK => Ok(PublishStatus::Valid),
+                StatusCode::ACCEPTED => Ok(PublishStatus::Invalid(text)),
+                _ => response
+                    .error_for_status()
+                    .map_err(Error::from)
+                    .map(|_| PublishStatus::Unknown),
+            })
+    }
+
    /// Returns the duties required of the given validator pubkeys in the given epoch.
    pub fn get_duties(
        &self,
        epoch: Epoch,
        validator_pubkeys: &[PublicKey],
-    ) -> impl Future<Item = Vec<ValidatorDuty>, Error = Error> {
+    ) -> impl Future<Item = Vec<ValidatorDutyBytes>, Error = Error> {
        let client = self.0.clone();

        let bulk_request = ValidatorDutiesRequest {
@@ -297,6 +342,31 @@ impl<E: EthSpec> Validator<E> {
            )
        })
    }
+
+    /// Subscribes a list of validators to particular slots for attestation production/publication.
+    pub fn subscribe(
+        &self,
+        subscriptions: Vec<ValidatorSubscription>,
+    ) -> impl Future<Item = PublishStatus, Error = Error> {
+        let client = self.0.clone();
+        self.url("subscribe")
+            .into_future()
+            .and_then(move |url| client.json_post::<_>(url, subscriptions))
+            .and_then(|mut response| {
+                response
+                    .text()
+                    .map(|text| (response, text))
+                    .map_err(Error::from)
+            })
+            .and_then(|(response, text)| match response.status() {
+                StatusCode::OK => Ok(PublishStatus::Valid),
+                StatusCode::ACCEPTED => Ok(PublishStatus::Invalid(text)),
+                _ => response
+                    .error_for_status()
+                    .map_err(Error::from)
+                    .map(|_| PublishStatus::Unknown),
+            })
+    }
 }

 /// Provides the functions on the `/beacon` endpoint of the node.
--- a/eth2/utils/rest_types/Cargo.toml
+++ b/eth2/utils/rest_types/Cargo.toml
@@ -0,0 +1,15 @@
+[package]
+name = "rest_types"
+version = "0.2.0"
+authors = ["Age Manning <Age@AgeManning.com>"]
+edition = "2018"
+
+[dependencies]
+types = { path = "../../types" }
+eth2_ssz_derive = { path = "../ssz_derive" }
+eth2_ssz = { path = "../ssz" }
+eth2_hashing = { path = "../eth2_hashing" }
+tree_hash = { path = "../tree_hash" }
+bls = { path = "../bls" }
+serde = { version = "1.0.102", features = ["derive"] }
+rayon = "1.3.0"
--- a/eth2/utils/rest_types/src/beacon.rs
+++ b/eth2/utils/rest_types/src/beacon.rs
@@ -0,0 +1,65 @@
+//! A collection of REST API types for interaction with the beacon node.
+
+use bls::PublicKeyBytes;
+use serde::{Deserialize, Serialize};
+use ssz_derive::{Decode, Encode};
+use types::beacon_state::EthSpec;
+use types::{BeaconState, CommitteeIndex, Hash256, SignedBeaconBlock, Slot, Validator};
+
+/// Information about a block that is at the head of a chain. May or may not represent the
+/// canonical head.
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
+pub struct HeadBeaconBlock {
+    pub beacon_block_root: Hash256,
+    pub beacon_block_slot: Slot,
+}
+
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
+#[serde(bound = "T: EthSpec")]
+pub struct BlockResponse<T: EthSpec> {
+    pub root: Hash256,
+    pub beacon_block: SignedBeaconBlock<T>,
+}
+
+/// Information about the block and state that are at head of the beacon chain.
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
+pub struct CanonicalHeadResponse {
+    pub slot: Slot,
+    pub block_root: Hash256,
+    pub state_root: Hash256,
+    pub finalized_slot: Slot,
+    pub finalized_block_root: Hash256,
+    pub justified_slot: Slot,
+    pub justified_block_root: Hash256,
+    pub previous_justified_slot: Slot,
+    pub previous_justified_block_root: Hash256,
+}
+
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
+pub struct ValidatorResponse {
+    pub pubkey: PublicKeyBytes,
+    pub validator_index: Option<usize>,
+    pub balance: Option<u64>,
+    pub validator: Option<Validator>,
+}
+
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
+pub struct ValidatorRequest {
+    /// If set to `None`, uses the canonical head state.
+    pub state_root: Option<Hash256>,
+    pub pubkeys: Vec<PublicKeyBytes>,
+}
+
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
+pub struct Committee {
+    pub slot: Slot,
+    pub index: CommitteeIndex,
+    pub committee: Vec<usize>,
+}
+
+#[derive(Clone, Debug, PartialEq, Serialize, Deserialize, Encode, Decode)]
+#[serde(bound = "T: EthSpec")]
+pub struct StateResponse<T: EthSpec> {
+    pub root: Hash256,
+    pub beacon_state: BeaconState<T>,
+}
--- a/eth2/utils/rest_types/src/lib.rs
+++ b/eth2/utils/rest_types/src/lib.rs
@@ -0,0 +1,15 @@
+//! A collection of types used to pass data across the rest HTTP API.
+//!
+//! This is primarily used by the validator client and the beacon node rest API.
+
+mod beacon;
+mod validator;
+
+pub use beacon::{
+    BlockResponse, CanonicalHeadResponse, Committee, HeadBeaconBlock, StateResponse,
+    ValidatorRequest, ValidatorResponse,
+};
+
+pub use validator::{
+    ValidatorDutiesRequest, ValidatorDuty, ValidatorDutyBytes, ValidatorSubscription,
+};
--- a/eth2/utils/rest_types/src/validator.rs
+++ b/eth2/utils/rest_types/src/validator.rs
@@ -0,0 +1,108 @@
+use bls::{PublicKey, PublicKeyBytes, Signature};
+use eth2_hashing::hash;
+use serde::{Deserialize, Serialize};
+use ssz_derive::{Decode, Encode};
+use std::convert::TryInto;
+use types::{ChainSpec, CommitteeIndex, Domain, Epoch, Fork, SignedRoot, Slot};
+
+/// A Validator duty with the validator public key represented a `PublicKeyBytes`.
+pub type ValidatorDutyBytes = ValidatorDutyBase<PublicKeyBytes>;
+/// A validator duty with the pubkey represented as a `PublicKey`.
+pub type ValidatorDuty = ValidatorDutyBase<PublicKey>;
+
+#[derive(PartialEq, Debug, Serialize, Deserialize, Clone)]
+pub struct ValidatorDutyBase<T> {
+    /// The validator's BLS public key, uniquely identifying them. _48-bytes, hex encoded with 0x prefix, case insensitive._
+    pub validator_pubkey: T,
+    /// The validator's index in `state.validators`
+    pub validator_index: Option<u64>,
+    /// The slot at which the validator must attest.
+    pub attestation_slot: Option<Slot>,
+    /// The index of the committee within `slot` of which the validator is a member.
+    pub attestation_committee_index: Option<CommitteeIndex>,
+    /// The position of the validator in the committee.
+    pub attestation_committee_position: Option<usize>,
+    /// The slots in which a validator must propose a block (can be empty).
+    pub block_proposal_slots: Vec<Slot>,
+    /// This provides the modulo: `max(1, len(committee) // TARGET_AGGREGATORS_PER_COMMITTEE)`
+    /// which allows the validator client to determine if this duty requires the validator to be
+    /// aggregate attestations.
+    pub aggregator_modulo: Option<u64>,
+}
+
+impl<T> ValidatorDutyBase<T> {
+    /// Given a `slot_signature` determines if the validator of this duty is an aggregator.
+    // Note that we assume the signature is for the associated pubkey to avoid the signature
+    // verification
+    pub fn is_aggregator(&self, slot_signature: &Signature) -> bool {
+        if let Some(modulo) = self.aggregator_modulo {
+            let signature_hash = hash(&slot_signature.as_bytes());
+            let signature_hash_int = u64::from_le_bytes(
+                signature_hash[0..8]
+                    .try_into()
+                    .expect("first 8 bytes of signature should always convert to fixed array"),
+            );
+            signature_hash_int % modulo == 0
+        } else {
+            false
+        }
+    }
+}
+
+#[derive(PartialEq, Debug, Serialize, Deserialize, Clone, Encode, Decode)]
+pub struct ValidatorDutiesRequest {
+    pub epoch: Epoch,
+    pub pubkeys: Vec<PublicKeyBytes>,
+}
+
+/// A validator subscription, created when a validator subscribes to a slot to perform optional aggregation
+/// duties.
+#[derive(PartialEq, Debug, Serialize, Deserialize, Clone, Encode, Decode)]
+pub struct ValidatorSubscription {
+    /// The validators index.
+    pub validator_index: u64,
+    /// The index of the committee within `slot` of which the validator is a member. Used by the
+    /// beacon node to quickly evaluate the associated `SubnetId`.
+    pub attestation_committee_index: CommitteeIndex,
+    /// The slot the validator is signing.
+    pub slot: Slot,
+    /// The signature of the slot by the validator.
+    pub slot_signature: Signature,
+}
+
+impl ValidatorSubscription {
+    pub fn new(
+        validator_index: u64,
+        attestation_committee_index: CommitteeIndex,
+        slot: Slot,
+        slot_signature: Signature,
+    ) -> Self {
+        ValidatorSubscription {
+            validator_index,
+            attestation_committee_index,
+            slot,
+            slot_signature,
+        }
+    }
+
+    /// Verifies the subscription signature.
+    pub fn verify(
+        &self,
+        pubkey: &PublicKey,
+        spec: &ChainSpec,
+        fork: &Fork,
+        slots_per_epoch: u64,
+    ) -> bool {
+        let domain = spec.get_domain(
+            self.slot.epoch(slots_per_epoch),
+            Domain::SelectionProof,
+            &fork,
+        );
+        let message = self.slot.signing_root(domain);
+        if self.slot_signature.verify(message.as_bytes(), pubkey) {
+            true
+        } else {
+            false
+        }
+    }
+}
--- a/eth2/utils/serde_hex/Cargo.toml
+++ b/eth2/utils/serde_hex/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "serde_hex"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/slot_clock/Cargo.toml
+++ b/eth2/utils/slot_clock/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "slot_clock"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

@@ -8,3 +8,4 @@ edition = "2018"
 types = { path = "../../types" }
 lazy_static = "1.4.0"
 lighthouse_metrics = { path = "../lighthouse_metrics" }
+parking_lot = "0.9.0"
--- a/eth2/utils/slot_clock/src/lib.rs
+++ b/eth2/utils/slot_clock/src/lib.rs
@@ -1,14 +1,15 @@
 #[macro_use]
 extern crate lazy_static;

+mod manual_slot_clock;
 mod metrics;
 mod system_time_slot_clock;
-mod testing_slot_clock;

 use std::time::Duration;

+pub use crate::manual_slot_clock::ManualSlotClock;
+pub use crate::manual_slot_clock::ManualSlotClock as TestingSlotClock;
 pub use crate::system_time_slot_clock::SystemTimeSlotClock;
-pub use crate::testing_slot_clock::TestingSlotClock;
 pub use metrics::scrape_for_metrics;
 pub use types::Slot;

@@ -16,13 +17,21 @@ pub use types::Slot;
 ///
 /// The clock is not required to be monotonically increasing and may go backwards.
 pub trait SlotClock: Send + Sync + Sized {
-    /// Creates a new slot clock where the first slot is `genesis_slot`, genesis occured
+    /// Creates a new slot clock where the first slot is `genesis_slot`, genesis occurred
    /// `genesis_duration` after the `UNIX_EPOCH` and each slot is `slot_duration` apart.
    fn new(genesis_slot: Slot, genesis_duration: Duration, slot_duration: Duration) -> Self;

    /// Returns the slot at this present time.
    fn now(&self) -> Option<Slot>;

+    /// Returns the present time as a duration since the UNIX epoch.
+    ///
+    /// Returns `None` if the present time is before the UNIX epoch (unlikely).
+    fn now_duration(&self) -> Option<Duration>;
+
+    /// Returns the slot of the given duration since the UNIX epoch.
+    fn slot_of(&self, now: Duration) -> Option<Slot>;
+
    /// Returns the duration between slots
    fn slot_duration(&self) -> Duration;

--- a/eth2/utils/slot_clock/src/manual_slot_clock.rs
+++ b/eth2/utils/slot_clock/src/manual_slot_clock.rs
@@ -0,0 +1,165 @@
+use super::SlotClock;
+use parking_lot::RwLock;
+use std::convert::TryInto;
+use std::time::Duration;
+use types::Slot;
+
+/// Determines the present slot based upon a manually-incremented UNIX timestamp.
+pub struct ManualSlotClock {
+    genesis_slot: Slot,
+    /// Duration from UNIX epoch to genesis.
+    genesis_duration: Duration,
+    /// Duration from UNIX epoch to right now.
+    current_time: RwLock<Duration>,
+    /// The length of each slot.
+    slot_duration: Duration,
+}
+
+impl Clone for ManualSlotClock {
+    fn clone(&self) -> Self {
+        ManualSlotClock {
+            genesis_slot: self.genesis_slot.clone(),
+            genesis_duration: self.genesis_duration.clone(),
+            current_time: RwLock::new(self.current_time.read().clone()),
+            slot_duration: self.slot_duration.clone(),
+        }
+    }
+}
+
+impl ManualSlotClock {
+    pub fn set_slot(&self, slot: u64) {
+        let slots_since_genesis = slot
+            .checked_sub(self.genesis_slot.as_u64())
+            .expect("slot must be post-genesis")
+            .try_into()
+            .expect("slot must fit within a u32");
+        *self.current_time.write() =
+            self.genesis_duration + self.slot_duration * slots_since_genesis;
+    }
+
+    pub fn advance_slot(&self) {
+        self.set_slot(self.now().unwrap().as_u64() + 1)
+    }
+
+    pub fn duration_to_next_slot_from(&self, now: Duration) -> Option<Duration> {
+        let genesis = self.genesis_duration;
+
+        let slot_start = |slot: Slot| -> Duration {
+            let slot = slot.as_u64() as u32;
+            genesis + slot * self.slot_duration
+        };
+
+        if now >= genesis {
+            Some(
+                slot_start(self.slot_of(now)? + 1)
+                    .checked_sub(now)
+                    .expect("The next slot cannot start before now"),
+            )
+        } else {
+            Some(
+                genesis
+                    .checked_sub(now)
+                    .expect("Control flow ensures genesis is greater than or equal to now"),
+            )
+        }
+    }
+
+    pub fn duration_to_next_epoch_from(
+        &self,
+        now: Duration,
+        slots_per_epoch: u64,
+    ) -> Option<Duration> {
+        let genesis = self.genesis_duration;
+
+        let slot_start = |slot: Slot| -> Duration {
+            let slot = slot.as_u64() as u32;
+            genesis + slot * self.slot_duration
+        };
+
+        let epoch_start_slot = self
+            .now()
+            .map(|slot| slot.epoch(slots_per_epoch))
+            .map(|epoch| (epoch + 1).start_slot(slots_per_epoch))?;
+
+        if now >= genesis {
+            Some(
+                slot_start(epoch_start_slot)
+                    .checked_sub(now)
+                    .expect("The next epoch cannot start before now"),
+            )
+        } else {
+            Some(
+                genesis
+                    .checked_sub(now)
+                    .expect("Control flow ensures genesis is greater than or equal to now"),
+            )
+        }
+    }
+}
+
+impl SlotClock for ManualSlotClock {
+    fn new(genesis_slot: Slot, genesis_duration: Duration, slot_duration: Duration) -> Self {
+        if slot_duration.as_millis() == 0 {
+            panic!("ManualSlotClock cannot have a < 1ms slot duration");
+        }
+
+        Self {
+            genesis_slot,
+            current_time: RwLock::new(genesis_duration.clone()),
+            genesis_duration,
+            slot_duration,
+        }
+    }
+
+    fn now(&self) -> Option<Slot> {
+        self.slot_of(*self.current_time.read())
+    }
+
+    fn now_duration(&self) -> Option<Duration> {
+        Some(*self.current_time.read())
+    }
+
+    fn slot_of(&self, now: Duration) -> Option<Slot> {
+        let genesis = self.genesis_duration;
+
+        if now >= genesis {
+            let since_genesis = now
+                .checked_sub(genesis)
+                .expect("Control flow ensures now is greater than or equal to genesis");
+            let slot =
+                Slot::from((since_genesis.as_millis() / self.slot_duration.as_millis()) as u64);
+            Some(slot + self.genesis_slot)
+        } else {
+            None
+        }
+    }
+
+    fn duration_to_next_slot(&self) -> Option<Duration> {
+        self.duration_to_next_slot_from(*self.current_time.read())
+    }
+
+    fn duration_to_next_epoch(&self, slots_per_epoch: u64) -> Option<Duration> {
+        self.duration_to_next_epoch_from(*self.current_time.read(), slots_per_epoch)
+    }
+
+    fn slot_duration(&self) -> Duration {
+        self.slot_duration
+    }
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_slot_now() {
+        let clock = ManualSlotClock::new(
+            Slot::new(10),
+            Duration::from_secs(0),
+            Duration::from_secs(1),
+        );
+        assert_eq!(clock.now(), Some(Slot::new(10)));
+        clock.set_slot(123);
+        assert_eq!(clock.now(), Some(Slot::new(123)));
+    }
+}
--- a/eth2/utils/slot_clock/src/system_time_slot_clock.rs
+++ b/eth2/utils/slot_clock/src/system_time_slot_clock.rs
@@ -1,4 +1,4 @@
-use super::SlotClock;
+use super::{ManualSlotClock, SlotClock};
 use std::time::{Duration, SystemTime, UNIX_EPOCH};
 use types::Slot;

@@ -7,95 +7,41 @@ pub use std::time::SystemTimeError;
 /// Determines the present slot based upon the present system time.
 #[derive(Clone)]
 pub struct SystemTimeSlotClock {
-    genesis_slot: Slot,
-    genesis_duration: Duration,
-    slot_duration: Duration,
+    clock: ManualSlotClock,
 }

 impl SlotClock for SystemTimeSlotClock {
    fn new(genesis_slot: Slot, genesis_duration: Duration, slot_duration: Duration) -> Self {
-        if slot_duration.as_millis() == 0 {
-            panic!("SystemTimeSlotClock cannot have a < 1ms slot duration.");
-        }
-
        Self {
-            genesis_slot,
-            genesis_duration,
-            slot_duration,
+            clock: ManualSlotClock::new(genesis_slot, genesis_duration, slot_duration),
        }
    }

    fn now(&self) -> Option<Slot> {
        let now = SystemTime::now().duration_since(UNIX_EPOCH).ok()?;
-        let genesis = self.genesis_duration;
+        self.clock.slot_of(now)
+    }

-        if now >= genesis {
-            let since_genesis = now
-                .checked_sub(genesis)
-                .expect("Control flow ensures now is greater than or equal to genesis");
-            let slot =
-                Slot::from((since_genesis.as_millis() / self.slot_duration.as_millis()) as u64);
-            Some(slot + self.genesis_slot)
-        } else {
-            None
-        }
+    fn now_duration(&self) -> Option<Duration> {
+        SystemTime::now().duration_since(UNIX_EPOCH).ok()
+    }
+
+    fn slot_of(&self, now: Duration) -> Option<Slot> {
+        self.clock.slot_of(now)
    }

    fn duration_to_next_slot(&self) -> Option<Duration> {
        let now = SystemTime::now().duration_since(UNIX_EPOCH).ok()?;
-        let genesis = self.genesis_duration;
-
-        let slot_start = |slot: Slot| -> Duration {
-            let slot = slot.as_u64() as u32;
-            genesis + slot * self.slot_duration
-        };
-
-        if now >= genesis {
-            Some(
-                slot_start(self.now()? + 1)
-                    .checked_sub(now)
-                    .expect("The next slot cannot start before now"),
-            )
-        } else {
-            Some(
-                genesis
-                    .checked_sub(now)
-                    .expect("Control flow ensures genesis is greater than or equal to now"),
-            )
-        }
+        self.clock.duration_to_next_slot_from(now)
    }

    fn duration_to_next_epoch(&self, slots_per_epoch: u64) -> Option<Duration> {
        let now = SystemTime::now().duration_since(UNIX_EPOCH).ok()?;
-        let genesis = self.genesis_duration;
-
-        let slot_start = |slot: Slot| -> Duration {
-            let slot = slot.as_u64() as u32;
-            genesis + slot * self.slot_duration
-        };
-
-        let epoch_start_slot = self
-            .now()
-            .map(|slot| slot.epoch(slots_per_epoch))
-            .map(|epoch| (epoch + 1).start_slot(slots_per_epoch))?;
-
-        if now >= genesis {
-            Some(
-                slot_start(epoch_start_slot)
-                    .checked_sub(now)
-                    .expect("The next epoch cannot start before now"),
-            )
-        } else {
-            Some(
-                genesis
-                    .checked_sub(now)
-                    .expect("Control flow ensures genesis is greater than or equal to now"),
-            )
-        }
+        self.clock.duration_to_next_epoch_from(now, slots_per_epoch)
    }

    fn slot_duration(&self) -> Duration {
-        self.slot_duration
+        self.clock.slot_duration()
    }
 }

--- a/eth2/utils/slot_clock/src/testing_slot_clock.rs
+++ b/eth2/utils/slot_clock/src/testing_slot_clock.rs
@@ -1,64 +0,0 @@
-use super::SlotClock;
-use std::sync::RwLock;
-use std::time::Duration;
-use types::Slot;
-
-/// A slot clock where the slot is manually set instead of being determined by the system time.
-///
-/// Useful for testing scenarios.
-pub struct TestingSlotClock {
-    slot: RwLock<Slot>,
-}
-
-impl TestingSlotClock {
-    pub fn set_slot(&self, slot: u64) {
-        *self.slot.write().expect("TestingSlotClock poisoned.") = Slot::from(slot);
-    }
-
-    pub fn advance_slot(&self) {
-        self.set_slot(self.now().unwrap().as_u64() + 1)
-    }
-}
-
-impl SlotClock for TestingSlotClock {
-    fn new(genesis_slot: Slot, _genesis_duration: Duration, _slot_duration: Duration) -> Self {
-        TestingSlotClock {
-            slot: RwLock::new(genesis_slot),
-        }
-    }
-
-    fn now(&self) -> Option<Slot> {
-        let slot = *self.slot.read().expect("TestingSlotClock poisoned.");
-        Some(slot)
-    }
-
-    /// Always returns a duration of 1 second.
-    fn duration_to_next_slot(&self) -> Option<Duration> {
-        Some(Duration::from_secs(1))
-    }
-
-    /// Always returns a duration of `1 * slots_per_epoch` second.
-    fn duration_to_next_epoch(&self, slots_per_epoch: u64) -> Option<Duration> {
-        Some(Duration::from_secs(slots_per_epoch))
-    }
-
-    /// Always returns a slot duration of 0 seconds.
-    fn slot_duration(&self) -> Duration {
-        Duration::from_secs(0)
-    }
-}
-
-#[cfg(test)]
-mod tests {
-    use super::*;
-
-    #[test]
-    fn test_slot_now() {
-        let null = Duration::from_secs(0);
-
-        let clock = TestingSlotClock::new(Slot::new(10), null, null);
-        assert_eq!(clock.now(), Some(Slot::new(10)));
-        clock.set_slot(123);
-        assert_eq!(clock.now(), Some(Slot::new(123)));
-    }
-}
--- a/eth2/utils/ssz_types/src/bitfield.rs
+++ b/eth2/utils/ssz_types/src/bitfield.rs
@@ -302,7 +302,7 @@ impl<T: BitfieldBehaviour> Bitfield<T> {

    /// Returns the value of the `i`'th bit.
    ///
-    /// Returns `None` if `i` is out-of-bounds of `self`.
+    /// Returns `Error` if `i` is out-of-bounds of `self`.
    pub fn get(&self, i: usize) -> Result<bool, Error> {
        if i < self.len {
            let byte = self
--- a/eth2/utils/swap_or_not_shuffle/Cargo.toml
+++ b/eth2/utils/swap_or_not_shuffle/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "swap_or_not_shuffle"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["Paul Hauner <paul@paulhauner.com>"]
 edition = "2018"

--- a/eth2/utils/test_random_derive/Cargo.toml
+++ b/eth2/utils/test_random_derive/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "test_random_derive"
-version = "0.1.0"
+version = "0.2.0"
 authors = ["thojest <thojest@gmail.com>"]
 edition = "2018"
 description = "Procedural derive macros for implementation of TestRandom trait"