gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-10 12:11:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update to frozen spec ❄️ (v0.8.1) (#444)

Browse Source 
* types: first updates for v0.8

* state_processing: epoch processing v0.8.0

* state_processing: block processing v0.8.0

* tree_hash_derive: support generics in SignedRoot

* types v0.8: update to use ssz_types

* state_processing v0.8: use ssz_types

* ssz_types: add bitwise methods and from_elem

* types: fix v0.8 FIXMEs

* ssz_types: add bitfield shift_up

* ssz_types: iterators and DerefMut for VariableList

* types,state_processing: use VariableList

* ssz_types: fix BitVector Decode impl

Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it
to be considered variable length!

* types: fix test modules for v0.8 update

* types: remove slow type-level arithmetic

* state_processing: fix tests for v0.8

* op_pool: update for v0.8

* ssz_types: Bitfield difference length-independent

Allow computing the difference of two bitfields of different lengths.

* Implement compact committee support

* epoch_processing: committee & active index roots

* state_processing: genesis state builder v0.8

* state_processing: implement v0.8.1

* Further improve tree_hash

* Strip examples, tests from cached_tree_hash

* Update TreeHash, un-impl CachedTreeHash

* Update bitfield TreeHash, un-impl CachedTreeHash

* Update FixedLenVec TreeHash, unimpl CachedTreeHash

* Update update tree_hash_derive for new TreeHash

* Fix TreeHash, un-impl CachedTreeHash for ssz_types

* Remove fixed_len_vec, ssz benches

SSZ benches relied upon fixed_len_vec -- it is easier to just delete
them and rebuild them later (when necessary)

* Remove boolean_bitfield crate

* Fix fake_crypto BLS compile errors

* Update ef_tests for new v.8 type params

* Update ef_tests submodule to v0.8.1 tag

* Make fixes to support parsing ssz ef_tests

* `compact_committee...` to `compact_committees...`

* Derive more traits for `CompactCommittee`

* Flip bitfield byte-endianness

* Fix tree_hash for bitfields

* Modify CLI output for ef_tests

* Bump ssz crate version

* Update ssz_types doc comment

* Del cached tree hash tests from ssz_static tests

* Tidy SSZ dependencies

* Rename ssz_types crate to eth2_ssz_types

* validator_client: update for v0.8

* ssz_types: update union/difference for bit order swap

* beacon_node: update for v0.8, EthSpec

* types: disable cached tree hash, update min spec

* state_processing: fix slot bug in committee update

* tests: temporarily disable fork choice harness test

See #447

* committee cache: prevent out-of-bounds access

In the case where we tried to access the committee of a shard that didn't have a committee in the
current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This
commit adds a check to make the failure safe and explicit.

* fix bug in get_indexed_attestation and simplify

There was a bug in our implementation of get_indexed_attestation whereby
incorrect "committee indices" were used to index into the custody bitfield. The
bug was only observable in the case where some bits of the custody bitfield were
set to 1. The implementation has been simplified to remove the bug, and a test
added.

* state_proc: workaround for compact committees bug

https://github.com/ethereum/eth2.0-specs/issues/1315

* v0.8: updates to make the EF tests pass

* Remove redundant max operation checks.
* Always supply both messages when checking attestation signatures -- allowing
  verification of an attestation with no signatures.
* Swap the order of the fork and domain constant in `get_domain`, to match
  the spec.

* rustfmt

* ef_tests: add new epoch processing tests

* Integrate v0.8 into master (compiles)

* Remove unused crates, fix clippy lints

* Replace v0.6.3 tags w/ v0.8.1

* Remove old comment

* Ensure lmd ghost tests only run in release

* Update readme
This commit is contained in:
Michael Sproul
2019-07-30 12:44:51 +10:00
committed by Paul Hauner
parent 177df12149
commit a236003a7b
184 changed files with 3332 additions and 4542 deletions
Download Patch File Download Diff File Expand all files Collapse all files

149
eth2/state_processing/src/per_block_processing.rs
View File

@@ -1,6 +1,8 @@
use crate::common::{initiate_validator_exit, slash_validator};
use errors::{BlockInvalid as Invalid, BlockProcessingError as Error, IntoWithIndex};
use rayon::prelude::*;
use std::collections::HashSet;
use std::iter::FromIterator;
use tree_hash::{SignedRoot, TreeHash};
use types::*;
@@ -8,30 +10,29 @@ pub use self::verify_attester_slashing::{
get_slashable_indices, get_slashable_indices_modular, verify_attester_slashing,
};
pub use self::verify_proposer_slashing::verify_proposer_slashing;
pub use validate_attestation::{
validate_attestation, validate_attestation_time_independent_only,
validate_attestation_without_signature,
pub use is_valid_indexed_attestation::{
is_valid_indexed_attestation, is_valid_indexed_attestation_without_signature,
};
pub use verify_attestation::{
verify_attestation, verify_attestation_time_independent_only,
verify_attestation_without_signature,
};
pub use verify_deposit::{
get_existing_validator_index, verify_deposit_index, verify_deposit_merkle_proof,
verify_deposit_signature,
get_existing_validator_index, verify_deposit_merkle_proof, verify_deposit_signature,
};
pub use verify_exit::{verify_exit, verify_exit_time_independent_only};
pub use verify_indexed_attestation::{
verify_indexed_attestation, verify_indexed_attestation_without_signature,
};
pub use verify_transfer::{
execute_transfer, verify_transfer, verify_transfer_time_independent_only,
};
pub mod block_processing_builder;
pub mod errors;
mod is_valid_indexed_attestation;
pub mod tests;
mod validate_attestation;
mod verify_attestation;
mod verify_attester_slashing;
mod verify_deposit;
mod verify_exit;
mod verify_indexed_attestation;
mod verify_proposer_slashing;
mod verify_transfer;
@@ -40,10 +41,10 @@ mod verify_transfer;
/// Returns `Ok(())` if the block is valid and the state was successfully updated. Otherwise
/// returns an error describing why the block was invalid or how the function failed to execute.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn per_block_processing<T: EthSpec>(
state: &mut BeaconState<T>,
block: &BeaconBlock,
block: &BeaconBlock<T>,
spec: &ChainSpec,
) -> Result<(), Error> {
per_block_processing_signature_optional(state, block, true, spec)
@@ -55,10 +56,10 @@ pub fn per_block_processing<T: EthSpec>(
/// Returns `Ok(())` if the block is valid and the state was successfully updated. Otherwise
/// returns an error describing why the block was invalid or how the function failed to execute.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn per_block_processing_without_verifying_block_signature<T: EthSpec>(
state: &mut BeaconState<T>,
block: &BeaconBlock,
block: &BeaconBlock<T>,
spec: &ChainSpec,
) -> Result<(), Error> {
per_block_processing_signature_optional(state, block, false, spec)
@@ -70,10 +71,10 @@ pub fn per_block_processing_without_verifying_block_signature<T: EthSpec>(
/// Returns `Ok(())` if the block is valid and the state was successfully updated. Otherwise
/// returns an error describing why the block was invalid or how the function failed to execute.
///
/// Spec v0.6.3
/// Spec v0.8.0
fn per_block_processing_signature_optional<T: EthSpec>(
mut state: &mut BeaconState<T>,
block: &BeaconBlock,
block: &BeaconBlock<T>,
should_verify_block_signature: bool,
spec: &ChainSpec,
) -> Result<(), Error> {
@@ -84,7 +85,7 @@ fn per_block_processing_signature_optional<T: EthSpec>(
state.build_committee_cache(RelativeEpoch::Current, spec)?;
process_randao(&mut state, &block, &spec)?;
process_eth1_data(&mut state, &block.body.eth1_data, spec)?;
process_eth1_data(&mut state, &block.body.eth1_data)?;
process_proposer_slashings(&mut state, &block.body.proposer_slashings, spec)?;
process_attester_slashings(&mut state, &block.body.attester_slashings, spec)?;
process_attestations(&mut state, &block.body.attestations, spec)?;
@@ -97,10 +98,10 @@ fn per_block_processing_signature_optional<T: EthSpec>(
/// Processes the block header.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_block_header<T: EthSpec>(
state: &mut BeaconState<T>,
block: &BeaconBlock,
block: &BeaconBlock<T>,
spec: &ChainSpec,
should_verify_block_signature: bool,
) -> Result<(), Error> {
@@ -109,18 +110,18 @@ pub fn process_block_header<T: EthSpec>(
let expected_previous_block_root =
Hash256::from_slice(&state.latest_block_header.signed_root());
verify!(
block.previous_block_root == expected_previous_block_root,
block.parent_root == expected_previous_block_root,
Invalid::ParentBlockRootMismatch {
state: expected_previous_block_root,
block: block.previous_block_root,
block: block.parent_root,
}
);
state.latest_block_header = block.temporary_block_header(spec);
state.latest_block_header = block.temporary_block_header();
// Verify proposer is not slashed
let proposer_idx = state.get_beacon_proposer_index(block.slot, RelativeEpoch::Current, spec)?;
let proposer = &state.validator_registry[proposer_idx];
let proposer = &state.validators[proposer_idx];
verify!(!proposer.slashed, Invalid::ProposerSlashed(proposer_idx));
if should_verify_block_signature {
@@ -132,13 +133,13 @@ pub fn process_block_header<T: EthSpec>(
/// Verifies the signature of a block.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn verify_block_signature<T: EthSpec>(
state: &BeaconState<T>,
block: &BeaconBlock,
block: &BeaconBlock<T>,
spec: &ChainSpec,
) -> Result<(), Error> {
let block_proposer = &state.validator_registry
let block_proposer = &state.validators
[state.get_beacon_proposer_index(block.slot, RelativeEpoch::Current, spec)?];
let domain = spec.get_domain(
@@ -160,16 +161,16 @@ pub fn verify_block_signature<T: EthSpec>(
/// Verifies the `randao_reveal` against the block's proposer pubkey and updates
/// `state.latest_randao_mixes`.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_randao<T: EthSpec>(
state: &mut BeaconState<T>,
block: &BeaconBlock,
block: &BeaconBlock<T>,
spec: &ChainSpec,
) -> Result<(), Error> {
let block_proposer = &state.validator_registry
let block_proposer = &state.validators
[state.get_beacon_proposer_index(block.slot, RelativeEpoch::Current, spec)?];
// Verify the RANDAO is a valid signature of the proposer.
// Verify RANDAO reveal.
verify!(
block.body.randao_reveal.verify(
&state.current_epoch().tree_hash_root()[..],
@@ -191,22 +192,21 @@ pub fn process_randao<T: EthSpec>(
/// Update the `state.eth1_data_votes` based upon the `eth1_data` provided.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_eth1_data<T: EthSpec>(
state: &mut BeaconState<T>,
eth1_data: &Eth1Data,
spec: &ChainSpec,
) -> Result<(), Error> {
state.eth1_data_votes.push(eth1_data.clone());
state.eth1_data_votes.push(eth1_data.clone())?;
let num_votes = state
.eth1_data_votes
.iter()
.filter(|vote| *vote == eth1_data)
.count() as u64;
.count();
if num_votes * 2 > spec.slots_per_eth1_voting_period {
state.latest_eth1_data = eth1_data.clone();
if num_votes * 2 > T::SlotsPerEth1VotingPeriod::to_usize() {
state.eth1_data = eth1_data.clone();
}
Ok(())
@@ -217,17 +217,12 @@ pub fn process_eth1_data<T: EthSpec>(
/// Returns `Ok(())` if the validation and state updates completed successfully, otherwise returns
/// an `Err` describing the invalid object or cause of failure.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_proposer_slashings<T: EthSpec>(
state: &mut BeaconState<T>,
proposer_slashings: &[ProposerSlashing],
spec: &ChainSpec,
) -> Result<(), Error> {
verify!(
proposer_slashings.len() as u64 <= spec.max_proposer_slashings,
Invalid::MaxProposerSlashingsExceeded
);
// Verify proposer slashings in parallel.
proposer_slashings
.par_iter()
@@ -250,21 +245,15 @@ pub fn process_proposer_slashings<T: EthSpec>(
/// Returns `Ok(())` if the validation and state updates completed successfully, otherwise returns
/// an `Err` describing the invalid object or cause of failure.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_attester_slashings<T: EthSpec>(
state: &mut BeaconState<T>,
attester_slashings: &[AttesterSlashing],
attester_slashings: &[AttesterSlashing<T>],
spec: &ChainSpec,
) -> Result<(), Error> {
verify!(
attester_slashings.len() as u64 <= spec.max_attester_slashings,
Invalid::MaxAttesterSlashingsExceed
);
// Verify the `IndexedAttestation`s in parallel (these are the resource-consuming objects, not
// the `AttesterSlashing`s themselves).
let mut indexed_attestations: Vec<&IndexedAttestation> =
Vec::with_capacity(attester_slashings.len() * 2);
let mut indexed_attestations: Vec<&_> = Vec::with_capacity(attester_slashings.len() * 2);
for attester_slashing in attester_slashings {
indexed_attestations.push(&attester_slashing.attestation_1);
indexed_attestations.push(&attester_slashing.attestation_2);
@@ -275,7 +264,7 @@ pub fn process_attester_slashings<T: EthSpec>(
.par_iter()
.enumerate()
.try_for_each(|(i, indexed_attestation)| {
verify_indexed_attestation(&state, indexed_attestation, spec)
is_valid_indexed_attestation(&state, indexed_attestation, spec)
.map_err(|e| e.into_with_index(i))
})?;
let all_indexed_attestations_have_been_checked = true;
@@ -308,17 +297,12 @@ pub fn process_attester_slashings<T: EthSpec>(
/// Returns `Ok(())` if the validation and state updates completed successfully, otherwise returns
/// an `Err` describing the invalid object or cause of failure.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_attestations<T: EthSpec>(
state: &mut BeaconState<T>,
attestations: &[Attestation],
attestations: &[Attestation<T>],
spec: &ChainSpec,
) -> Result<(), Error> {
verify!(
attestations.len() as u64 <= spec.max_attestations,
Invalid::MaxAttestationsExceeded
);
// Ensure the previous epoch cache exists.
state.build_committee_cache(RelativeEpoch::Previous, spec)?;
@@ -327,25 +311,27 @@ pub fn process_attestations<T: EthSpec>(
.par_iter()
.enumerate()
.try_for_each(|(i, attestation)| {
validate_attestation(state, attestation, spec).map_err(|e| e.into_with_index(i))
verify_attestation(state, attestation, spec).map_err(|e| e.into_with_index(i))
})?;
// Update the state in series.
let proposer_index =
state.get_beacon_proposer_index(state.slot, RelativeEpoch::Current, spec)? as u64;
for attestation in attestations {
let attestation_slot = state.get_attestation_slot(&attestation.data)?;
let attestation_slot = state.get_attestation_data_slot(&attestation.data)?;
let pending_attestation = PendingAttestation {
aggregation_bitfield: attestation.aggregation_bitfield.clone(),
aggregation_bits: attestation.aggregation_bits.clone(),
data: attestation.data.clone(),
inclusion_delay: (state.slot - attestation_slot).as_u64(),
proposer_index,
};
if attestation.data.target_epoch == state.current_epoch() {
state.current_epoch_attestations.push(pending_attestation)
if attestation.data.target.epoch == state.current_epoch() {
state.current_epoch_attestations.push(pending_attestation)?;
} else {
state.previous_epoch_attestations.push(pending_attestation)
state
.previous_epoch_attestations
.push(pending_attestation)?;
}
}
@@ -357,7 +343,7 @@ pub fn process_attestations<T: EthSpec>(
/// Returns `Ok(())` if the validation and state updates completed successfully, otherwise returns
/// an `Err` describing the invalid object or cause of failure.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_deposits<T: EthSpec>(
state: &mut BeaconState<T>,
deposits: &[Deposit],
@@ -366,8 +352,8 @@ pub fn process_deposits<T: EthSpec>(
verify!(
deposits.len() as u64
== std::cmp::min(
spec.max_deposits,
state.latest_eth1_data.deposit_count - state.deposit_index
T::MaxDeposits::to_u64(),
state.eth1_data.deposit_count - state.eth1_deposit_index
),
Invalid::DepositCountInvalid
);
@@ -377,14 +363,13 @@ pub fn process_deposits<T: EthSpec>(
.par_iter()
.enumerate()
.try_for_each(|(i, deposit)| {
verify_deposit_merkle_proof(state, deposit, spec).map_err(|e| e.into_with_index(i))
verify_deposit_merkle_proof(state, deposit, state.eth1_deposit_index + i as u64, spec)
.map_err(|e| e.into_with_index(i))
})?;
// Check `state.deposit_index` and update the state in series.
// Update the state in series.
for (i, deposit) in deposits.iter().enumerate() {
verify_deposit_index(state, deposit).map_err(|e| e.into_with_index(i))?;
state.deposit_index += 1;
state.eth1_deposit_index += 1;
// Ensure the state's pubkey cache is fully up-to-date, it will be used to check to see if the
// depositing validator already exists in the registry.
@@ -421,8 +406,8 @@ pub fn process_deposits<T: EthSpec>(
),
slashed: false,
};
state.validator_registry.push(validator);
state.balances.push(deposit.data.amount);
state.validators.push(validator)?;
state.balances.push(deposit.data.amount)?;
}
}
@@ -434,17 +419,12 @@ pub fn process_deposits<T: EthSpec>(
/// Returns `Ok(())` if the validation and state updates completed successfully, otherwise returns
/// an `Err` describing the invalid object or cause of failure.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_exits<T: EthSpec>(
state: &mut BeaconState<T>,
voluntary_exits: &[VoluntaryExit],
spec: &ChainSpec,
) -> Result<(), Error> {
verify!(
voluntary_exits.len() as u64 <= spec.max_voluntary_exits,
Invalid::MaxExitsExceeded
);
// Verify exits in parallel.
voluntary_exits
.par_iter()
@@ -466,15 +446,16 @@ pub fn process_exits<T: EthSpec>(
/// Returns `Ok(())` if the validation and state updates completed successfully, otherwise returns
/// an `Err` describing the invalid object or cause of failure.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_transfers<T: EthSpec>(
state: &mut BeaconState<T>,
transfers: &[Transfer],
spec: &ChainSpec,
) -> Result<(), Error> {
// Verify that there are no duplicate transfers
verify!(
transfers.len() as u64 <= spec.max_transfers,
Invalid::MaxTransfersExceed
transfers.len() == HashSet::<_>::from_iter(transfers).len(),
Invalid::DuplicateTransfers
);
transfers