gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-15 10:52:43 +00:00
Code Issues Packages Projects Releases Wiki Activity

Update to frozen spec ❄️ (v0.8.1) (#444)

Browse Source 
* types: first updates for v0.8

* state_processing: epoch processing v0.8.0

* state_processing: block processing v0.8.0

* tree_hash_derive: support generics in SignedRoot

* types v0.8: update to use ssz_types

* state_processing v0.8: use ssz_types

* ssz_types: add bitwise methods and from_elem

* types: fix v0.8 FIXMEs

* ssz_types: add bitfield shift_up

* ssz_types: iterators and DerefMut for VariableList

* types,state_processing: use VariableList

* ssz_types: fix BitVector Decode impl

Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it
to be considered variable length!

* types: fix test modules for v0.8 update

* types: remove slow type-level arithmetic

* state_processing: fix tests for v0.8

* op_pool: update for v0.8

* ssz_types: Bitfield difference length-independent

Allow computing the difference of two bitfields of different lengths.

* Implement compact committee support

* epoch_processing: committee & active index roots

* state_processing: genesis state builder v0.8

* state_processing: implement v0.8.1

* Further improve tree_hash

* Strip examples, tests from cached_tree_hash

* Update TreeHash, un-impl CachedTreeHash

* Update bitfield TreeHash, un-impl CachedTreeHash

* Update FixedLenVec TreeHash, unimpl CachedTreeHash

* Update update tree_hash_derive for new TreeHash

* Fix TreeHash, un-impl CachedTreeHash for ssz_types

* Remove fixed_len_vec, ssz benches

SSZ benches relied upon fixed_len_vec -- it is easier to just delete
them and rebuild them later (when necessary)

* Remove boolean_bitfield crate

* Fix fake_crypto BLS compile errors

* Update ef_tests for new v.8 type params

* Update ef_tests submodule to v0.8.1 tag

* Make fixes to support parsing ssz ef_tests

* `compact_committee...` to `compact_committees...`

* Derive more traits for `CompactCommittee`

* Flip bitfield byte-endianness

* Fix tree_hash for bitfields

* Modify CLI output for ef_tests

* Bump ssz crate version

* Update ssz_types doc comment

* Del cached tree hash tests from ssz_static tests

* Tidy SSZ dependencies

* Rename ssz_types crate to eth2_ssz_types

* validator_client: update for v0.8

* ssz_types: update union/difference for bit order swap

* beacon_node: update for v0.8, EthSpec

* types: disable cached tree hash, update min spec

* state_processing: fix slot bug in committee update

* tests: temporarily disable fork choice harness test

See #447

* committee cache: prevent out-of-bounds access

In the case where we tried to access the committee of a shard that didn't have a committee in the
current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This
commit adds a check to make the failure safe and explicit.

* fix bug in get_indexed_attestation and simplify

There was a bug in our implementation of get_indexed_attestation whereby
incorrect "committee indices" were used to index into the custody bitfield. The
bug was only observable in the case where some bits of the custody bitfield were
set to 1. The implementation has been simplified to remove the bug, and a test
added.

* state_proc: workaround for compact committees bug

https://github.com/ethereum/eth2.0-specs/issues/1315

* v0.8: updates to make the EF tests pass

* Remove redundant max operation checks.
* Always supply both messages when checking attestation signatures -- allowing
  verification of an attestation with no signatures.
* Swap the order of the fork and domain constant in `get_domain`, to match
  the spec.

* rustfmt

* ef_tests: add new epoch processing tests

* Integrate v0.8 into master (compiles)

* Remove unused crates, fix clippy lints

* Replace v0.6.3 tags w/ v0.8.1

* Remove old comment

* Ensure lmd ghost tests only run in release

* Update readme
This commit is contained in:
Michael Sproul
2019-07-30 12:44:51 +10:00
committed by Paul Hauner
parent 177df12149
commit a236003a7b
184 changed files with 3332 additions and 4542 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
eth2/state_processing/src/per_epoch_processing/apply_rewards.rs
View File

@@ -32,7 +32,7 @@ impl std::ops::AddAssign for Delta {
/// Apply attester and proposer rewards.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_rewards_and_penalties<T: EthSpec>(
state: &mut BeaconState<T>,
validator_statuses: &mut ValidatorStatuses,
@@ -45,7 +45,7 @@ pub fn process_rewards_and_penalties<T: EthSpec>(
// Guard against an out-of-bounds during the validator balance update.
if validator_statuses.statuses.len() != state.balances.len()
|| validator_statuses.statuses.len() != state.validator_registry.len()
|| validator_statuses.statuses.len() != state.validators.len()
{
return Err(Error::ValidatorStatusesInconsistent);
}
@@ -74,7 +74,7 @@ pub fn process_rewards_and_penalties<T: EthSpec>(
/// For each attesting validator, reward the proposer who was first to include their attestation.
///
/// Spec v0.6.3
/// Spec v0.8.0
fn get_proposer_deltas<T: EthSpec>(
deltas: &mut Vec<Delta>,
state: &BeaconState<T>,
@@ -85,7 +85,7 @@ fn get_proposer_deltas<T: EthSpec>(
// Update statuses with the information from winning roots.
validator_statuses.process_winning_roots(state, winning_root_for_shards, spec)?;
for validator in &validator_statuses.statuses {
for (index, validator) in validator_statuses.statuses.iter().enumerate() {
if validator.is_previous_epoch_attester {
let inclusion = validator
.inclusion_info
@@ -93,7 +93,7 @@ fn get_proposer_deltas<T: EthSpec>(
let base_reward = get_base_reward(
state,
inclusion.proposer_index,
index,
validator_statuses.total_balances.current_epoch,
spec,
)?;
@@ -111,14 +111,14 @@ fn get_proposer_deltas<T: EthSpec>(
/// Apply rewards for participation in attestations during the previous epoch.
///
/// Spec v0.6.3
/// Spec v0.8.0
fn get_attestation_deltas<T: EthSpec>(
deltas: &mut Vec<Delta>,
state: &BeaconState<T>,
validator_statuses: &ValidatorStatuses,
spec: &ChainSpec,
) -> Result<(), Error> {
let finality_delay = (state.previous_epoch() - state.finalized_epoch).as_u64();
let finality_delay = (state.previous_epoch() - state.finalized_checkpoint.epoch).as_u64();
for (index, validator) in validator_statuses.statuses.iter().enumerate() {
let base_reward = get_base_reward(
@@ -128,7 +128,7 @@ fn get_attestation_deltas<T: EthSpec>(
spec,
)?;
let delta = get_attestation_delta(
let delta = get_attestation_delta::<T>(
&validator,
&validator_statuses.total_balances,
base_reward,
@@ -144,8 +144,8 @@ fn get_attestation_deltas<T: EthSpec>(
/// Determine the delta for a single validator, sans proposer rewards.
///
/// Spec v0.6.3
fn get_attestation_delta(
/// Spec v0.8.0
fn get_attestation_delta<T: EthSpec>(
validator: &ValidatorStatus,
total_balances: &TotalBalances,
base_reward: u64,
@@ -174,10 +174,17 @@ fn get_attestation_delta(
if validator.is_previous_epoch_attester && !validator.is_slashed {
delta.reward(base_reward * total_attesting_balance / total_balance);
// Inclusion speed bonus
let proposer_reward = base_reward / spec.proposer_reward_quotient;
let max_attester_reward = base_reward - proposer_reward;
let inclusion = validator
.inclusion_info
.expect("It is a logic error for an attester not to have an inclusion distance.");
delta.reward(base_reward * spec.min_attestation_inclusion_delay / inclusion.distance);
delta.reward(
max_attester_reward
* (T::SlotsPerEpoch::to_u64() + spec.min_attestation_inclusion_delay
- inclusion.distance)
/ T::SlotsPerEpoch::to_u64(),
);
} else {
delta.penalize(base_reward);
}
@@ -224,7 +231,7 @@ fn get_attestation_delta(
/// Calculate the deltas based upon the winning roots for attestations during the previous epoch.
///
/// Spec v0.6.3
/// Spec v0.8.0
fn get_crosslink_deltas<T: EthSpec>(
deltas: &mut Vec<Delta>,
state: &BeaconState<T>,
@@ -258,7 +265,7 @@ fn get_crosslink_deltas<T: EthSpec>(
/// Returns the base reward for some validator.
///
/// Spec v0.6.3
/// Spec v0.8.0
fn get_base_reward<T: EthSpec>(
state: &BeaconState<T>,
index: usize,
@@ -269,9 +276,10 @@ fn get_base_reward<T: EthSpec>(
if total_active_balance == 0 {
Ok(0)
} else {
let adjusted_quotient = total_active_balance.integer_sqrt() / spec.base_reward_quotient;
Ok(state.get_effective_balance(index, spec)?
/ adjusted_quotient
/ spec.base_rewards_per_epoch)
Ok(
state.get_effective_balance(index, spec)? * spec.base_reward_factor
/ total_active_balance.integer_sqrt()
/ spec.base_rewards_per_epoch,
)
}
}

7
eth2/state_processing/src/per_epoch_processing/errors.rs
View File

@@ -17,6 +17,7 @@ pub enum EpochProcessingError {
InclusionSlotsInconsistent(usize),
BeaconStateError(BeaconStateError),
InclusionError(InclusionError),
SszTypesError(ssz_types::Error),
}
impl From<InclusionError> for EpochProcessingError {
@@ -31,6 +32,12 @@ impl From<BeaconStateError> for EpochProcessingError {
}
}
impl From<ssz_types::Error> for EpochProcessingError {
fn from(e: ssz_types::Error) -> EpochProcessingError {
EpochProcessingError::SszTypesError(e)
}
}
#[derive(Debug, PartialEq)]
pub enum InclusionError {
/// The validator did not participate in an attestation in this period.

31
eth2/state_processing/src/per_epoch_processing/process_slashings.rs
View File

@@ -2,30 +2,23 @@ use types::{BeaconStateError as Error, *};
/// Process slashings.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_slashings<T: EthSpec>(
state: &mut BeaconState<T>,
current_total_balance: u64,
total_balance: u64,
spec: &ChainSpec,
) -> Result<(), Error> {
let current_epoch = state.current_epoch();
let epoch = state.current_epoch();
let sum_slashings = state.get_all_slashings().iter().sum::<u64>();
let total_at_start = state.get_slashed_balance(current_epoch + 1)?;
let total_at_end = state.get_slashed_balance(current_epoch)?;
let total_penalties = total_at_end - total_at_start;
for (index, validator) in state.validator_registry.iter().enumerate() {
let should_penalize = current_epoch.as_usize() + T::LatestSlashedExitLength::to_usize() / 2
== validator.withdrawable_epoch.as_usize();
if validator.slashed && should_penalize {
let effective_balance = state.get_effective_balance(index, spec)?;
let penalty = std::cmp::max(
effective_balance * std::cmp::min(total_penalties * 3, current_total_balance)
/ current_total_balance,
effective_balance / spec.min_slashing_penalty_quotient,
);
for (index, validator) in state.validators.iter().enumerate() {
if validator.slashed
&& epoch + T::EpochsPerSlashingsVector::to_u64() / 2 == validator.withdrawable_epoch
{
let increment = spec.effective_balance_increment;
let penalty_numerator = validator.effective_balance / increment
* std::cmp::min(sum_slashings * 3, total_balance);
let penalty = penalty_numerator / total_balance * increment;
safe_sub_assign!(state.balances[index], penalty);
}

16
eth2/state_processing/src/per_epoch_processing/registry_updates.rs
View File

@@ -5,7 +5,7 @@ use types::*;
/// Performs a validator registry update, if required.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn process_registry_updates<T: EthSpec>(
state: &mut BeaconState<T>,
spec: &ChainSpec,
@@ -17,14 +17,14 @@ pub fn process_registry_updates<T: EthSpec>(
let current_epoch = state.current_epoch();
let is_eligible = |validator: &Validator| {
validator.activation_eligibility_epoch == spec.far_future_epoch
&& validator.effective_balance >= spec.max_effective_balance
&& validator.effective_balance == spec.max_effective_balance
};
let is_exiting_validator = |validator: &Validator| {
validator.is_active_at(current_epoch)
&& validator.effective_balance <= spec.ejection_balance
};
let (eligible_validators, exiting_validators): (Vec<_>, Vec<_>) = state
.validator_registry
.validators
.iter()
.enumerate()
.filter(|(_, validator)| is_eligible(validator) || is_exiting_validator(validator))
@@ -36,7 +36,7 @@ pub fn process_registry_updates<T: EthSpec>(
}
});
for index in eligible_validators {
state.validator_registry[index].activation_eligibility_epoch = current_epoch;
state.validators[index].activation_eligibility_epoch = current_epoch;
}
for index in exiting_validators {
initiate_validator_exit(state, index, spec)?;
@@ -44,22 +44,22 @@ pub fn process_registry_updates<T: EthSpec>(
// Queue validators eligible for activation and not dequeued for activation prior to finalized epoch
let activation_queue = state
.validator_registry
.validators
.iter()
.enumerate()
.filter(|(_, validator)| {
validator.activation_eligibility_epoch != spec.far_future_epoch
&& validator.activation_epoch
>= state.get_delayed_activation_exit_epoch(state.finalized_epoch, spec)
>= state.compute_activation_exit_epoch(state.finalized_checkpoint.epoch, spec)
})
.sorted_by_key(|(_, validator)| validator.activation_eligibility_epoch)
.map(|(index, _)| index)
.collect_vec();
let churn_limit = state.get_churn_limit(spec)? as usize;
let delayed_activation_epoch = state.get_delayed_activation_exit_epoch(current_epoch, spec);
let delayed_activation_epoch = state.compute_activation_exit_epoch(current_epoch, spec);
for index in activation_queue.into_iter().take(churn_limit) {
let validator = &mut state.validator_registry[index];
let validator = &mut state.validators[index];
if validator.activation_epoch == spec.far_future_epoch {
validator.activation_epoch = delayed_activation_epoch;
}

40
eth2/state_processing/src/per_epoch_processing/validator_statuses.rs
View File

@@ -1,5 +1,5 @@
use super::WinningRootHashSet;
use crate::common::get_attesting_indices_unsorted;
use crate::common::get_attesting_indices;
use types::*;
/// Sets the boolean `var` on `self` to be true if it is true on `other`. Otherwise leaves `self`
@@ -162,15 +162,15 @@ impl ValidatorStatuses {
/// - Active validators
/// - Total balances for the current and previous epochs.
///
/// Spec v0.6.3
/// Spec v0.8.1
pub fn new<T: EthSpec>(
state: &BeaconState<T>,
spec: &ChainSpec,
) -> Result<Self, BeaconStateError> {
let mut statuses = Vec::with_capacity(state.validator_registry.len());
let mut statuses = Vec::with_capacity(state.validators.len());
let mut total_balances = TotalBalances::default();
for (i, validator) in state.validator_registry.iter().enumerate() {
for (i, validator) in state.validators.iter().enumerate() {
let effective_balance = state.get_effective_balance(i, spec)?;
let mut status = ValidatorStatus {
is_slashed: validator.slashed,
@@ -202,7 +202,7 @@ impl ValidatorStatuses {
/// Process some attestations from the given `state` updating the `statuses` and
/// `total_balances` fields.
///
/// Spec v0.6.3
/// Spec v0.8.1
pub fn process_attestations<T: EthSpec>(
&mut self,
state: &BeaconState<T>,
@@ -213,24 +213,23 @@ impl ValidatorStatuses {
.iter()
.chain(state.current_epoch_attestations.iter())
{
let attesting_indices =
get_attesting_indices_unsorted(state, &a.data, &a.aggregation_bitfield)?;
let attesting_indices = get_attesting_indices(state, &a.data, &a.aggregation_bits)?;
let mut status = ValidatorStatus::default();
// Profile this attestation, updating the total balances and generating an
// `ValidatorStatus` object that applies to all participants in the attestation.
if is_from_epoch(a, state.current_epoch()) {
if a.data.target.epoch == state.current_epoch() {
status.is_current_epoch_attester = true;
if target_matches_epoch_start_block(a, state, state.current_epoch())? {
status.is_current_epoch_target_attester = true;
}
} else if is_from_epoch(a, state.previous_epoch()) {
} else if a.data.target.epoch == state.previous_epoch() {
status.is_previous_epoch_attester = true;
// The inclusion slot and distance are only required for previous epoch attesters.
let attestation_slot = state.get_attestation_slot(&a.data)?;
let attestation_slot = state.get_attestation_data_slot(&a.data)?;
let inclusion_slot = attestation_slot + a.inclusion_delay;
let relative_epoch =
RelativeEpoch::from_slot(state.slot, inclusion_slot, T::slots_per_epoch())?;
@@ -289,7 +288,7 @@ impl ValidatorStatuses {
/// Update the `statuses` for each validator based upon whether or not they attested to the
/// "winning" shard block root for the previous epoch.
///
/// Spec v0.6.3
/// Spec v0.8.1
pub fn process_winning_roots<T: EthSpec>(
&mut self,
state: &BeaconState<T>,
@@ -321,37 +320,30 @@ impl ValidatorStatuses {
}
}
/// Returns `true` if some `PendingAttestation` is from the supplied `epoch`.
///
/// Spec v0.6.3
fn is_from_epoch(a: &PendingAttestation, epoch: Epoch) -> bool {
a.data.target_epoch == epoch
}
/// Returns `true` if the attestation's FFG target is equal to the hash of the `state`'s first
/// beacon block in the given `epoch`.
///
/// Spec v0.6.3
/// Spec v0.8.1
fn target_matches_epoch_start_block<T: EthSpec>(
a: &PendingAttestation,
a: &PendingAttestation<T>,
state: &BeaconState<T>,
epoch: Epoch,
) -> Result<bool, BeaconStateError> {
let slot = epoch.start_slot(T::slots_per_epoch());
let state_boundary_root = *state.get_block_root(slot)?;
Ok(a.data.target_root == state_boundary_root)
Ok(a.data.target.root == state_boundary_root)
}
/// Returns `true` if a `PendingAttestation` and `BeaconState` share the same beacon block hash for
/// the current slot of the `PendingAttestation`.
///
/// Spec v0.6.3
/// Spec v0.8.1
fn has_common_beacon_block_root<T: EthSpec>(
a: &PendingAttestation,
a: &PendingAttestation<T>,
state: &BeaconState<T>,
) -> Result<bool, BeaconStateError> {
let attestation_slot = state.get_attestation_slot(&a.data)?;
let attestation_slot = state.get_attestation_data_slot(&a.data)?;
let state_block_root = *state.get_block_root(attestation_slot)?;
Ok(a.data.beacon_block_root == state_block_root)

86
eth2/state_processing/src/per_epoch_processing/winning_root.rs
View File

@@ -1,4 +1,4 @@
use crate::common::get_attesting_indices_unsorted;
use crate::common::get_attesting_indices;
use std::collections::{HashMap, HashSet};
use tree_hash::TreeHash;
use types::*;
@@ -16,65 +16,48 @@ impl WinningRoot {
/// A winning root is "better" than another if it has a higher `total_attesting_balance`. Ties
/// are broken by favouring the higher `crosslink_data_root` value.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn is_better_than(&self, other: &Self) -> bool {
(
self.total_attesting_balance,
self.crosslink.crosslink_data_root,
) > (
other.total_attesting_balance,
other.crosslink.crosslink_data_root,
)
(self.total_attesting_balance, self.crosslink.data_root)
> (other.total_attesting_balance, other.crosslink.data_root)
}
}
/// Returns the `crosslink_data_root` with the highest total attesting balance for the given shard.
/// Breaks ties by favouring the smaller `crosslink_data_root` hash.
/// Returns the crosslink `data_root` with the highest total attesting balance for the given shard.
/// Breaks ties by favouring the smaller crosslink `data_root` hash.
///
/// The `WinningRoot` object also contains additional fields that are useful in later stages of
/// per-epoch processing.
///
/// Spec v0.6.3
/// Spec v0.8.0
pub fn winning_root<T: EthSpec>(
state: &BeaconState<T>,
shard: u64,
epoch: Epoch,
spec: &ChainSpec,
) -> Result<Option<WinningRoot>, BeaconStateError> {
let shard_attestations: Vec<&PendingAttestation> = state
let attestations: Vec<&_> = state
.get_matching_source_attestations(epoch)?
.iter()
.filter(|a| a.data.shard == shard)
.filter(|a| a.data.crosslink.shard == shard)
.collect();
let mut shard_crosslinks = Vec::with_capacity(shard_attestations.len());
for att in shard_attestations {
shard_crosslinks.push((
att,
state.get_crosslink_from_attestation_data(&att.data, spec)?,
));
}
// Build a map from crosslinks to attestations that support that crosslink.
let mut candidate_crosslink_map = HashMap::new();
let current_shard_crosslink_root = state.get_current_crosslink(shard)?.tree_hash_root();
let candidate_crosslinks = shard_crosslinks.into_iter().filter(|(_, c)| {
c.previous_crosslink_root.as_bytes() == &current_shard_crosslink_root[..]
|| c.tree_hash_root() == current_shard_crosslink_root
});
// Build a map from candidate crosslink to attestations that support that crosslink.
let mut candidate_crosslink_map: HashMap<Crosslink, Vec<&PendingAttestation>> = HashMap::new();
for (attestation, crosslink) in candidate_crosslinks {
let supporting_attestations = candidate_crosslink_map
.entry(crosslink)
.or_insert_with(Vec::new);
supporting_attestations.push(attestation);
}
if candidate_crosslink_map.is_empty() {
return Ok(None);
for a in attestations {
if a.data.crosslink.parent_root.as_bytes() == &current_shard_crosslink_root[..]
|| a.data.crosslink.tree_hash_root() == current_shard_crosslink_root
{
let supporting_attestations = candidate_crosslink_map
.entry(&a.data.crosslink)
.or_insert_with(Vec::new);
supporting_attestations.push(a);
}
}
// Find the maximum crosslink.
let mut winning_root = None;
for (crosslink, attestations) in candidate_crosslink_map {
let attesting_validator_indices =
@@ -83,7 +66,7 @@ pub fn winning_root<T: EthSpec>(
state.get_total_balance(&attesting_validator_indices, spec)?;
let candidate = WinningRoot {
crosslink,
crosslink: crosslink.clone(),
attesting_validator_indices,
total_attesting_balance,
};
@@ -102,24 +85,15 @@ pub fn winning_root<T: EthSpec>(
pub fn get_unslashed_attesting_indices_unsorted<T: EthSpec>(
state: &BeaconState<T>,
attestations: &[&PendingAttestation],
attestations: &[&PendingAttestation<T>],
) -> Result<Vec<usize>, BeaconStateError> {
let mut output = HashSet::new();
for a in attestations {
output.extend(get_attesting_indices_unsorted(
state,
&a.data,
&a.aggregation_bitfield,
)?);
output.extend(get_attesting_indices(state, &a.data, &a.aggregation_bits)?);
}
Ok(output
.into_iter()
.filter(|index| {
state
.validator_registry
.get(*index)
.map_or(false, |v| !v.slashed)
})
.filter(|index| state.validators.get(*index).map_or(false, |v| !v.slashed))
.collect())
}
@@ -131,16 +105,18 @@ mod tests {
fn is_better_than() {
let worse = WinningRoot {
crosslink: Crosslink {
epoch: Epoch::new(0),
previous_crosslink_root: Hash256::from_slice(&[0; 32]),
crosslink_data_root: Hash256::from_slice(&[1; 32]),
shard: 0,
start_epoch: Epoch::new(0),
end_epoch: Epoch::new(1),
parent_root: Hash256::from_slice(&[0; 32]),
data_root: Hash256::from_slice(&[1; 32]),
},
attesting_validator_indices: vec![],
total_attesting_balance: 42,
};
let mut better = worse.clone();
better.crosslink.crosslink_data_root = Hash256::from_slice(&[2; 32]);
better.crosslink.data_root = Hash256::from_slice(&[2; 32]);
assert!(better.is_better_than(&worse));