gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-11 18:04:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

v1.1.6 Fork Choice changes (#2822)

Browse Source 
## Issue Addressed

Resolves: https://github.com/sigp/lighthouse/issues/2741
Includes: https://github.com/sigp/lighthouse/pull/2853 so that we can get ssz static tests passing here on v1.1.6. If we want to merge that first, we can make this diff slightly smaller

## Proposed Changes

- Changes the `justified_epoch` and `finalized_epoch` in the `ProtoArrayNode` each to an `Option<Checkpoint>`. The `Option` is necessary only for the migration, so not ideal. But does allow us to add a default logic to `None` on these fields during the database migration.
- Adds a database migration from a legacy fork choice struct to the new one, search for all necessary block roots in fork choice by iterating through blocks in the db.
- updates related to https://github.com/ethereum/consensus-specs/pull/2727
  -  We will have to update the persisted forkchoice to make sure the justified checkpoint stored is correct according to the updated fork choice logic. This boils down to setting the forkchoice store's justified checkpoint to the justified checkpoint of the block that advanced the finalized checkpoint to the current one. 
  - AFAICT there's no migration steps necessary for the update to allow applying attestations from prior blocks, but would appreciate confirmation on that
- I updated the consensus spec tests to v1.1.6 here, but they will fail until we also implement the proposer score boost updates. I confirmed that the previously failing scenario `new_finalized_slot_is_justified_checkpoint_ancestor` will now pass after the boost updates, but haven't confirmed _all_ tests will pass because I just quickly stubbed out the proposer boost test scenario formatting.
- This PR now also includes proposer boosting https://github.com/ethereum/consensus-specs/pull/2730

## Additional Info
I realized checking justified and finalized roots in fork choice makes it more likely that we trigger this bug: https://github.com/ethereum/consensus-specs/pull/2727

It's possible the combination of justified checkpoint and finalized checkpoint in the forkchoice store is different from in any block in fork choice. So when trying to startup our store's justified checkpoint seems invalid to the rest of fork choice (but it should be valid). When this happens we get an `InvalidBestNode` error and fail to start up. So I'm including that bugfix in this branch.

Todo:

- [x] Fix fork choice tests
- [x] Self review
- [x] Add fix for https://github.com/ethereum/consensus-specs/pull/2727
- [x] Rebase onto Kintusgi 
- [x] Fix `num_active_validators` calculation as @michaelsproul pointed out
- [x] Clean up db migrations

Co-authored-by: realbigsean <seananderson33@gmail.com>
This commit is contained in:
realbigsean
2021-12-13 20:43:22 +00:00
parent e391b32858
commit b22ac95d7f
44 changed files with 1802 additions and 658 deletions
Download Patch File Download Diff File Expand all files Collapse all files

122
consensus/proto_array/src/proto_array_fork_choice.rs
View File

@@ -1,11 +1,11 @@
use crate::error::Error;
use crate::proto_array::ProtoArray;
use crate::ssz_container::{LegacySszContainer, SszContainer};
use crate::proto_array::{ProposerBoost, ProtoArray};
use crate::ssz_container::SszContainer;
use serde_derive::{Deserialize, Serialize};
use ssz::{Decode, Encode};
use ssz_derive::{Decode, Encode};
use std::collections::HashMap;
use types::{AttestationShufflingId, Epoch, Hash256, Slot};
use types::{AttestationShufflingId, ChainSpec, Checkpoint, Epoch, EthSpec, Hash256, Slot};
pub const DEFAULT_PRUNE_THRESHOLD: usize = 256;
@@ -63,8 +63,8 @@ pub struct Block {
pub target_root: Hash256,
pub current_epoch_shuffling_id: AttestationShufflingId,
pub next_epoch_shuffling_id: AttestationShufflingId,
pub justified_epoch: Epoch,
pub finalized_epoch: Epoch,
pub justified_checkpoint: Checkpoint,
pub finalized_checkpoint: Checkpoint,
/// Indicates if an execution node has marked this block as valid. Also contains the execution
/// block hash.
pub execution_status: ExecutionStatus,
@@ -109,33 +109,33 @@ impl ProtoArrayForkChoice {
pub fn new(
finalized_block_slot: Slot,
finalized_block_state_root: Hash256,
justified_epoch: Epoch,
finalized_epoch: Epoch,
finalized_root: Hash256,
justified_checkpoint: Checkpoint,
finalized_checkpoint: Checkpoint,
current_epoch_shuffling_id: AttestationShufflingId,
next_epoch_shuffling_id: AttestationShufflingId,
execution_status: ExecutionStatus,
) -> Result<Self, String> {
let mut proto_array = ProtoArray {
prune_threshold: DEFAULT_PRUNE_THRESHOLD,
justified_epoch,
finalized_epoch,
justified_checkpoint,
finalized_checkpoint,
nodes: Vec::with_capacity(1),
indices: HashMap::with_capacity(1),
previous_proposer_boost: ProposerBoost::default(),
};
let block = Block {
slot: finalized_block_slot,
root: finalized_root,
root: finalized_checkpoint.root,
parent_root: None,
state_root: finalized_block_state_root,
// We are using the finalized_root as the target_root, since it always lies on an
// epoch boundary.
target_root: finalized_root,
target_root: finalized_checkpoint.root,
current_epoch_shuffling_id,
next_epoch_shuffling_id,
justified_epoch,
finalized_epoch,
justified_checkpoint,
finalized_checkpoint,
execution_status,
};
@@ -176,12 +176,13 @@ impl ProtoArrayForkChoice {
.map_err(|e| format!("process_block_error: {:?}", e))
}
pub fn find_head(
pub fn find_head<E: EthSpec>(
&mut self,
justified_epoch: Epoch,
justified_root: Hash256,
finalized_epoch: Epoch,
justified_checkpoint: Checkpoint,
finalized_checkpoint: Checkpoint,
justified_state_balances: &[u64],
proposer_boost_root: Hash256,
spec: &ChainSpec,
) -> Result<Hash256, String> {
let old_balances = &mut self.balances;
@@ -196,13 +197,20 @@ impl ProtoArrayForkChoice {
.map_err(|e| format!("find_head compute_deltas failed: {:?}", e))?;
self.proto_array
.apply_score_changes(deltas, justified_epoch, finalized_epoch)
.apply_score_changes::<E>(
deltas,
justified_checkpoint,
finalized_checkpoint,
new_balances,
proposer_boost_root,
spec,
)
.map_err(|e| format!("find_head apply_score_changes failed: {:?}", e))?;
*old_balances = new_balances.to_vec();
self.proto_array
.find_head(&justified_root)
.find_head(&justified_checkpoint.root)
.map_err(|e| format!("find_head failed: {:?}", e))
}
@@ -236,18 +244,27 @@ impl ProtoArrayForkChoice {
.and_then(|i| self.proto_array.nodes.get(i))
.map(|parent| parent.root);
Some(Block {
slot: block.slot,
root: block.root,
parent_root,
state_root: block.state_root,
target_root: block.target_root,
current_epoch_shuffling_id: block.current_epoch_shuffling_id.clone(),
next_epoch_shuffling_id: block.next_epoch_shuffling_id.clone(),
justified_epoch: block.justified_epoch,
finalized_epoch: block.finalized_epoch,
execution_status: block.execution_status,
})
// If a node does not have a `finalized_checkpoint` or `justified_checkpoint` populated,
// it means it is not a descendant of the finalized checkpoint, so it is valid to return
// `None` here.
if let (Some(justified_checkpoint), Some(finalized_checkpoint)) =
(block.justified_checkpoint, block.finalized_checkpoint)
{
Some(Block {
slot: block.slot,
root: block.root,
parent_root,
state_root: block.state_root,
target_root: block.target_root,
current_epoch_shuffling_id: block.current_epoch_shuffling_id.clone(),
next_epoch_shuffling_id: block.next_epoch_shuffling_id.clone(),
justified_checkpoint,
finalized_checkpoint,
execution_status: block.execution_status,
})
} else {
None
}
}
/// Returns `true` if the `descendant_root` has an ancestor with `ancestor_root`. Always
@@ -295,28 +312,19 @@ impl ProtoArrayForkChoice {
.map_err(|e| format!("Failed to decode ProtoArrayForkChoice: {:?}", e))
}
/// Only used for SSZ deserialization of the persisted fork choice during the database migration
/// from schema 5 to schema 6.
pub fn from_bytes_legacy(bytes: &[u8]) -> Result<Self, String> {
LegacySszContainer::from_ssz_bytes(bytes)
.map(|legacy_container| {
let container: SszContainer = legacy_container.into();
container.into()
})
.map_err(|e| {
format!(
"Failed to decode ProtoArrayForkChoice during schema migration: {:?}",
e
)
})
}
/// Returns a read-lock to core `ProtoArray` struct.
///
/// Should only be used when encoding/decoding during troubleshooting.
pub fn core_proto_array(&self) -> &ProtoArray {
&self.proto_array
}
/// Returns a mutable reference to the core `ProtoArray` struct.
///
/// Should only be used during database schema migrations.
pub fn core_proto_array_mut(&mut self) -> &mut ProtoArray {
&mut self.proto_array
}
}
/// Returns a list of `deltas`, where there is one delta for each of the indices in
@@ -412,12 +420,16 @@ mod test_compute_deltas {
AttestationShufflingId::from_components(Epoch::new(0), Hash256::zero());
let execution_status = ExecutionStatus::irrelevant();
let genesis_checkpoint = Checkpoint {
epoch: genesis_epoch,
root: finalized_root,
};
let mut fc = ProtoArrayForkChoice::new(
genesis_slot,
state_root,
genesis_epoch,
genesis_epoch,
finalized_root,
genesis_checkpoint,
genesis_checkpoint,
junk_shuffling_id.clone(),
junk_shuffling_id.clone(),
execution_status,
@@ -434,8 +446,8 @@ mod test_compute_deltas {
target_root: finalized_root,
current_epoch_shuffling_id: junk_shuffling_id.clone(),
next_epoch_shuffling_id: junk_shuffling_id.clone(),
justified_epoch: genesis_epoch,
finalized_epoch: genesis_epoch,
justified_checkpoint: genesis_checkpoint,
finalized_checkpoint: genesis_checkpoint,
execution_status,
})
.unwrap();
@@ -450,8 +462,8 @@ mod test_compute_deltas {
target_root: finalized_root,
current_epoch_shuffling_id: junk_shuffling_id.clone(),
next_epoch_shuffling_id: junk_shuffling_id,
justified_epoch: genesis_epoch,
finalized_epoch: genesis_epoch,
justified_checkpoint: genesis_checkpoint,
finalized_checkpoint: genesis_checkpoint,
execution_status,
})
.unwrap();