gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-20 21:34:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

v1.1.6 Fork Choice changes (#2822)

Browse Source 
## Issue Addressed

Resolves: https://github.com/sigp/lighthouse/issues/2741
Includes: https://github.com/sigp/lighthouse/pull/2853 so that we can get ssz static tests passing here on v1.1.6. If we want to merge that first, we can make this diff slightly smaller

## Proposed Changes

- Changes the `justified_epoch` and `finalized_epoch` in the `ProtoArrayNode` each to an `Option<Checkpoint>`. The `Option` is necessary only for the migration, so not ideal. But does allow us to add a default logic to `None` on these fields during the database migration.
- Adds a database migration from a legacy fork choice struct to the new one, search for all necessary block roots in fork choice by iterating through blocks in the db.
- updates related to https://github.com/ethereum/consensus-specs/pull/2727
  -  We will have to update the persisted forkchoice to make sure the justified checkpoint stored is correct according to the updated fork choice logic. This boils down to setting the forkchoice store's justified checkpoint to the justified checkpoint of the block that advanced the finalized checkpoint to the current one. 
  - AFAICT there's no migration steps necessary for the update to allow applying attestations from prior blocks, but would appreciate confirmation on that
- I updated the consensus spec tests to v1.1.6 here, but they will fail until we also implement the proposer score boost updates. I confirmed that the previously failing scenario `new_finalized_slot_is_justified_checkpoint_ancestor` will now pass after the boost updates, but haven't confirmed _all_ tests will pass because I just quickly stubbed out the proposer boost test scenario formatting.
- This PR now also includes proposer boosting https://github.com/ethereum/consensus-specs/pull/2730

## Additional Info
I realized checking justified and finalized roots in fork choice makes it more likely that we trigger this bug: https://github.com/ethereum/consensus-specs/pull/2727

It's possible the combination of justified checkpoint and finalized checkpoint in the forkchoice store is different from in any block in fork choice. So when trying to startup our store's justified checkpoint seems invalid to the rest of fork choice (but it should be valid). When this happens we get an `InvalidBestNode` error and fail to start up. So I'm including that bugfix in this branch.

Todo:

- [x] Fix fork choice tests
- [x] Self review
- [x] Add fix for https://github.com/ethereum/consensus-specs/pull/2727
- [x] Rebase onto Kintusgi 
- [x] Fix `num_active_validators` calculation as @michaelsproul pointed out
- [x] Clean up db migrations

Co-authored-by: realbigsean <seananderson33@gmail.com>
This commit is contained in:
realbigsean
2021-12-13 20:43:22 +00:00
parent e391b32858
commit b22ac95d7f
44 changed files with 1802 additions and 658 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
testing/ef_tests/src/cases/fork_choice.rs
View File

@@ -1,6 +1,7 @@
use super::*;
use crate::decode::{ssz_decode_file, ssz_decode_file_with, ssz_decode_state, yaml_decode_file};
use ::fork_choice::PayloadVerificationStatus;
use beacon_chain::slot_clock::SlotClock;
use beacon_chain::{
attestation_verification::{
obtain_indexed_attestation_and_committees_per_slot, VerifiedAttestation,
@@ -23,10 +24,6 @@ pub struct PowBlock {
pub block_hash: Hash256,
pub parent_hash: Hash256,
pub total_difficulty: Uint256,
// This field is not used and I expect it to be removed. See:
//
// https://github.com/ethereum/consensus-specs/pull/2720
pub difficulty: Uint256,
}
#[derive(Debug, Clone, Copy, PartialEq, Deserialize)]
@@ -46,6 +43,7 @@ pub struct Checks {
justified_checkpoint_root: Option<Hash256>,
finalized_checkpoint: Option<Checkpoint>,
best_justified_checkpoint: Option<Checkpoint>,
proposer_boost_root: Option<Hash256>,
}
#[derive(Debug, Clone, Deserialize)]
@@ -74,6 +72,15 @@ pub struct ForkChoiceTest<E: EthSpec> {
pub steps: Vec<Step<SignedBeaconBlock<E>, Attestation<E>, PowBlock>>,
}
/// Spec for fork choice tests, with proposer boosting enabled.
///
/// This function can be deleted once `ChainSpec::mainnet` enables proposer boosting by default.
pub fn fork_choice_spec<E: EthSpec>(fork_name: ForkName) -> ChainSpec {
let mut spec = testing_spec::<E>(fork_name);
spec.proposer_score_boost = Some(70);
spec
}
impl<E: EthSpec> LoadCase for ForkChoiceTest<E> {
fn load_from_dir(path: &Path, fork_name: ForkName) -> Result<Self, Error> {
let description = path
@@ -83,7 +90,7 @@ impl<E: EthSpec> LoadCase for ForkChoiceTest<E> {
.to_str()
.expect("path must be valid OsStr")
.to_string();
let spec = &testing_spec::<E>(fork_name);
let spec = &fork_choice_spec::<E>(fork_name);
let steps: Vec<Step<String, String, String>> = yaml_decode_file(&path.join("steps.yaml"))?;
// Resolve the object names in `steps.yaml` into actual decoded block/attestation objects.
let steps = steps
@@ -145,18 +152,15 @@ impl<E: EthSpec> Case for ForkChoiceTest<E> {
}
fn result(&self, _case_index: usize, fork_name: ForkName) -> Result<(), Error> {
let tester = Tester::new(self, testing_spec::<E>(fork_name))?;
let tester = Tester::new(self, fork_choice_spec::<E>(fork_name))?;
// The reason for this failure is documented here:
// TODO(merge): enable these tests before production.
// This test will fail until this PR is merged and released:
//
// https://github.com/sigp/lighthouse/issues/2741
//
// We should eventually solve the above issue and remove this `SkippedKnownFailure`.
if self.description == "new_finalized_slot_is_justified_checkpoint_ancestor"
// https://github.com/ethereum/consensus-specs/pull/2760
if self.description == "shorter_chain_but_heavier_weight"
// This test is skipped until we can do retrospective confirmations of the terminal
// block after an optimistic sync.
//
// TODO(merge): enable this test before production.
|| self.description == "block_lookup_failed"
{
return Err(Error::SkippedKnownFailure);
@@ -180,6 +184,7 @@ impl<E: EthSpec> Case for ForkChoiceTest<E> {
justified_checkpoint_root,
finalized_checkpoint,
best_justified_checkpoint,
proposer_boost_root,
} = checks.as_ref();
if let Some(expected_head) = head {
@@ -211,6 +216,10 @@ impl<E: EthSpec> Case for ForkChoiceTest<E> {
tester
.check_best_justified_checkpoint(*expected_best_justified_checkpoint)?;
}
if let Some(expected_proposer_boost_root) = proposer_boost_root {
tester.check_expected_proposer_boost_root(*expected_proposer_boost_root)?;
}
}
}
}
@@ -352,11 +361,19 @@ impl<E: EthSpec> Tester<E> {
)
.unwrap();
let block_delay = self
.harness
.chain
.slot_clock
.seconds_from_current_slot_start(self.spec.seconds_per_slot)
.unwrap();
let (block, _) = block.deconstruct();
let result = self.harness.chain.fork_choice.write().on_block(
self.harness.chain.slot().unwrap(),
&block,
block_root,
block_delay,
&state,
PayloadVerificationStatus::Irrelevant,
&self.harness.chain.spec,
@@ -494,6 +511,18 @@ impl<E: EthSpec> Tester<E> {
expected_checkpoint,
)
}
pub fn check_expected_proposer_boost_root(
&self,
expected_proposer_boost_root: Hash256,
) -> Result<(), Error> {
let proposer_boost_root = self.harness.chain.fork_choice.read().proposer_boost_root();
check_equal(
"proposer_boost_root",
proposer_boost_root,
expected_proposer_boost_root,
)
}
}
/// Checks that the `head` checkpoint from the beacon chain head matches the `fc` checkpoint gleaned

2
testing/ef_tests/src/cases/transition.rs
View File

@@ -72,7 +72,7 @@ impl<E: EthSpec> Case for TransitionTest<E> {
fn is_enabled_for_fork(fork_name: ForkName) -> bool {
// Upgrades exist targeting all forks except phase0/base.
// Transition tests also need BLS.
// FIXME(merge): enable merge tests once available
// FIXME(merge): Merge transition tests are now available but not yet passing
cfg!(not(feature = "fake_crypto"))
&& fork_name != ForkName::Base
&& fork_name != ForkName::Merge