v0.12 fork choice update (#1229)

* Incomplete scraps * Add progress on new fork choice impl * Further progress * First complete compiling version * Remove chain reference * Add new lmd_ghost crate * Start integrating into beacon chain * Update `milagro_bls` to new release (#1183) * Update milagro_bls to new release Signed-off-by: Kirk Baird <baird.k@outlook.com> * Tidy up fake cryptos Signed-off-by: Kirk Baird <baird.k@outlook.com> * move SecretHash to bls and put plaintext back Signed-off-by: Kirk Baird <baird.k@outlook.com> * Update state processing for v0.12 * Fix EF test runners for v0.12 * Fix some tests * Fix broken attestation verification test * More test fixes * Rough beacon chain impl working * Remove fork_choice_2 * Remove checkpoint manager * Half finished ssz impl * Add missed file * Add persistence * Tidy, fix some compile errors * Remove RwLock from ProtoArrayForkChoice * Fix store-based compile errors * Add comments, tidy * Move function out of ForkChoice struct * Start testing * More testing * Fix compile error * Tidy beacon_chain::fork_choice * Queue attestations from the current slot * Allow fork choice to handle prior-to-genesis start * Improve error granularity * Test attestation dequeuing * Process attestations during block * Store target root in fork choice * Move fork choice verification into new crate * Update tests * Consensus updates for v0.12 (#1228) * Update state processing for v0.12 * Fix EF test runners for v0.12 * Fix some tests * Fix broken attestation verification test * More test fixes * Fix typo found in review * Add `Block` struct to ProtoArray * Start fixing get_ancestor * Add rough progress on testing * Get fork choice tests working * Progress with testing * Fix partialeq impl * Move slot clock from fc_store * Improve testing * Add testing for best justified * Add clone back to SystemTimeSlotClock * Add balances test * Start adding balances cache again * Wire-in balances cache * Improve tests * Remove commented-out tests * Remove beacon_chain::ForkChoice * Rename crates * Update wider codebase to new fork_choice layout * Move advance_slot in test harness * Tidy ForkChoice::update_time * Fix verification tests * Fix compile error with iter::once * Fix fork choice tests * Ensure block attestations are processed * Fix failing beacon_chain tests * Add first invalid block check * Add finalized block check * Progress with testing, new store builder * Add fixes to get_ancestor * Fix old genesis justification test * Fix remaining fork choice tests * Change root iteration method * Move on_verified_block * Remove unused method * Start adding attestation verification tests * Add invalid ffg target test * Add target epoch test * Add queued attestation test * Remove old fork choice verification tests * Tidy, add test * Move fork choice lock drop * Rename BeaconForkChoiceStore * Add comments, tidy BeaconForkChoiceStore * Update metrics, rename fork_choice_store.rs * Remove genesis_block_root from ForkChoice * Tidy * Update fork_choice comments * Tidy, add comments * Tidy, simplify ForkChoice, fix compile issue * Tidy, removed dead file * Increase http request timeout * Fix failing rest_api test * Set HTTP timeout back to 5s * Apply fix to get_ancestor * Address Michael's comments * Fix typo * Revert "Fix broken attestation verification test" This reverts commit 722cdc903b. Co-authored-by: Kirk Baird <baird.k@outlook.com> Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2026-04-20 06:18:31 +00:00 · 2020-06-17 11:10:22 +10:00
parent 1a4de898bc
commit 764cb2d32a
51 changed files with 2641 additions and 1376 deletions
--- a/consensus/proto_array/src/fork_choice_test_definition.rs
+++ b/consensus/proto_array/src/fork_choice_test_definition.rs
@@ -0,0 +1,189 @@
+mod ffg_updates;
+mod no_votes;
+mod votes;
+
+use crate::proto_array_fork_choice::{Block, ProtoArrayForkChoice};
+use serde_derive::{Deserialize, Serialize};
+use types::{Epoch, Hash256, Slot};
+
+pub use ffg_updates::*;
+pub use no_votes::*;
+pub use votes::*;
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub enum Operation {
+    FindHead {
+        justified_epoch: Epoch,
+        justified_root: Hash256,
+        finalized_epoch: Epoch,
+        justified_state_balances: Vec<u64>,
+        expected_head: Hash256,
+    },
+    InvalidFindHead {
+        justified_epoch: Epoch,
+        justified_root: Hash256,
+        finalized_epoch: Epoch,
+        justified_state_balances: Vec<u64>,
+    },
+    ProcessBlock {
+        slot: Slot,
+        root: Hash256,
+        parent_root: Hash256,
+        justified_epoch: Epoch,
+        finalized_epoch: Epoch,
+    },
+    ProcessAttestation {
+        validator_index: usize,
+        block_root: Hash256,
+        target_epoch: Epoch,
+    },
+    Prune {
+        finalized_root: Hash256,
+        prune_threshold: usize,
+        expected_len: usize,
+    },
+}
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct ForkChoiceTestDefinition {
+    pub finalized_block_slot: Slot,
+    pub justified_epoch: Epoch,
+    pub finalized_epoch: Epoch,
+    pub finalized_root: Hash256,
+    pub operations: Vec<Operation>,
+}
+
+impl ForkChoiceTestDefinition {
+    pub fn run(self) {
+        let mut fork_choice = ProtoArrayForkChoice::new(
+            self.finalized_block_slot,
+            Hash256::zero(),
+            self.justified_epoch,
+            self.finalized_epoch,
+            self.finalized_root,
+        )
+        .expect("should create fork choice struct");
+
+        for (op_index, op) in self.operations.into_iter().enumerate() {
+            match op.clone() {
+                Operation::FindHead {
+                    justified_epoch,
+                    justified_root,
+                    finalized_epoch,
+                    justified_state_balances,
+                    expected_head,
+                } => {
+                    let head = fork_choice
+                        .find_head(
+                            justified_epoch,
+                            justified_root,
+                            finalized_epoch,
+                            &justified_state_balances,
+                        )
+                        .expect(&format!(
+                            "find_head op at index {} returned error",
+                            op_index
+                        ));
+
+                    assert_eq!(
+                        head, expected_head,
+                        "Operation at index {} failed checks. Operation: {:?}",
+                        op_index, op
+                    );
+                    check_bytes_round_trip(&fork_choice);
+                }
+                Operation::InvalidFindHead {
+                    justified_epoch,
+                    justified_root,
+                    finalized_epoch,
+                    justified_state_balances,
+                } => {
+                    let result = fork_choice.find_head(
+                        justified_epoch,
+                        justified_root,
+                        finalized_epoch,
+                        &justified_state_balances,
+                    );
+
+                    assert!(
+                        result.is_err(),
+                        "Operation at index {} . Operation: {:?}",
+                        op_index,
+                        op
+                    );
+                    check_bytes_round_trip(&fork_choice);
+                }
+                Operation::ProcessBlock {
+                    slot,
+                    root,
+                    parent_root,
+                    justified_epoch,
+                    finalized_epoch,
+                } => {
+                    let block = Block {
+                        slot,
+                        root,
+                        parent_root: Some(parent_root),
+                        state_root: Hash256::zero(),
+                        target_root: Hash256::zero(),
+                        justified_epoch,
+                        finalized_epoch,
+                    };
+                    fork_choice.process_block(block).expect(&format!(
+                        "process_block op at index {} returned error",
+                        op_index
+                    ));
+                    check_bytes_round_trip(&fork_choice);
+                }
+                Operation::ProcessAttestation {
+                    validator_index,
+                    block_root,
+                    target_epoch,
+                } => {
+                    fork_choice
+                        .process_attestation(validator_index, block_root, target_epoch)
+                        .expect(&format!(
+                            "process_attestation op at index {} returned error",
+                            op_index
+                        ));
+                    check_bytes_round_trip(&fork_choice);
+                }
+                Operation::Prune {
+                    finalized_root,
+                    prune_threshold,
+                    expected_len,
+                } => {
+                    fork_choice.set_prune_threshold(prune_threshold);
+                    fork_choice
+                        .maybe_prune(finalized_root)
+                        .expect("update_finalized_root op at index {} returned error");
+
+                    // Ensure that no pruning happened.
+                    assert_eq!(
+                        fork_choice.len(),
+                        expected_len,
+                        "Prune op at index {} failed with {} instead of {}",
+                        op_index,
+                        fork_choice.len(),
+                        expected_len
+                    );
+                }
+            }
+        }
+    }
+}
+
+/// Gives a hash that is not the zero hash (unless i is `usize::max_value)`.
+fn get_hash(i: u64) -> Hash256 {
+    Hash256::from_low_u64_be(i)
+}
+
+fn check_bytes_round_trip(original: &ProtoArrayForkChoice) {
+    let bytes = original.as_bytes();
+    let decoded =
+        ProtoArrayForkChoice::from_bytes(&bytes).expect("fork choice should decode from bytes");
+    assert!(
+        *original == decoded,
+        "fork choice should encode and decode without change"
+    );
+}