gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-21 13:54:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

Use async code when interacting with EL (#3244)

Browse Source 
## Overview

This rather extensive PR achieves two primary goals:

1. Uses the finalized/justified checkpoints of fork choice (FC), rather than that of the head state.
2. Refactors fork choice, block production and block processing to `async` functions.

Additionally, it achieves:

- Concurrent forkchoice updates to the EL and cache pruning after a new head is selected.
- Concurrent "block packing" (attestations, etc) and execution payload retrieval during block production.
- Concurrent per-block-processing and execution payload verification during block processing.
- The `Arc`-ification of `SignedBeaconBlock` during block processing (it's never mutated, so why not?):
    - I had to do this to deal with sending blocks into spawned tasks.
    - Previously we were cloning the beacon block at least 2 times during each block processing, these clones are either removed or turned into cheaper `Arc` clones.
    - We were also `Box`-ing and un-`Box`-ing beacon blocks as they moved throughout the networking crate. This is not a big deal, but it's nice to avoid shifting things between the stack and heap.
    - Avoids cloning *all the blocks* in *every chain segment* during sync.
    - It also has the potential to clean up our code where we need to pass an *owned* block around so we can send it back in the case of an error (I didn't do much of this, my PR is already big enough 😅)
- The `BeaconChain::HeadSafetyStatus` struct was removed. It was an old relic from prior merge specs.

For motivation for this change, see https://github.com/sigp/lighthouse/pull/3244#issuecomment-1160963273

## Changes to `canonical_head` and `fork_choice`

Previously, the `BeaconChain` had two separate fields:

```
canonical_head: RwLock<Snapshot>,
fork_choice: RwLock<BeaconForkChoice>
```

Now, we have grouped these values under a single struct:

```
canonical_head: CanonicalHead {
  cached_head: RwLock<Arc<Snapshot>>,
  fork_choice: RwLock<BeaconForkChoice>
} 
```

Apart from ergonomics, the only *actual* change here is wrapping the canonical head snapshot in an `Arc`. This means that we no longer need to hold the `cached_head` (`canonical_head`, in old terms) lock when we want to pull some values from it. This was done to avoid deadlock risks by preventing functions from acquiring (and holding) the `cached_head` and `fork_choice` locks simultaneously.

## Breaking Changes

### The `state` (root) field in the `finalized_checkpoint` SSE event

Consider the scenario where epoch `n` is just finalized, but `start_slot(n)` is skipped. There are two state roots we might in the `finalized_checkpoint` SSE event:

1. The state root of the finalized block, which is `get_block(finalized_checkpoint.root).state_root`.
4. The state root at slot of `start_slot(n)`, which would be the state from (1), but "skipped forward" through any skip slots.

Previously, Lighthouse would choose (2). However, we can see that when [Teku generates that event](de2b2801c8/data/beaconrestapi/src/main/java/tech/pegasys/teku/beaconrestapi/handlers/v1/events/EventSubscriptionManager.java (L171-L182)) it uses [`getStateRootFromBlockRoot`](de2b2801c8/data/provider/src/main/java/tech/pegasys/teku/api/ChainDataProvider.java (L336-L341)) which uses (1).

I have switched Lighthouse from (2) to (1). I think it's a somewhat arbitrary choice between the two, where (1) is easier to compute and is consistent with Teku.

## Notes for Reviewers

I've renamed `BeaconChain::fork_choice` to `BeaconChain::recompute_head`. Doing this helped ensure I broke all previous uses of fork choice and I also find it more descriptive. It describes an action and can't be confused with trying to get a reference to the `ForkChoice` struct.

I've changed the ordering of SSE events when a block is received. It used to be `[block, finalized, head]` and now it's `[block, head, finalized]`. It was easier this way and I don't think we were making any promises about SSE event ordering so it's not "breaking".

I've made it so fork choice will run when it's first constructed. I did this because I wanted to have a cached version of the last call to `get_head`. Ensuring `get_head` has been run *at least once* means that the cached values doesn't need to wrapped in an `Option`. This was fairly simple, it just involved passing a `slot` to the constructor so it knows *when* it's being run. When loading a fork choice from the store and a slot clock isn't handy I've just used the `slot` that was saved in the `fork_choice_store`. That seems like it would be a faithful representation of the slot when we saved it.

I added the `genesis_time: u64` to the `BeaconChain`. It's small, constant and nice to have around.

Since we're using FC for the fin/just checkpoints, we no longer get the `0x00..00` roots at genesis. You can see I had to remove a work-around in `ef-tests` here: b56be3bc2. I can't find any reason why this would be an issue, if anything I think it'll be better since the genesis-alias has caught us out a few times (0x00..00 isn't actually a real root). Edit: I did find a case where the `network` expected the 0x00..00 alias and patched it here: 3f26ac3e2.

You'll notice a lot of changes in tests. Generally, tests should be functionally equivalent. Here are the things creating the most diff-noise in tests:
- Changing tests to be `tokio::async` tests.
- Adding `.await` to fork choice, block processing and block production functions.
- Refactor of the `canonical_head` "API" provided by the `BeaconChain`. E.g., `chain.canonical_head.cached_head()` instead of `chain.canonical_head.read()`.
- Wrapping `SignedBeaconBlock` in an `Arc`.
- In the `beacon_chain/tests/block_verification`, we can't use the `lazy_static` `CHAIN_SEGMENT` variable anymore since it's generated with an async function. We just generate it in each test, not so efficient but hopefully insignificant.

I had to disable `rayon` concurrent tests in the `fork_choice` tests. This is because the use of `rayon` and `block_on` was causing a panic.

Co-authored-by: Mac L <mjladson@pm.me>
This commit is contained in:
Paul Hauner
2022-07-03 05:36:50 +00:00
parent e5212f1320
commit be4e261e74
106 changed files with 6515 additions and 4538 deletions
Download Patch File Download Diff File Expand all files Collapse all files

154
beacon_node/beacon_chain/src/test_utils.rs
View File

@@ -515,13 +515,38 @@ where
}
pub fn get_current_state(&self) -> BeaconState<E> {
self.chain.head().unwrap().beacon_state
self.chain.head_beacon_state_cloned()
}
pub fn get_current_state_and_root(&self) -> (BeaconState<E>, Hash256) {
let head = self.chain.head().unwrap();
let head = self.chain.head_snapshot();
let state_root = head.beacon_state_root();
(head.beacon_state, state_root)
(
head.beacon_state.clone_with_only_committee_caches(),
state_root,
)
}
pub fn head_slot(&self) -> Slot {
self.chain.canonical_head.cached_head().head_slot()
}
pub fn head_block_root(&self) -> Hash256 {
self.chain.canonical_head.cached_head().head_block_root()
}
pub fn finalized_checkpoint(&self) -> Checkpoint {
self.chain
.canonical_head
.cached_head()
.finalized_checkpoint()
}
pub fn justified_checkpoint(&self) -> Checkpoint {
self.chain
.canonical_head
.cached_head()
.justified_checkpoint()
}
pub fn get_current_slot(&self) -> Slot {
@@ -565,7 +590,7 @@ where
state.get_block_root(slot).unwrap() == state.get_block_root(slot - 1).unwrap()
}
pub fn make_block(
pub async fn make_block(
&self,
mut state: BeaconState<E>,
slot: Slot,
@@ -599,6 +624,7 @@ where
Some(graffiti),
ProduceBlockVerification::VerifyRandao,
)
.await
.unwrap();
let signed_block = block.sign(
@@ -613,7 +639,7 @@ where
/// Useful for the `per_block_processing` tests. Creates a block, and returns the state after
/// caches are built but before the generated block is processed.
pub fn make_block_return_pre_state(
pub async fn make_block_return_pre_state(
&self,
mut state: BeaconState<E>,
slot: Slot,
@@ -649,6 +675,7 @@ where
Some(graffiti),
ProduceBlockVerification::VerifyRandao,
)
.await
.unwrap();
let signed_block = block.sign(
@@ -1098,11 +1125,11 @@ where
let mut attestation_2 = attestation_1.clone();
attestation_2.data.index += 1;
let fork = self.chain.canonical_head.cached_head().head_fork();
for attestation in &mut [&mut attestation_1, &mut attestation_2] {
for &i in &attestation.attesting_indices {
let sk = &self.validator_keypairs[i as usize].sk;
let fork = self.chain.head_info().unwrap().fork;
let genesis_validators_root = self.chain.genesis_validators_root;
let domain = self.chain.spec.get_domain(
@@ -1156,11 +1183,11 @@ where
attestation_2.data.index += 1;
let fork = self.chain.canonical_head.cached_head().head_fork();
for attestation in &mut [&mut attestation_1, &mut attestation_2] {
for &i in &attestation.attesting_indices {
let sk = &self.validator_keypairs[i as usize].sk;
let fork = self.chain.head_info().unwrap().fork;
let genesis_validators_root = self.chain.genesis_validators_root;
let domain = self.chain.spec.get_domain(
@@ -1182,19 +1209,14 @@ where
}
pub fn make_proposer_slashing(&self, validator_index: u64) -> ProposerSlashing {
let mut block_header_1 = self
.chain
.head_beacon_block()
.unwrap()
.message()
.block_header();
let mut block_header_1 = self.chain.head_beacon_block().message().block_header();
block_header_1.proposer_index = validator_index;
let mut block_header_2 = block_header_1.clone();
block_header_2.state_root = Hash256::zero();
let sk = &self.validator_keypairs[validator_index as usize].sk;
let fork = self.chain.head_info().unwrap().fork;
let fork = self.chain.canonical_head.cached_head().head_fork();
let genesis_validators_root = self.chain.genesis_validators_root;
let mut signed_block_headers = vec![block_header_1, block_header_2]
@@ -1212,7 +1234,7 @@ where
pub fn make_voluntary_exit(&self, validator_index: u64, epoch: Epoch) -> SignedVoluntaryExit {
let sk = &self.validator_keypairs[validator_index as usize].sk;
let fork = self.chain.head_info().unwrap().fork;
let fork = self.chain.canonical_head.cached_head().head_fork();
let genesis_validators_root = self.chain.genesis_validators_root;
VoluntaryExit {
@@ -1235,7 +1257,7 @@ where
/// Create a new block, apply `block_modifier` to it, sign it and return it.
///
/// The state returned is a pre-block state at the same slot as the produced block.
pub fn make_block_with_modifier(
pub async fn make_block_with_modifier(
&self,
state: BeaconState<E>,
slot: Slot,
@@ -1244,7 +1266,7 @@ where
assert_ne!(slot, 0, "can't produce a block at slot 0");
assert!(slot >= state.slot());
let (block, state) = self.make_block_return_pre_state(state, slot);
let (block, state) = self.make_block_return_pre_state(state, slot).await;
let (mut block, _) = block.deconstruct();
block_modifier(&mut block);
@@ -1332,23 +1354,25 @@ where
(deposits, state)
}
pub fn process_block(
pub async fn process_block(
&self,
slot: Slot,
block: SignedBeaconBlock<E>,
) -> Result<SignedBeaconBlockHash, BlockError<E>> {
self.set_current_slot(slot);
let block_hash: SignedBeaconBlockHash = self.chain.process_block(block)?.into();
self.chain.fork_choice()?;
let block_hash: SignedBeaconBlockHash =
self.chain.process_block(Arc::new(block)).await?.into();
self.chain.recompute_head_at_current_slot().await?;
Ok(block_hash)
}
pub fn process_block_result(
pub async fn process_block_result(
&self,
block: SignedBeaconBlock<E>,
) -> Result<SignedBeaconBlockHash, BlockError<E>> {
let block_hash: SignedBeaconBlockHash = self.chain.process_block(block)?.into();
self.chain.fork_choice().unwrap();
let block_hash: SignedBeaconBlockHash =
self.chain.process_block(Arc::new(block)).await?.into();
self.chain.recompute_head_at_current_slot().await?;
Ok(block_hash)
}
@@ -1403,14 +1427,14 @@ where
self.chain.slot_clock.set_slot(slot.into());
}
pub fn add_block_at_slot(
pub async fn add_block_at_slot(
&self,
slot: Slot,
state: BeaconState<E>,
) -> Result<(SignedBeaconBlockHash, SignedBeaconBlock<E>, BeaconState<E>), BlockError<E>> {
self.set_current_slot(slot);
let (block, new_state) = self.make_block(state, slot);
let block_hash = self.process_block(slot, block.clone())?;
let (block, new_state) = self.make_block(state, slot).await;
let block_hash = self.process_block(slot, block.clone()).await?;
Ok((block_hash, block, new_state))
}
@@ -1427,19 +1451,19 @@ where
self.process_attestations(attestations);
}
pub fn add_attested_block_at_slot(
pub async fn add_attested_block_at_slot(
&self,
slot: Slot,
state: BeaconState<E>,
state_root: Hash256,
validators: &[usize],
) -> Result<(SignedBeaconBlockHash, BeaconState<E>), BlockError<E>> {
let (block_hash, block, state) = self.add_block_at_slot(slot, state)?;
let (block_hash, block, state) = self.add_block_at_slot(slot, state).await?;
self.attest_block(&state, state_root, block_hash, &block, validators);
Ok((block_hash, state))
}
pub fn add_attested_blocks_at_slots(
pub async fn add_attested_blocks_at_slots(
&self,
state: BeaconState<E>,
state_root: Hash256,
@@ -1448,9 +1472,10 @@ where
) -> AddBlocksResult<E> {
assert!(!slots.is_empty());
self.add_attested_blocks_at_slots_given_lbh(state, state_root, slots, validators, None)
.await
}
fn add_attested_blocks_at_slots_given_lbh(
async fn add_attested_blocks_at_slots_given_lbh(
&self,
mut state: BeaconState<E>,
state_root: Hash256,
@@ -1467,6 +1492,7 @@ where
for slot in slots {
let (block_hash, new_state) = self
.add_attested_block_at_slot(*slot, state, state_root, validators)
.await
.unwrap();
state = new_state;
block_hash_from_slot.insert(*slot, block_hash);
@@ -1488,7 +1514,7 @@ where
/// epoch at a time.
///
/// Chains is a vec of `(state, slots, validators)` tuples.
pub fn add_blocks_on_multiple_chains(
pub async fn add_blocks_on_multiple_chains(
&self,
chains: Vec<(BeaconState<E>, Vec<Slot>, Vec<usize>)>,
) -> Vec<AddBlocksResult<E>> {
@@ -1547,7 +1573,8 @@ where
&epoch_slots,
&validators,
Some(head_block),
);
)
.await;
block_hashes.extend(new_block_hashes);
state_hashes.extend(new_state_hashes);
@@ -1596,18 +1623,18 @@ where
/// Deprecated: Use make_block() instead
///
/// Returns a newly created block, signed by the proposer for the given slot.
pub fn build_block(
pub async fn build_block(
&self,
state: BeaconState<E>,
slot: Slot,
_block_strategy: BlockStrategy,
) -> (SignedBeaconBlock<E>, BeaconState<E>) {
self.make_block(state, slot)
self.make_block(state, slot).await
}
/// Uses `Self::extend_chain` to build the chain out to the `target_slot`.
pub fn extend_to_slot(&self, target_slot: Slot) -> Hash256 {
if self.chain.slot().unwrap() == self.chain.head_info().unwrap().slot {
pub async fn extend_to_slot(&self, target_slot: Slot) -> Hash256 {
if self.chain.slot().unwrap() == self.chain.canonical_head.cached_head().head_slot() {
self.advance_slot();
}
@@ -1618,7 +1645,7 @@ where
.checked_add(1)
.unwrap();
self.extend_slots(num_slots)
self.extend_slots(num_slots).await
}
/// Uses `Self::extend_chain` to `num_slots` blocks.
@@ -1627,8 +1654,8 @@ where
///
/// - BlockStrategy::OnCanonicalHead,
/// - AttestationStrategy::AllValidators,
pub fn extend_slots(&self, num_slots: usize) -> Hash256 {
if self.chain.slot().unwrap() == self.chain.head_info().unwrap().slot {
pub async fn extend_slots(&self, num_slots: usize) -> Hash256 {
if self.chain.slot().unwrap() == self.chain.canonical_head.cached_head().head_slot() {
self.advance_slot();
}
@@ -1637,6 +1664,7 @@ where
BlockStrategy::OnCanonicalHead,
AttestationStrategy::AllValidators,
)
.await
}
/// Deprecated: Use add_attested_blocks_at_slots() instead
@@ -1650,7 +1678,7 @@ where
///
/// The `attestation_strategy` dictates which validators will attest to the newly created
/// blocks.
pub fn extend_chain(
pub async fn extend_chain(
&self,
num_blocks: usize,
block_strategy: BlockStrategy,
@@ -1685,8 +1713,9 @@ where
AttestationStrategy::SomeValidators(vals) => vals,
};
let state_root = state.update_tree_hash_cache().unwrap();
let (_, _, last_produced_block_hash, _) =
self.add_attested_blocks_at_slots(state, state_root, &slots, &validators);
let (_, _, last_produced_block_hash, _) = self
.add_attested_blocks_at_slots(state, state_root, &slots, &validators)
.await;
last_produced_block_hash.into()
}
@@ -1700,41 +1729,40 @@ where
/// then built `faulty_fork_blocks`.
///
/// Returns `(honest_head, faulty_head)`, the roots of the blocks at the top of each chain.
pub fn generate_two_forks_by_skipping_a_block(
pub async fn generate_two_forks_by_skipping_a_block(
&self,
honest_validators: &[usize],
faulty_validators: &[usize],
honest_fork_blocks: usize,
faulty_fork_blocks: usize,
) -> (Hash256, Hash256) {
let initial_head_slot = self
.chain
.head()
.expect("should get head")
.beacon_block
.slot();
let initial_head_slot = self.chain.head_snapshot().beacon_block.slot();
// Move to the next slot so we may produce some more blocks on the head.
self.advance_slot();
// Extend the chain with blocks where only honest validators agree.
let honest_head = self.extend_chain(
honest_fork_blocks,
BlockStrategy::OnCanonicalHead,
AttestationStrategy::SomeValidators(honest_validators.to_vec()),
);
let honest_head = self
.extend_chain(
honest_fork_blocks,
BlockStrategy::OnCanonicalHead,
AttestationStrategy::SomeValidators(honest_validators.to_vec()),
)
.await;
// Go back to the last block where all agreed, and build blocks upon it where only faulty nodes
// agree.
let faulty_head = self.extend_chain(
faulty_fork_blocks,
BlockStrategy::ForkCanonicalChainAt {
previous_slot: initial_head_slot,
// `initial_head_slot + 2` means one slot is skipped.
first_slot: initial_head_slot + 2,
},
AttestationStrategy::SomeValidators(faulty_validators.to_vec()),
);
let faulty_head = self
.extend_chain(
faulty_fork_blocks,
BlockStrategy::ForkCanonicalChainAt {
previous_slot: initial_head_slot,
// `initial_head_slot + 2` means one slot is skipped.
first_slot: initial_head_slot + 2,
},
AttestationStrategy::SomeValidators(faulty_validators.to_vec()),
)
.await;
assert_ne!(honest_head, faulty_head, "forks should be distinct");