gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-03 00:31:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Optimise payload attributes calculation and add SSE (#4027)

Browse Source 
## Issue Addressed

Closes #3896
Closes #3998
Closes #3700

## Proposed Changes

- Optimise the calculation of withdrawals for payload attributes by avoiding state clones, avoiding unnecessary state advances and reading from the snapshot cache if possible.
- Use the execution layer's payload attributes cache to avoid re-calculating payload attributes. I actually implemented a new LRU cache just for withdrawals but it had the exact same key and most of the same data as the existing payload attributes cache, so I deleted it.
- Add a new SSE event that fires when payloadAttributes are calculated. This is useful for block builders, a la https://github.com/ethereum/beacon-APIs/issues/244.
- Add a new CLI flag `--always-prepare-payload` which forces payload attributes to be sent with every fcU regardless of connected proposers. This is intended for use by builders/relays.

For maximum effect, the flags I've been using to run Lighthouse in "payload builder mode" are:

```
--always-prepare-payload \
--prepare-payload-lookahead 12000 \
--suggested-fee-recipient 0x0000000000000000000000000000000000000000
```

The fee recipient is required so Lighthouse has something to pack in the payload attributes (it can be ignored by the builder). The lookahead causes fcU to be sent at the start of every slot rather than at 8s. As usual, fcU will also be sent after each change of head block. I think this combination is sufficient for builders to build on all viable heads. Often there will be two fcU (and two payload attributes) sent for the same slot: one sent at the start of the slot with the head from `n - 1` as the parent, and one sent after the block arrives with `n` as the parent.

Example usage of the new event stream:

```bash
curl -N "http://localhost:5052/eth/v1/events?topics=payload_attributes"
```

## Additional Info

- [x] Tests added by updating the proposer re-org tests. This has the benefit of testing the proposer re-org code paths with withdrawals too, confirming that the new changes don't interact poorly.
- [ ] Benchmarking with `blockdreamer` on devnet-7 showed promising results but I'm yet to do a comparison to `unstable`.


Co-authored-by: Michael Sproul <micsproul@gmail.com>
This commit is contained in:
Michael Sproul
2023-03-05 23:43:30 +00:00
parent 6e15533b54
commit 01556f6f01
11 changed files with 539 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files

94
beacon_node/http_api/tests/interactive_tests.rs
View File

@@ -2,13 +2,15 @@
use crate::common::*;
use beacon_chain::{
chain_config::ReOrgThreshold,
test_utils::{AttestationStrategy, BlockStrategy},
test_utils::{AttestationStrategy, BlockStrategy, SyncCommitteeStrategy},
};
use eth2::types::DepositContractData;
use execution_layer::{ForkchoiceState, PayloadAttributes};
use parking_lot::Mutex;
use slot_clock::SlotClock;
use state_processing::state_advance::complete_state_advance;
use state_processing::{
per_block_processing::get_expected_withdrawals, state_advance::complete_state_advance,
};
use std::collections::HashMap;
use std::sync::Arc;
use std::time::Duration;
@@ -106,13 +108,15 @@ pub struct ReOrgTest {
percent_head_votes: usize,
should_re_org: bool,
misprediction: bool,
/// Whether to expect withdrawals to change on epoch boundaries.
expect_withdrawals_change_on_epoch: bool,
}
impl Default for ReOrgTest {
/// Default config represents a regular easy re-org.
fn default() -> Self {
Self {
head_slot: Slot::new(30),
head_slot: Slot::new(E::slots_per_epoch() - 2),
parent_distance: 1,
head_distance: 1,
re_org_threshold: 20,
@@ -122,6 +126,7 @@ impl Default for ReOrgTest {
percent_head_votes: 0,
should_re_org: true,
misprediction: false,
expect_withdrawals_change_on_epoch: false,
}
}
}
@@ -136,13 +141,40 @@ pub async fn proposer_boost_re_org_zero_weight() {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
pub async fn proposer_boost_re_org_epoch_boundary() {
proposer_boost_re_org_test(ReOrgTest {
head_slot: Slot::new(31),
head_slot: Slot::new(E::slots_per_epoch() - 1),
should_re_org: false,
..Default::default()
})
.await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
pub async fn proposer_boost_re_org_epoch_boundary_skip1() {
// Proposing a block on a boundary after a skip will change the set of expected withdrawals
// sent in the payload attributes.
proposer_boost_re_org_test(ReOrgTest {
head_slot: Slot::new(2 * E::slots_per_epoch() - 2),
head_distance: 2,
should_re_org: false,
expect_withdrawals_change_on_epoch: true,
..Default::default()
})
.await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
pub async fn proposer_boost_re_org_epoch_boundary_skip32() {
// Propose a block at 64 after a whole epoch of skipped slots.
proposer_boost_re_org_test(ReOrgTest {
head_slot: Slot::new(E::slots_per_epoch() - 1),
head_distance: E::slots_per_epoch() + 1,
should_re_org: false,
expect_withdrawals_change_on_epoch: true,
..Default::default()
})
.await;
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
pub async fn proposer_boost_re_org_slot_after_epoch_boundary() {
proposer_boost_re_org_test(ReOrgTest {
@@ -187,7 +219,7 @@ pub async fn proposer_boost_re_org_finality() {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
pub async fn proposer_boost_re_org_parent_distance() {
proposer_boost_re_org_test(ReOrgTest {
head_slot: Slot::new(30),
head_slot: Slot::new(E::slots_per_epoch() - 2),
parent_distance: 2,
should_re_org: false,
..Default::default()
@@ -198,7 +230,7 @@ pub async fn proposer_boost_re_org_parent_distance() {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
pub async fn proposer_boost_re_org_head_distance() {
proposer_boost_re_org_test(ReOrgTest {
head_slot: Slot::new(29),
head_slot: Slot::new(E::slots_per_epoch() - 3),
head_distance: 2,
should_re_org: false,
..Default::default()
@@ -209,7 +241,7 @@ pub async fn proposer_boost_re_org_head_distance() {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
pub async fn proposer_boost_re_org_very_unhealthy() {
proposer_boost_re_org_test(ReOrgTest {
head_slot: Slot::new(31),
head_slot: Slot::new(E::slots_per_epoch() - 1),
parent_distance: 2,
head_distance: 2,
percent_parent_votes: 10,
@@ -225,7 +257,6 @@ pub async fn proposer_boost_re_org_very_unhealthy() {
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
pub async fn proposer_boost_re_org_weight_misprediction() {
proposer_boost_re_org_test(ReOrgTest {
head_slot: Slot::new(30),
percent_empty_votes: 70,
percent_head_votes: 30,
should_re_org: false,
@@ -254,12 +285,13 @@ pub async fn proposer_boost_re_org_test(
percent_head_votes,
should_re_org,
misprediction,
expect_withdrawals_change_on_epoch,
}: ReOrgTest,
) {
assert!(head_slot > 0);
// We require a network with execution enabled so we can check EL message timings.
let mut spec = ForkName::Merge.make_genesis_spec(E::default_spec());
// Test using Capella so that we simulate conditions as similar to mainnet as possible.
let mut spec = ForkName::Capella.make_genesis_spec(E::default_spec());
spec.terminal_total_difficulty = 1.into();
// Ensure there are enough validators to have `attesters_per_slot`.
@@ -323,13 +355,15 @@ pub async fn proposer_boost_re_org_test(
)
.await;
// Create some chain depth.
// Create some chain depth. Sign sync committee signatures so validator balances don't dip
// below 32 ETH and become ineligible for withdrawals.
harness.advance_slot();
harness
.extend_chain(
.extend_chain_with_sync(
num_initial as usize,
BlockStrategy::OnCanonicalHead,
AttestationStrategy::AllValidators,
SyncCommitteeStrategy::AllValidators,
)
.await;
@@ -364,6 +398,16 @@ pub async fn proposer_boost_re_org_test(
let slot_b = slot_a + parent_distance;
let slot_c = slot_b + head_distance;
// We need to transition to at least epoch 2 in order to trigger
// `process_rewards_and_penalties`. This allows us to test withdrawals changes at epoch
// boundaries.
if expect_withdrawals_change_on_epoch {
assert!(
slot_c.epoch(E::slots_per_epoch()) >= 2,
"for withdrawals to change, test must end at an epoch >= 2"
);
}
harness.advance_slot();
let (block_a_root, block_a, state_a) = harness
.add_block_at_slot(slot_a, harness.get_current_state())
@@ -457,6 +501,10 @@ pub async fn proposer_boost_re_org_test(
// Produce block C.
// Advance state_b so we can get the proposer.
assert_eq!(state_b.slot(), slot_b);
let pre_advance_withdrawals = get_expected_withdrawals(&state_b, &harness.chain.spec)
.unwrap()
.to_vec();
complete_state_advance(&mut state_b, None, slot_c, &harness.chain.spec).unwrap();
let proposer_index = state_b
@@ -514,6 +562,28 @@ pub async fn proposer_boost_re_org_test(
.unwrap();
let payload_attribs = first_update.payload_attributes.as_ref().unwrap();
// Check that withdrawals from the payload attributes match those computed from the parent's
// advanced state.
let expected_withdrawals = if should_re_org {
let mut state_a_advanced = state_a.clone();
complete_state_advance(&mut state_a_advanced, None, slot_c, &harness.chain.spec).unwrap();
get_expected_withdrawals(&state_a_advanced, &harness.chain.spec)
} else {
get_expected_withdrawals(&state_b, &harness.chain.spec)
}
.unwrap()
.to_vec();
let payload_attribs_withdrawals = payload_attribs.withdrawals().unwrap();
assert_eq!(expected_withdrawals, *payload_attribs_withdrawals);
assert!(!expected_withdrawals.is_empty());
if should_re_org
|| expect_withdrawals_change_on_epoch
&& slot_c.epoch(E::slots_per_epoch()) != slot_b.epoch(E::slots_per_epoch())
{
assert_ne!(expected_withdrawals, pre_advance_withdrawals);
}
let lookahead = slot_clock
.start_of(slot_c)
.unwrap()