gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-04-17 12:58:31 +00:00
Code Issues Packages Projects Releases Wiki Activity

Separate execution payloads in the DB (#3157)

Browse Source 
## Proposed Changes

Reduce post-merge disk usage by not storing finalized execution payloads in Lighthouse's database.

⚠️ **This is achieved in a backwards-incompatible way for networks that have already merged** ⚠️. Kiln users and shadow fork enjoyers will be unable to downgrade after running the code from this PR. The upgrade migration may take several minutes to run, and can't be aborted after it begins.

The main changes are:

- New column in the database called `ExecPayload`, keyed by beacon block root.
- The `BeaconBlock` column now stores blinded blocks only.
- Lots of places that previously used full blocks now use blinded blocks, e.g. analytics APIs, block replay in the DB, etc.
- On finalization:
    - `prune_abanonded_forks` deletes non-canonical payloads whilst deleting non-canonical blocks.
    - `migrate_db` deletes finalized canonical payloads whilst deleting finalized states.
- Conversions between blinded and full blocks are implemented in a compositional way, duplicating some work from Sean's PR #3134.
- The execution layer has a new `get_payload_by_block_hash` method that reconstructs a payload using the EE's `eth_getBlockByHash` call.
   - I've tested manually that it works on Kiln, using Geth and Nethermind.
   - This isn't necessarily the most efficient method, and new engine APIs are being discussed to improve this: https://github.com/ethereum/execution-apis/pull/146.
   - We're depending on the `ethers` master branch, due to lots of recent changes. We're also using a workaround for https://github.com/gakonst/ethers-rs/issues/1134.
- Payload reconstruction is used in the HTTP API via `BeaconChain::get_block`, which is now `async`. Due to the `async` fn, the `blocking_json` wrapper has been removed.
- Payload reconstruction is used in network RPC to serve blocks-by-{root,range} responses. Here the `async` adjustment is messier, although I think I've managed to come up with a reasonable compromise: the handlers take the `SendOnDrop` by value so that they can drop it on _task completion_ (after the `fn` returns). Still, this is introducing disk reads onto core executor threads, which may have a negative performance impact (thoughts appreciated).

## Additional Info

- [x] For performance it would be great to remove the cloning of full blocks when converting them to blinded blocks to write to disk. I'm going to experiment with a `put_block` API that takes the block by value, breaks it into a blinded block and a payload, stores the blinded block, and then re-assembles the full block for the caller.
- [x] We should measure the latency of blocks-by-root and blocks-by-range responses.
- [x] We should add integration tests that stress the payload reconstruction (basic tests done, issue for more extensive tests: https://github.com/sigp/lighthouse/issues/3159)
- [x] We should (manually) test the schema v9 migration from several prior versions, particularly as blocks have changed on disk and some migrations rely on being able to load blocks.

Co-authored-by: Paul Hauner <paul@paulhauner.com>
This commit is contained in:
Michael Sproul
2022-05-12 00:42:17 +00:00
parent be59fd9af7
commit bcdd960ab1
62 changed files with 2009 additions and 392 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
beacon_node/beacon_chain/tests/attestation_production.rs
View File

@@ -55,11 +55,15 @@ fn produces_attestations() {
Slot::from(num_blocks_produced)
};
let block = chain
let blinded_block = chain
.block_at_slot(block_slot, WhenSlotSkipped::Prev)
.expect("should get block")
.expect("block should not be skipped");
let block_root = block.message().tree_hash_root();
let block_root = blinded_block.message().tree_hash_root();
let block = chain
.store
.make_full_block(&block_root, blinded_block)
.unwrap();
let epoch_boundary_slot = state
.current_epoch()

2
beacon_node/beacon_chain/tests/attestation_verification.rs
View File

@@ -975,7 +975,7 @@ fn attestation_that_skips_epochs() {
let block_slot = harness
.chain
.store
.get_block(&block_root)
.get_blinded_block(&block_root)
.expect("should not error getting block")
.expect("should find attestation block")
.message()

12
beacon_node/beacon_chain/tests/block_verification.rs
View File

@@ -46,6 +46,18 @@ fn get_chain_segment() -> Vec<BeaconSnapshot<E>> {
.chain_dump()
.expect("should dump chain")
.into_iter()
.map(|snapshot| {
let full_block = harness
.chain
.store
.make_full_block(&snapshot.beacon_block_root, snapshot.beacon_block)
.unwrap();
BeaconSnapshot {
beacon_block_root: snapshot.beacon_block_root,
beacon_block: full_block,
beacon_state: snapshot.beacon_state,
}
})
.skip(1)
.collect()
}

47
beacon_node/beacon_chain/tests/payload_invalidation.rs
View File

@@ -71,7 +71,7 @@ impl InvalidPayloadRig {
fn block_hash(&self, block_root: Hash256) -> ExecutionBlockHash {
self.harness
.chain
.get_block(&block_root)
.get_blinded_block(&block_root)
.unwrap()
.unwrap()
.message()
@@ -273,7 +273,12 @@ impl InvalidPayloadRig {
}
assert_eq!(
self.harness.chain.get_block(&block_root).unwrap().unwrap(),
self.harness
.chain
.store
.get_full_block(&block_root)
.unwrap()
.unwrap(),
block,
"block from db must match block imported"
);
@@ -311,7 +316,11 @@ impl InvalidPayloadRig {
assert_eq!(block_in_forkchoice, None);
assert!(
self.harness.chain.get_block(&block_root).unwrap().is_none(),
self.harness
.chain
.get_blinded_block(&block_root)
.unwrap()
.is_none(),
"invalid block cannot be accessed via get_block"
);
} else {
@@ -427,7 +436,7 @@ fn justified_checkpoint_becomes_invalid() {
let parent_root_of_justified = rig
.harness
.chain
.get_block(&justified_checkpoint.root)
.get_blinded_block(&justified_checkpoint.root)
.unwrap()
.unwrap()
.parent_root();
@@ -643,7 +652,13 @@ fn invalidates_all_descendants() {
assert!(rig.execution_status(fork_block_root).is_invalid());
for root in blocks {
let slot = rig.harness.chain.get_block(&root).unwrap().unwrap().slot();
let slot = rig
.harness
.chain
.get_blinded_block(&root)
.unwrap()
.unwrap()
.slot();
// Fork choice doesn't have info about pre-finalization, nothing to check here.
if slot < finalized_slot {
@@ -707,7 +722,13 @@ fn switches_heads() {
assert!(rig.execution_status(fork_block_root).is_optimistic());
for root in blocks {
let slot = rig.harness.chain.get_block(&root).unwrap().unwrap().slot();
let slot = rig
.harness
.chain
.get_blinded_block(&root)
.unwrap()
.unwrap()
.slot();
// Fork choice doesn't have info about pre-finalization, nothing to check here.
if slot < finalized_slot {
@@ -739,9 +760,17 @@ fn invalid_during_processing() {
];
// 0 should be present in the chain.
assert!(rig.harness.chain.get_block(&roots[0]).unwrap().is_some());
assert!(rig
.harness
.chain
.get_blinded_block(&roots[0])
.unwrap()
.is_some());
// 1 should *not* be present in the chain.
assert_eq!(rig.harness.chain.get_block(&roots[1]).unwrap(), None);
assert_eq!(
rig.harness.chain.get_blinded_block(&roots[1]).unwrap(),
None
);
// 2 should be the head.
let head = rig.harness.chain.head_info().unwrap();
assert_eq!(head.block_root, roots[2]);
@@ -760,7 +789,7 @@ fn invalid_after_optimistic_sync() {
];
for root in &roots {
assert!(rig.harness.chain.get_block(root).unwrap().is_some());
assert!(rig.harness.chain.get_blinded_block(root).unwrap().is_some());
}
// 2 should be the head.

47
beacon_node/beacon_chain/tests/store_tests.rs
View File

@@ -313,7 +313,10 @@ fn epoch_boundary_state_attestation_processing() {
for (attestation, subnet_id) in late_attestations.into_iter().flatten() {
// load_epoch_boundary_state is idempotent!
let block_root = attestation.data.beacon_block_root;
let block = store.get_block(&block_root).unwrap().expect("block exists");
let block = store
.get_blinded_block(&block_root)
.unwrap()
.expect("block exists");
let epoch_boundary_state = store
.load_epoch_boundary_state(&block.state_root())
.expect("no error")
@@ -603,7 +606,7 @@ fn delete_blocks_and_states() {
);
let faulty_head_block = store
.get_block(&faulty_head.into())
.get_blinded_block(&faulty_head.into())
.expect("no errors")
.expect("faulty head block exists");
@@ -645,7 +648,7 @@ fn delete_blocks_and_states() {
break;
}
store.delete_block(&block_root).unwrap();
assert_eq!(store.get_block(&block_root).unwrap(), None);
assert_eq!(store.get_blinded_block(&block_root).unwrap(), None);
}
// Deleting frozen states should do nothing
@@ -890,7 +893,12 @@ fn shuffling_compatible_short_fork() {
}
fn get_state_for_block(harness: &TestHarness, block_root: Hash256) -> BeaconState<E> {
let head_block = harness.chain.get_block(&block_root).unwrap().unwrap();
let head_block = harness
.chain
.store
.get_blinded_block(&block_root)
.unwrap()
.unwrap();
harness
.chain
.get_state(&head_block.state_root(), Some(head_block.slot()))
@@ -1695,7 +1703,7 @@ fn check_all_blocks_exist<'a>(
blocks: impl Iterator<Item = &'a SignedBeaconBlockHash>,
) {
for &block_hash in blocks {
let block = harness.chain.get_block(&block_hash.into()).unwrap();
let block = harness.chain.get_blinded_block(&block_hash.into()).unwrap();
assert!(
block.is_some(),
"expected block {:?} to be in DB",
@@ -1742,7 +1750,7 @@ fn check_no_blocks_exist<'a>(
blocks: impl Iterator<Item = &'a SignedBeaconBlockHash>,
) {
for &block_hash in blocks {
let block = harness.chain.get_block(&block_hash.into()).unwrap();
let block = harness.chain.get_blinded_block(&block_hash.into()).unwrap();
assert!(
block.is_none(),
"did not expect block {:?} to be in the DB",
@@ -1988,7 +1996,12 @@ fn weak_subjectivity_sync() {
.unwrap()
.unwrap();
let wss_checkpoint = harness.chain.head_info().unwrap().finalized_checkpoint;
let wss_block = harness.get_block(wss_checkpoint.root.into()).unwrap();
let wss_block = harness
.chain
.store
.get_full_block(&wss_checkpoint.root)
.unwrap()
.unwrap();
let wss_state = full_store
.get_state(&wss_block.state_root(), None)
.unwrap()
@@ -2042,8 +2055,14 @@ fn weak_subjectivity_sync() {
for snapshot in new_blocks {
let block = &snapshot.beacon_block;
let full_block = harness
.chain
.store
.make_full_block(&snapshot.beacon_block_root, block.clone())
.unwrap();
beacon_chain.slot_clock.set_slot(block.slot().as_u64());
beacon_chain.process_block(block.clone()).unwrap();
beacon_chain.process_block(full_block).unwrap();
beacon_chain.fork_choice().unwrap();
// Check that the new block's state can be loaded correctly.
@@ -2085,13 +2104,13 @@ fn weak_subjectivity_sync() {
.map(|s| s.beacon_block.clone())
.collect::<Vec<_>>();
beacon_chain
.import_historical_block_batch(&historical_blocks)
.import_historical_block_batch(historical_blocks.clone())
.unwrap();
assert_eq!(beacon_chain.store.get_oldest_block_slot(), 0);
// Resupplying the blocks should not fail, they can be safely ignored.
beacon_chain
.import_historical_block_batch(&historical_blocks)
.import_historical_block_batch(historical_blocks)
.unwrap();
// The forwards iterator should now match the original chain
@@ -2114,7 +2133,7 @@ fn weak_subjectivity_sync() {
.unwrap()
.map(Result::unwrap)
{
let block = store.get_block(&block_root).unwrap().unwrap();
let block = store.get_blinded_block(&block_root).unwrap().unwrap();
assert_eq!(block.slot(), slot);
}
@@ -2574,7 +2593,7 @@ fn check_iterators(harness: &TestHarness) {
}
fn get_finalized_epoch_boundary_blocks(
dump: &[BeaconSnapshot<MinimalEthSpec>],
dump: &[BeaconSnapshot<MinimalEthSpec, BlindedPayload<MinimalEthSpec>>],
) -> HashSet<SignedBeaconBlockHash> {
dump.iter()
.cloned()
@@ -2582,7 +2601,9 @@ fn get_finalized_epoch_boundary_blocks(
.collect()
}
fn get_blocks(dump: &[BeaconSnapshot<MinimalEthSpec>]) -> HashSet<SignedBeaconBlockHash> {
fn get_blocks(
dump: &[BeaconSnapshot<MinimalEthSpec, BlindedPayload<MinimalEthSpec>>],
) -> HashSet<SignedBeaconBlockHash> {
dump.iter()
.cloned()
.map(|checkpoint| checkpoint.beacon_block_root.into())

12
beacon_node/beacon_chain/tests/tests.rs
View File

@@ -744,7 +744,11 @@ fn block_roots_skip_slot_behaviour() {
"WhenSlotSkipped::Prev should accurately return the prior skipped block"
);
let expected_block = harness.chain.get_block(&skipped_root).unwrap().unwrap();
let expected_block = harness
.chain
.get_blinded_block(&skipped_root)
.unwrap()
.unwrap();
assert_eq!(
harness
@@ -782,7 +786,11 @@ fn block_roots_skip_slot_behaviour() {
"WhenSlotSkipped::None and WhenSlotSkipped::Prev should be equal on non-skipped slot"
);
let expected_block = harness.chain.get_block(&skips_prev).unwrap().unwrap();
let expected_block = harness
.chain
.get_blinded_block(&skips_prev)
.unwrap()
.unwrap();
assert_eq!(
harness