gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-04-19 22:08:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

Drop head tracker for summaries DAG (#6744)

Browse Source 
The head tracker is a persisted piece of state that must be kept in sync with the fork-choice. It has been a source of pruning issues in the past, so we want to remove it
- see https://github.com/sigp/lighthouse/issues/1785

When implementing tree-states in the hot DB we have to change the pruning routine (more details below) so we want to do those changes first in isolation.
- see https://github.com/sigp/lighthouse/issues/6580
- If you want to see the full feature of tree-states hot https://github.com/dapplion/lighthouse/pull/39

Closes https://github.com/sigp/lighthouse/issues/1785


  **Current DB migration routine**

- Locate abandoned heads with head tracker
- Use a roots iterator to collect the ancestors of those heads can be pruned
- Delete those abandoned blocks / states
- Migrate the newly finalized chain to the freezer

In summary, it computes what it has to delete and keeps the rest. Then it migrates data to the freezer. If the abandoned forks routine has a bug it can break the freezer migration.

**Proposed migration routine (this PR)**

- Migrate the newly finalized chain to the freezer
- Load all state summaries from disk
- From those, just knowing the head and finalized block compute two sets: (1) descendants of finalized (2) newly finalized chain
- Iterate all summaries, if a summary does not belong to set (1) or (2), delete

This strategy is more sound as it just checks what's there in the hot DB, computes what it has to keep and deletes the rest. Because it does not rely and 3rd pieces of data we can drop the head tracker and pruning checkpoint. Since the DB migration happens **first** now, as long as the computation of the sets to keep is correct we won't have pruning issues.
This commit is contained in:
Lion - dapplion
2025-04-07 01:23:52 -03:00
committed by GitHub
parent b5d40e3db0
commit 70850fe58d
27 changed files with 1110 additions and 983 deletions
Download Patch File Download Diff File Expand all files Collapse all files

44
beacon_node/beacon_chain/src/block_verification.rs
View File

@@ -1453,22 +1453,8 @@ impl<T: BeaconChainTypes> ExecutionPendingBlock<T> {
let catchup_timer = metrics::start_timer(&metrics::BLOCK_PROCESSING_CATCHUP_STATE);
// Stage a batch of operations to be completed atomically if this block is imported
// successfully. If there is a skipped slot, we include the state root of the pre-state,
// which may be an advanced state that was stored in the DB with a `temporary` flag.
let mut state = parent.pre_state;
let mut confirmed_state_roots =
if block.slot() > state.slot() && state.slot() > parent.beacon_block.slot() {
// Advanced pre-state. Delete its temporary flag.
let pre_state_root = state.update_tree_hash_cache()?;
vec![pre_state_root]
} else {
// Pre state is either unadvanced, or should not be stored long-term because there
// is no skipped slot between `parent` and `block`.
vec![]
};
// The block must have a higher slot than its parent.
if block.slot() <= parent.beacon_block.slot() {
return Err(BlockError::BlockIsNotLaterThanParent {
@@ -1515,38 +1501,29 @@ impl<T: BeaconChainTypes> ExecutionPendingBlock<T> {
// processing, but we get early access to it.
let state_root = state.update_tree_hash_cache()?;
// Store the state immediately, marking it as temporary, and staging the deletion
// of its temporary status as part of the larger atomic operation.
// Store the state immediately.
let txn_lock = chain.store.hot_db.begin_rw_transaction();
let state_already_exists =
chain.store.load_hot_state_summary(&state_root)?.is_some();
let state_batch = if state_already_exists {
// If the state exists, it could be temporary or permanent, but in neither case
// should we rewrite it or store a new temporary flag for it. We *will* stage
// the temporary flag for deletion because it's OK to double-delete the flag,
// and we don't mind if another thread gets there first.
// If the state exists, we do not need to re-write it.
vec![]
} else {
vec![
if state.slot() % T::EthSpec::slots_per_epoch() == 0 {
StoreOp::PutState(state_root, &state)
} else {
StoreOp::PutStateSummary(
state_root,
HotStateSummary::new(&state_root, &state)?,
)
},
StoreOp::PutStateTemporaryFlag(state_root),
]
vec![if state.slot() % T::EthSpec::slots_per_epoch() == 0 {
StoreOp::PutState(state_root, &state)
} else {
StoreOp::PutStateSummary(
state_root,
HotStateSummary::new(&state_root, &state)?,
)
}]
};
chain
.store
.do_atomically_with_block_and_blobs_cache(state_batch)?;
drop(txn_lock);
confirmed_state_roots.push(state_root);
state_root
};
@@ -1713,7 +1690,6 @@ impl<T: BeaconChainTypes> ExecutionPendingBlock<T> {
state,
parent_block: parent.beacon_block,
parent_eth1_finalization_data,
confirmed_state_roots,
consensus_context,
},
payload_verification_handle,