Refined payload pruning (#3587)

## Proposed Changes

Improve the payload pruning feature in several ways:

- Payload pruning is now entirely optional. It is enabled by default but can be disabled with `--prune-payloads false`. The previous `--prune-payloads-on-startup` flag from #3565 is removed.
- Initial payload pruning on startup now runs in a background thread. This thread will always load the split state, which is a small fraction of its total work (up to ~300ms) and then backtrack from that state. This pruning process ran in 2m5s on one Prater node with good I/O and 16m on a node with slower I/O.
- To work with the optional payload pruning the database function `try_load_full_block` will now attempt to load execution payloads for finalized slots _if_ pruning is currently disabled. This gives users an opt-out for the extensive traffic between the CL and EL for reconstructing payloads.

## Additional Info

If the `prune-payloads` flag is toggled on and off then the on-startup check may not see any payloads to delete and fail to clean them up. In this case the `lighthouse db prune_payloads` command should be used to force a manual sweep of the database.
This commit is contained in:
Michael Sproul
2022-09-19 07:58:49 +00:00
parent f2ac0738d8
commit 507bb9dad4
6 changed files with 85 additions and 39 deletions

View File

@@ -266,13 +266,6 @@ where
self.genesis_time = Some(genesis_state.genesis_time());
// Prune finalized execution payloads.
if store.get_config().prune_payloads_on_init {
store
.try_prune_execution_payloads(false)
.map_err(|e| format!("Error pruning execution payloads: {e:?}"))?;
}
self.op_pool = Some(
store
.get_item::<PersistedOperationPool<TEthSpec>>(&OP_POOL_DB_KEY)
@@ -863,6 +856,20 @@ where
beacon_chain.store_migrator.process_reconstruction();
}
// Prune finalized execution payloads in the background.
if beacon_chain.store.get_config().prune_payloads {
let store = beacon_chain.store.clone();
let log = log.clone();
beacon_chain.task_executor.spawn_blocking(
move || {
if let Err(e) = store.try_prune_execution_payloads(false) {
error!(log, "Error pruning payloads in background"; "error" => ?e);
}
},
"prune_payloads_background",
);
}
Ok(beacon_chain)
}
}