Refined payload pruning (#3587)

## Proposed Changes

Improve the payload pruning feature in several ways:

- Payload pruning is now entirely optional. It is enabled by default but can be disabled with `--prune-payloads false`. The previous `--prune-payloads-on-startup` flag from #3565 is removed.
- Initial payload pruning on startup now runs in a background thread. This thread will always load the split state, which is a small fraction of its total work (up to ~300ms) and then backtrack from that state. This pruning process ran in 2m5s on one Prater node with good I/O and 16m on a node with slower I/O.
- To work with the optional payload pruning the database function `try_load_full_block` will now attempt to load execution payloads for finalized slots _if_ pruning is currently disabled. This gives users an opt-out for the extensive traffic between the CL and EL for reconstructing payloads.

## Additional Info

If the `prune-payloads` flag is toggled on and off then the on-startup check may not see any payloads to delete and fail to clean them up. In this case the `lighthouse db prune_payloads` command should be used to force a manual sweep of the database.
This commit is contained in:
Michael Sproul
2022-09-19 07:58:49 +00:00
parent f2ac0738d8
commit 507bb9dad4
6 changed files with 85 additions and 39 deletions

View File

@@ -21,8 +21,8 @@ pub struct StoreConfig {
pub compact_on_init: bool,
/// Whether to compact the database during database pruning.
pub compact_on_prune: bool,
/// Whether to try pruning execution payloads on initialization.
pub prune_payloads_on_init: bool,
/// Whether to prune payloads on initialization and finalization.
pub prune_payloads: bool,
}
/// Variant of `StoreConfig` that gets written to disk. Contains immutable configuration params.
@@ -45,7 +45,7 @@ impl Default for StoreConfig {
block_cache_size: DEFAULT_BLOCK_CACHE_SIZE,
compact_on_init: false,
compact_on_prune: true,
prune_payloads_on_init: true,
prune_payloads: true,
}
}
}