Tree states to support per-slot state diffs (#4652)

* Support per slot state diffs

* Store HierarchyConfig on disk. Support storing hdiffs at per slot level.

* Revert HierachyConfig change for testing.

* Add validity check for the hierarchy config when opening the DB.

* Update HDiff tests.

* Fix `get_cold_state` panic when the diff for the slot isn't stored.

* Use slots instead of epochs for storing snapshots in freezer DB.

* Add snapshot buffer to `diff_buffer_cache` instead of loading it from db every time.

* Add `hierarchy-exponents` cli flag to beacon node.

* Add test for `StorageStrategy::ReplayFrom` and ignore a flaky test.

* Drop hierarchy_config in tests for more frequent snapshot and fix an issue where hdiff wasn't stored unless it's a epoch boundary slot.
This commit is contained in:
Jimmy Chen
2023-09-11 10:19:40 +10:00
committed by GitHub
parent e373e9a107
commit 1e4ee7aa5e
8 changed files with 311 additions and 112 deletions

View File

@@ -533,6 +533,21 @@ pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
[default: 8192 (mainnet) or 64 (minimal)]")
.takes_value(true)
)
.arg(
Arg::with_name("hierarchy-exponents")
.long("hierarchy-exponents")
.value_name("EXPONENTS")
.help("Specifies the frequency for storing full state snapshots and hierarchical \
diffs in the freezer DB. Accepts a comma-separated list of ascending \
exponents. Each exponent defines an interval for storing diffs to the layer \
above. The last exponent defines the interval for full snapshots. \
For example, a config of '4,8,12' would store a full snapshot every \
4096 (2^12) slots, first-level diffs every 256 (2^8) slots, and second-level \
diffs every 16 (2^4) slots. \
Cannot be changed after initialization. \
[default: 5,9,11,13,16,18,21]")
.takes_value(true)
)
.arg(
Arg::with_name("epochs-per-migration")
.long("epochs-per-migration")