gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-05-07 16:55:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

Modularise slasher backend (#3443)

Browse Source 
## Proposed Changes

Enable multiple database backends for the slasher, either MDBX (default) or LMDB. The backend can be selected using `--slasher-backend={lmdb,mdbx}`.

## Additional Info

In order to abstract over the two library's different handling of database lifetimes I've used `Box::leak` to give the `Environment` type a `'static` lifetime. This was the only way I could think of using 100% safe code to construct a self-referential struct `SlasherDB`, where the `OpenDatabases` refers to the `Environment`. I think this is OK, as the `Environment` is expected to live for the life of the program, and both database engines leave the database in a consistent state after each write. The memory claimed for memory-mapping will be freed by the OS and appropriately flushed regardless of whether the `Environment` is actually dropped.

We are depending on two `sigp` forks of `libmdbx-rs` and `lmdb-rs`, to give us greater control over MDBX OS support and LMDB's version.
This commit is contained in:
Michael Sproul
2022-08-15 01:30:56 +00:00
parent 71fd0b42f2
commit 92d597ad23
25 changed files with 905 additions and 230 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
book/src/slasher.md
View File

@@ -43,6 +43,34 @@ By default the slasher stores data in the `slasher_db` directory inside the beac
e.g. `~/.lighthouse/{network}/beacon/slasher_db`. You can use this flag to change that storage
directory.
### Database Backend
* Flag: `--slasher-backend NAME`
* Argument: one of `mdbx`, `lmdb` or `disabled`
* Default: `mdbx`
Since Lighthouse v2.6.0 it is possible to use one of several database backends with the slasher:
- MDBX (default)
- LMDB
The advantage of MDBX is that it performs compaction, resulting in less disk usage over time. The
disadvantage is that upstream MDBX has removed support for Windows and macOS, so Lighthouse is stuck
on an older version. If bugs are found in our pinned version of MDBX it may be deprecated in future.
LMDB does not have compaction but is more stable upstream than MDBX. It is not currently recommended
to use the LMDB backend on Windows.
More backends may be added in future.
### Switching Backends
If you change database backends and want to reclaim the space used by the old backend you can
delete the following files from your `slasher_db` directory:
* removing MDBX: delete `mdbx.dat` and `mdbx.lck`
* removing LMDB: delete `data.mdb` and `lock.mdb`
### History Length
* Flag: `--slasher-history-length EPOCHS`
@@ -65,7 +93,7 @@ changed after initialization.
* Argument: maximum size of the database in gigabytes
* Default: 256 GB
The slasher uses MDBX as its backing store, which places a hard limit on the size of the database
Both database backends LMDB and MDBX place a hard limit on the size of the database
file. You can use the `--slasher-max-db-size` flag to set this limit. It can be adjusted after
initialization if the limit is reached.
@@ -85,10 +113,6 @@ where `V` is the validator count and `N` is the history length.
You should set the maximum size higher than the estimate to allow room for growth in the validator
count.
> NOTE: In Lighthouse v2.1.0 the slasher database was switched from LMDB to MDBX. Unlike LMDB, MDBX
> does garbage collection of free pages and is capable of shrinking the database file and preventing
> it from growing indefinitely.
### Update Period
* Flag: `--slasher-update-period SECONDS`