Modularise slasher backend (#3443)

## Proposed Changes

Enable multiple database backends for the slasher, either MDBX (default) or LMDB. The backend can be selected using `--slasher-backend={lmdb,mdbx}`.

## Additional Info

In order to abstract over the two library's different handling of database lifetimes I've used `Box::leak` to give the `Environment` type a `'static` lifetime. This was the only way I could think of using 100% safe code to construct a self-referential struct `SlasherDB`, where the `OpenDatabases` refers to the `Environment`. I think this is OK, as the `Environment` is expected to live for the life of the program, and both database engines leave the database in a consistent state after each write. The memory claimed for memory-mapping will be freed by the OS and appropriately flushed regardless of whether the `Environment` is actually dropped.

We are depending on two `sigp` forks of `libmdbx-rs` and `lmdb-rs`, to give us greater control over MDBX OS support and LMDB's version.

book/src/cross-compiling.md

@@ -38,3 +38,9 @@ make build-aarch64
 
 The `lighthouse` binary will be compiled inside a Docker container and placed
 in `lighthouse/target/aarch64-unknown-linux-gnu/release`.
+
+## Feature Flags
+
+When using the makefile the set of features used for building can be controlled with
+the environment variable `CROSS_FEATURES`. See [Feature
+ Flags](./installation-source.md#feature-flags) for available features.

book/src/installation-source.md

@@ -107,6 +107,23 @@ git checkout ${VERSION}
 make
 ```
 
+## Feature Flags
+
+You can customise the features that Lighthouse is built with using the `FEATURES` environment
+variable. E.g.
+
+```
+env FEATURES="gnosis,slasher-lmdb" make
+```
+
+Commonly used features include:
+
+* `gnosis`: support for the Gnosis Beacon Chain.
+* `portable`: support for legacy hardware.
+* `modern`: support for exclusively modern hardware.
+* `slasher-mdbx`: support for the MDBX slasher backend (enabled by default).
+* `slasher-lmdb`: support for the LMDB slasher backend.
+
 ## Troubleshooting
 
 ### Command is not found

book/src/slasher.md

@@ -43,6 +43,34 @@ By default the slasher stores data in the `slasher_db` directory inside the beac
 e.g. `~/.lighthouse/{network}/beacon/slasher_db`. You can use this flag to change that storage
 directory.
 
+### Database Backend
+
+* Flag: `--slasher-backend NAME`
+* Argument: one of `mdbx`, `lmdb` or `disabled`
+* Default: `mdbx`
+
+Since Lighthouse v2.6.0 it is possible to use one of several database backends with the slasher:
+
+- MDBX (default)
+- LMDB
+
+The advantage of MDBX is that it performs compaction, resulting in less disk usage over time. The
+disadvantage is that upstream MDBX has removed support for Windows and macOS, so Lighthouse is stuck
+on an older version. If bugs are found in our pinned version of MDBX it may be deprecated in future.
+
+LMDB does not have compaction but is more stable upstream than MDBX. It is not currently recommended
+to use the LMDB backend on Windows.
+
+More backends may be added in future.
+
+### Switching Backends
+
+If you change database backends and want to reclaim the space used by the old backend you can
+delete the following files from your `slasher_db` directory:
+
+* removing MDBX: delete `mdbx.dat` and `mdbx.lck`
+* removing LMDB: delete `data.mdb` and `lock.mdb`
+
 ### History Length
 
 * Flag: `--slasher-history-length EPOCHS`
@@ -65,7 +93,7 @@ changed after initialization.
 * Argument: maximum size of the database in gigabytes
 * Default: 256 GB
 
-The slasher uses MDBX as its backing store, which places a hard limit on the size of the database
+Both database backends LMDB and MDBX place a hard limit on the size of the database
 file. You can use the `--slasher-max-db-size` flag to set this limit. It can be adjusted after
 initialization if the limit is reached.
 
@@ -85,10 +113,6 @@ where `V` is the validator count and `N` is the history length.
 You should set the maximum size higher than the estimate to allow room for growth in the validator
 count.
 
-> NOTE: In Lighthouse v2.1.0 the slasher database was switched from LMDB to MDBX. Unlike LMDB, MDBX
-> does garbage collection of free pages and is capable of shrinking the database file and preventing
-> it from growing indefinitely.
-
 ### Update Period
 
 * Flag: `--slasher-update-period SECONDS`