Modularise slasher backend (#3443)

## Proposed Changes

Enable multiple database backends for the slasher, either MDBX (default) or LMDB. The backend can be selected using `--slasher-backend={lmdb,mdbx}`.

## Additional Info

In order to abstract over the two library's different handling of database lifetimes I've used `Box::leak` to give the `Environment` type a `'static` lifetime. This was the only way I could think of using 100% safe code to construct a self-referential struct `SlasherDB`, where the `OpenDatabases` refers to the `Environment`. I think this is OK, as the `Environment` is expected to live for the life of the program, and both database engines leave the database in a consistent state after each write. The memory claimed for memory-mapping will be freed by the OS and appropriately flushed regardless of whether the `Environment` is actually dropped.

We are depending on two `sigp` forks of `libmdbx-rs` and `lmdb-rs`, to give us greater control over MDBX OS support and LMDB's version.
This commit is contained in:
Michael Sproul
2022-08-15 01:30:56 +00:00
parent 71fd0b42f2
commit 92d597ad23
25 changed files with 905 additions and 230 deletions

View File

@@ -7,6 +7,7 @@ autotests = false
rust-version = "1.62"
[features]
default = ["slasher-mdbx"]
# Writes debugging .ssz files to /tmp during block processing.
write_ssz_files = ["beacon_node/write_ssz_files"]
# Compiles the BLS crypto code so that the binary is portable across machines.
@@ -19,6 +20,10 @@ milagro = ["bls/milagro"]
spec-minimal = []
# Support Gnosis spec and Gnosis Beacon Chain.
gnosis = []
# Support slasher MDBX backend.
slasher-mdbx = ["slasher/mdbx"]
# Support slasher LMDB backend.
slasher-lmdb = ["slasher/lmdb"]
[dependencies]
beacon_node = { "path" = "../beacon_node" }
@@ -48,6 +53,7 @@ malloc_utils = { path = "../common/malloc_utils" }
directory = { path = "../common/directory" }
unused_port = { path = "../common/unused_port" }
database_manager = { path = "../database_manager" }
slasher = { path = "../slasher" }
[dev-dependencies]
tempfile = "3.1.0"

View File

@@ -1288,6 +1288,32 @@ fn slasher_broadcast_flag() {
assert!(slasher_config.broadcast);
});
}
#[test]
fn slasher_backend_default() {
CommandLineTest::new()
.flag("slasher", None)
.run_with_zero_port()
.with_config(|config| {
let slasher_config = config.slasher.as_ref().unwrap();
assert_eq!(slasher_config.backend, slasher::DatabaseBackend::Mdbx);
});
}
#[test]
fn slasher_backend_override_to_default() {
// Hard to test this flag because all but one backend is disabled by default and the backend
// called "disabled" results in a panic.
CommandLineTest::new()
.flag("slasher", None)
.flag("slasher-backend", Some("mdbx"))
.run_with_zero_port()
.with_config(|config| {
let slasher_config = config.slasher.as_ref().unwrap();
assert_eq!(slasher_config.backend, slasher::DatabaseBackend::Mdbx);
});
}
#[test]
pub fn malloc_tuning_flag() {
CommandLineTest::new()