De-duplicate attestations in the slasher (#2767)

## Issue Addressed

Closes https://github.com/sigp/lighthouse/issues/2112
Closes https://github.com/sigp/lighthouse/issues/1861

## Proposed Changes

Collect attestations by validator index in the slasher, and use the magic of reference counting to automatically discard redundant attestations. This results in us storing only 1-2% of the attestations observed when subscribed to all subnets, which carries over to a 50-100x reduction in data stored 🎉 

## Additional Info

There's some nuance to the configuration of the `slot-offset`. It has a profound effect on the effictiveness of de-duplication, see the docs added to the book for an explanation: 5442e695e5/book/src/slasher.md (slot-offset)
This commit is contained in:
Michael Sproul
2021-11-08 00:01:09 +00:00
parent fadb8b2b2b
commit df02639b71
13 changed files with 252 additions and 93 deletions

View File

@@ -795,6 +795,25 @@ fn slasher_update_period_flag() {
});
}
#[test]
fn slasher_slot_offset() {
// TODO: check that the offset is actually stored, once the config is un-hacked
// See: https://github.com/sigp/lighthouse/pull/2767#discussion_r741610402
CommandLineTest::new()
.flag("slasher", None)
.flag("slasher-max-db-size", Some("16"))
.flag("slasher-slot-offset", Some("11.25"))
.run();
}
#[test]
#[should_panic]
fn slasher_slot_offset_nan() {
CommandLineTest::new()
.flag("slasher", None)
.flag("slasher-max-db-size", Some("16"))
.flag("slasher-slot-offset", Some("NaN"))
.run();
}
#[test]
fn slasher_history_length_flag() {
CommandLineTest::new()
.flag("slasher", None)