Alternative (to BeaconChainHarness) BeaconChain testing API (#1380)

The PR:

* Adds the ability to generate a crucial test scenario that isn't possible with `BeaconChainHarness` (i.e. two blocks occupying the same slot; previously forks necessitated skipping slots):

![image](https://user-images.githubusercontent.com/165678/88195404-4bce3580-cc40-11ea-8c08-b48d2e1d5959.png)

* New testing API: Instead of repeatedly calling add_block(), you generate a sorted `Vec<Slot>` and leave it up to the framework to generate blocks at those slots.
* Jumping backwards to an earlier epoch is a hard error, so that tests necessarily generate blocks in a epoch-by-epoch manner.
* Configures the test logger so that output is printed on the console in case a test fails.  The logger also plays well with `--nocapture`, contrary to the existing testing framework
* Rewrites existing fork pruning tests to use the new API
* Adds a tests that triggers finalization at a non epoch boundary slot
* Renamed `BeaconChainYoke` to `BeaconChainTestingRig` because the former has been too confusing
* Fixed multiple tests (e.g. `block_production_different_shuffling_long`, `delete_blocks_and_states`, `shuffling_compatible_simple_fork`) that relied on a weird (and accidental) feature of the old `BeaconChainHarness` that attestations aren't produced for epochs earlier than the current one, thus masking potential bugs in test cases.

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
This commit is contained in:
Adam Szkoda
2020-08-26 09:24:55 +00:00
parent 30bb7aecfb
commit d9f4819fe0
17 changed files with 1220 additions and 824 deletions

View File

@@ -1978,7 +1978,7 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
self.head_tracker.clone(),
old_finalized_checkpoint,
new_finalized_checkpoint,
);
)?;
let _ = self.event_handler.register(EventKind::BeaconFinalization {
epoch: new_finalized_checkpoint.epoch,
@@ -2070,10 +2070,11 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
.beacon_block_root;
let mut visited: HashSet<Hash256> = HashSet::new();
let mut finalized_blocks: HashSet<Hash256> = HashSet::new();
let mut justified_blocks: HashSet<Hash256> = HashSet::new();
let genesis_block_hash = Hash256::zero();
writeln!(output, "digraph beacon {{").unwrap();
writeln!(output, "\t_{:?}[label=\"genesis\"];", genesis_block_hash).unwrap();
writeln!(output, "\t_{:?}[label=\"zero\"];", genesis_block_hash).unwrap();
// Canonical head needs to be processed first as otherwise finalized blocks aren't detected
// properly.
@@ -2104,6 +2105,8 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
.unwrap()
.unwrap();
finalized_blocks.insert(state.finalized_checkpoint.root);
justified_blocks.insert(state.current_justified_checkpoint.root);
justified_blocks.insert(state.previous_justified_checkpoint.root);
}
if block_hash == canonical_head_hash {
@@ -2124,6 +2127,15 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
signed_beacon_block.slot()
)
.unwrap();
} else if justified_blocks.contains(&block_hash) {
writeln!(
output,
"\t_{:?}[label=\"{} ({})\" shape=cds];",
block_hash,
block_hash,
signed_beacon_block.slot()
)
.unwrap();
} else {
writeln!(
output,
@@ -2153,6 +2165,11 @@ impl<T: BeaconChainTypes> BeaconChain<T> {
let mut file = std::fs::File::create(file_name).unwrap();
self.dump_as_dot(&mut file);
}
// Should be used in tests only
pub fn set_graffiti(&mut self, graffiti: Graffiti) {
self.graffiti = graffiti;
}
}
impl<T: BeaconChainTypes> Drop for BeaconChain<T> {