gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-03 00:31:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Prepare for public testnet (#628)

Browse Source 
* Update to spec v0.9.0

* Update to v0.9.1

* Bump spec tags for v0.9.1

* Formatting, fix CI failures

* Resolve accidental KeyPair merge conflict

* Document new BeaconState functions

* Add `validator` changes from `validator-to-rest`

* Add initial (failing) REST api tests

* Fix signature parsing

* Add more tests

* Refactor http router

* Add working tests for publish beacon block

* Add validator duties tests

* Move account_manager under `lighthouse` binary

* Unify logfile handling in `environment` crate.

* Fix incorrect cache drops in `advance_caches`

* Update fork choice for v0.9.1

* Add `deposit_contract` crate

* Add progress on validator onboarding

* Add unfinished attesation code

* Update account manager CLI

* Write eth1 data file as hex string

* Integrate ValidatorDirectory with validator_client

* Move ValidatorDirectory into validator_client

* Clean up some FIXMEs

* Add beacon_chain_sim

* Fix a few docs/logs

* Expand `beacon_chain_sim`

* Fix spec for `beacon_chain_sim

* More testing for api

* Start work on attestation endpoint

* Reject empty attestations

* Allow attestations to genesis block

* Add working tests for `rest_api` validator endpoint

* Remove grpc from beacon_node

* Start heavy refactor of validator client

- Block production is working

* Prune old validator client files

* Start works on attestation service

* Add attestation service to validator client

* Use full pubkey for validator directories

* Add validator duties post endpoint

* Use par_iter for keypair generation

* Use bulk duties request in validator client

* Add version http endpoint tests

* Add interop keys and startup wait

* Ensure a prompt exit

* Add duties pruning

* Fix compile error in beacon node tests

* Add github workflow

* Modify rust.yaml

* Modify gitlab actions

* Add to CI file

* Add sudo to CI npm install

* Move cargo fmt to own job in tests

* Fix cargo fmt in CI

* Add rustup update before cargo fmt

* Change name of CI job

* Make other CI jobs require cargo fmt

* Add CI badge

* Remove gitlab and travis files

* Add different http timeout for debug

* Update docker file, use makefile in CI

* Use make in the dockerfile, skip the test

* Use the makefile for debug GI test

* Update book

* Tidy grpc and misc things

* Apply discv5 fixes

* Address other minor issues

* Fix warnings

* Attempt fix for addr parsing

* Tidy validator config, CLIs

* Tidy comments

* Tidy signing, reduce ForkService duplication

* Fail if skipping too many slots

* Set default recent genesis time to 0

* Add custom http timeout to validator

* Fix compile bug in node_test_rig

* Remove old bootstrap flag from val CLI

* Update docs

* Tidy val client

* Change val client log levels

* Add comments, more validity checks

* Fix compile error, add comments

* Undo changes to eth2-libp2p/src

* Reduce duplication of keypair generation

* Add more logging for validator duties

* Fix beacon_chain_sim, nitpicks

* Fix compile error, minor nits

* Update to use v0.9.2 version of deposit contract

* Add efforts to automate eth1 testnet deployment

* Fix lcli testnet deployer

* Modify bn CLI to parse eth2_testnet_dir

* Progress with account_manager deposit tools

* Make account manager submit deposits

* Add password option for submitting deposits

* Allow custom deposit amount

* Add long names to lcli clap

* Add password option to lcli deploy command

* Fix minor bugs whilst testing

* Address Michael's comments

* Add refund-deposit-contract to lcli

* Use time instead of skip count for denying long skips

* Improve logging for eth1

* Fix bug with validator services exiting on error

* Drop the block cache after genesis

* Modify eth1 testnet config

* Improve eth1 logging

* Make validator wait until genesis time

* Fix bug in eth1 voting

* Add more logging to eth1 voting

* Handle errors in eth1 http module

* Set SECONDS_PER_DAY to sensible minimum

* Shorten delay before testnet start

* Ensure eth1 block is produced without any votes

* Improve eth1 logging

* Fix broken tests in eth1

* Tidy code in rest_api

* Fix failing test in deposit_contract

* Make CLI args more consistent

* Change validator/duties endpoint

* Add time-based skip slot limiting

* Add new error type missed in previous commit

* Add log when waiting for genesis

* Refactor beacon node CLI

* Remove unused dep

* Add lcli eth1-genesis command

* Fix bug in master merge

* Apply clippy lints to beacon node

* Add support for YamlConfig in Eth2TestnetDir

* Upgrade tesnet deposit contract version

* Remove unnecessary logging and correct formatting

* Add a hardcoded eth2 testnet config

* Ensure http server flag works. Overwrite configs with flags.

* Ensure boot nodes are loaded from testnet dir

* Fix account manager CLI bugs

* Fix bugs with beacon node cli

* Allow testnet dir without boot nodes

* Write genesis state as SSZ

* Remove ---/n from the start of testnet_dir files

* Set default libp2p address

* Tidy account manager CLI, add logging

* Add check to see if testnet dir exists

* Apply reviewers suggestions

* Add HeadTracker struct

* Add fork choice persistence

* Shorten slot time for simulator

* Add the /beacon/heads API endpoint

* Update hardcoded testnet

* Add tests for BeaconChain persistence + fix bugs

* Extend BeaconChain persistence testing

* Ensure chain is finalized b4 persistence tests

* Ensure boot_enr.yaml is include in binary

* Refactor beacon_chain_sim

* Move files about in beacon sim

* Update beacon_chain_sim

* Fix bug with deposit inclusion

* Increase log in genesis service, fix todo

* Tidy sim, fix broken rest_api tests

* Fix more broken tests

* Update testnet

* Fix broken rest api test

* Tidy account manager CLI

* Use tempdir for account manager

* Stop hardcoded testnet dir from creating dir

* Rename Eth2TestnetDir to Eth2TestnetConfig

* Change hardcoded -> hard_coded

* Tidy account manager

* Add log to account manager

* Tidy, ensure head tracker is loaded from disk

* Tidy beacon chain builder

* Tidy eth1_chain

* Adds log support for simulator

* Revert "Adds log support for simulator"

This reverts commit ec77c66a05.

* Adds log support for simulator

* Tidy after self-review

* Change default log level

* Address Michael's delicious PR comments

* Fix off-by-one in tests
This commit is contained in:
Paul Hauner
2019-12-03 15:28:57 +11:00
committed by GitHub
parent 4b4bc6247d
commit a0549e3842
74 changed files with 3421 additions and 1298 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
tests/beacon_chain_sim/Cargo.toml
View File

@@ -10,3 +10,8 @@ edition = "2018"
node_test_rig = { path = "../node_test_rig" }
types = { path = "../../eth2/types" }
validator_client = { path = "../../validator_client" }
parking_lot = "0.9.0"
futures = "0.1.29"
tokio = "0.1.22"
eth1_test_rig = { path = "../eth1_test_rig" }
env_logger = "0.7.1"

136
tests/beacon_chain_sim/src/checks.rs Normal file
View File

@@ -0,0 +1,136 @@
use crate::local_network::LocalNetwork;
use futures::{stream, Future, IntoFuture, Stream};
use std::time::{Duration, Instant};
use tokio::timer::Delay;
use types::{Epoch, EthSpec, Slot, Unsigned};
/// Checks that all of the validators have on-boarded by the start of the second eth1 voting
/// period.
pub fn verify_initial_validator_count<E: EthSpec>(
network: LocalNetwork<E>,
slot_duration: Duration,
initial_validator_count: usize,
) -> impl Future<Item = (), Error = String> {
slot_delay(Slot::new(1), slot_duration)
.and_then(move |()| verify_validator_count(network, initial_validator_count))
}
/// Checks that all of the validators have on-boarded by the start of the second eth1 voting
/// period.
pub fn verify_validator_onboarding<E: EthSpec>(
network: LocalNetwork<E>,
slot_duration: Duration,
expected_validator_count: usize,
) -> impl Future<Item = (), Error = String> {
slot_delay(
Slot::new(E::SlotsPerEth1VotingPeriod::to_u64()),
slot_duration,
)
.and_then(move |()| verify_validator_count(network, expected_validator_count))
}
/// Checks that the chain has made the first possible finalization.
///
/// Intended to be run as soon as chain starts.
pub fn verify_first_finalization<E: EthSpec>(
network: LocalNetwork<E>,
slot_duration: Duration,
) -> impl Future<Item = (), Error = String> {
epoch_delay(Epoch::new(4), slot_duration, E::slots_per_epoch())
.and_then(|()| verify_all_finalized_at(network, Epoch::new(2)))
}
/// Delays for `epochs`, plus half a slot extra.
fn epoch_delay(
epochs: Epoch,
slot_duration: Duration,
slots_per_epoch: u64,
) -> impl Future<Item = (), Error = String> {
let duration = slot_duration * (epochs.as_u64() * slots_per_epoch) as u32 + slot_duration / 2;
Delay::new(Instant::now() + duration).map_err(|e| format!("Epoch delay failed: {:?}", e))
}
/// Delays for `slots`, plus half a slot extra.
fn slot_delay(slots: Slot, slot_duration: Duration) -> impl Future<Item = (), Error = String> {
let duration = slot_duration * slots.as_u64() as u32 + slot_duration / 2;
Delay::new(Instant::now() + duration).map_err(|e| format!("Epoch delay failed: {:?}", e))
}
/// Verifies that all beacon nodes in the given network have a head state that has a finalized
/// epoch of `epoch`.
fn verify_all_finalized_at<E: EthSpec>(
network: LocalNetwork<E>,
epoch: Epoch,
) -> impl Future<Item = (), Error = String> {
network
.remote_nodes()
.into_future()
.and_then(|remote_nodes| {
stream::unfold(remote_nodes.into_iter(), |mut iter| {
iter.next().map(|remote_node| {
remote_node
.http
.beacon()
.get_head()
.map(|head| head.finalized_slot.epoch(E::slots_per_epoch()))
.map(|epoch| (epoch, iter))
.map_err(|e| format!("Get head via http failed: {:?}", e))
})
})
.collect()
})
.and_then(move |epochs| {
if epochs.iter().any(|node_epoch| *node_epoch != epoch) {
Err(format!(
"Nodes are not finalized at epoch {}. Finalized epochs: {:?}",
epoch, epochs
))
} else {
Ok(())
}
})
}
/// Verifies that all beacon nodes in the given `network` have a head state that contains
/// `expected_count` validators.
fn verify_validator_count<E: EthSpec>(
network: LocalNetwork<E>,
expected_count: usize,
) -> impl Future<Item = (), Error = String> {
network
.remote_nodes()
.into_future()
.and_then(|remote_nodes| {
stream::unfold(remote_nodes.into_iter(), |mut iter| {
iter.next().map(|remote_node| {
let beacon = remote_node.http.beacon();
beacon
.get_head()
.map_err(|e| format!("Get head via http failed: {:?}", e))
.and_then(move |head| {
beacon
.get_state_by_root(head.state_root)
.map(|(state, _root)| state)
.map_err(|e| format!("Get state root via http failed: {:?}", e))
})
.map(|state| (state.validators.len(), iter))
})
})
.collect()
})
.and_then(move |validator_counts| {
if validator_counts
.iter()
.any(|count| *count != expected_count)
{
Err(format!(
"Nodes do not all have {} validators in their state. Validator counts: {:?}",
expected_count, validator_counts
))
} else {
Ok(())
}
})
}

152
tests/beacon_chain_sim/src/local_network.rs Normal file
View File

@@ -0,0 +1,152 @@
use futures::{Future, IntoFuture};
use node_test_rig::{
environment::RuntimeContext, ClientConfig, LocalBeaconNode, LocalValidatorClient,
RemoteBeaconNode, ValidatorConfig,
};
use parking_lot::RwLock;
use std::ops::Deref;
use std::sync::Arc;
use types::EthSpec;
/// Helper struct to reduce `Arc` usage.
pub struct Inner<E: EthSpec> {
context: RuntimeContext<E>,
beacon_nodes: RwLock<Vec<LocalBeaconNode<E>>>,
validator_clients: RwLock<Vec<LocalValidatorClient<E>>>,
}
/// Represents a set of interconnected `LocalBeaconNode` and `LocalValidatorClient`.
///
/// Provides functions to allow adding new beacon nodes and validators.
pub struct LocalNetwork<E: EthSpec> {
inner: Arc<Inner<E>>,
}
impl<E: EthSpec> Clone for LocalNetwork<E> {
fn clone(&self) -> Self {
Self {
inner: self.inner.clone(),
}
}
}
impl<E: EthSpec> Deref for LocalNetwork<E> {
type Target = Inner<E>;
fn deref(&self) -> &Self::Target {
self.inner.deref()
}
}
impl<E: EthSpec> LocalNetwork<E> {
/// Creates a new network with a single `BeaconNode`.
pub fn new(
context: RuntimeContext<E>,
beacon_config: ClientConfig,
) -> impl Future<Item = Self, Error = String> {
LocalBeaconNode::production(context.service_context("boot_node".into()), beacon_config).map(
|beacon_node| Self {
inner: Arc::new(Inner {
context,
beacon_nodes: RwLock::new(vec![beacon_node]),
validator_clients: RwLock::new(vec![]),
}),
},
)
}
/// Returns the number of beacon nodes in the network.
///
/// Note: does not count nodes that are external to this `LocalNetwork` that may have connected
/// (e.g., another Lighthouse process on the same machine.)
pub fn beacon_node_count(&self) -> usize {
self.beacon_nodes.read().len()
}
/// Returns the number of validator clients in the network.
///
/// Note: does not count nodes that are external to this `LocalNetwork` that may have connected
/// (e.g., another Lighthouse process on the same machine.)
pub fn validator_client_count(&self) -> usize {
self.validator_clients.read().len()
}
/// Adds a beacon node to the network, connecting to the 0'th beacon node via ENR.
pub fn add_beacon_node(
&self,
mut beacon_config: ClientConfig,
) -> impl Future<Item = (), Error = String> {
let self_1 = self.clone();
self.beacon_nodes
.read()
.first()
.map(|boot_node| {
beacon_config.network.boot_nodes.push(
boot_node
.client
.enr()
.expect("bootnode must have a network"),
);
})
.expect("should have atleast one node");
let index = self.beacon_nodes.read().len();
LocalBeaconNode::production(
self.context.service_context(format!("node_{}", index)),
beacon_config,
)
.map(move |beacon_node| {
self_1.beacon_nodes.write().push(beacon_node);
})
}
/// Adds a validator client to the network, connecting it to the beacon node with index
/// `beacon_node`.
pub fn add_validator_client(
&self,
mut validator_config: ValidatorConfig,
beacon_node: usize,
keypair_indices: Vec<usize>,
) -> impl Future<Item = (), Error = String> {
let index = self.validator_clients.read().len();
let context = self.context.service_context(format!("validator_{}", index));
let self_1 = self.clone();
self.beacon_nodes
.read()
.get(beacon_node)
.map(move |beacon_node| {
let socket_addr = beacon_node
.client
.http_listen_addr()
.expect("Must have http started");
validator_config.http_server =
format!("http://{}:{}", socket_addr.ip(), socket_addr.port());
validator_config
})
.ok_or_else(|| format!("No beacon node for index {}", beacon_node))
.into_future()
.and_then(move |validator_config| {
LocalValidatorClient::production_with_insecure_keypairs(
context,
validator_config,
&keypair_indices,
)
})
.map(move |validator_client| self_1.validator_clients.write().push(validator_client))
}
/// For all beacon nodes in `Self`, return a HTTP client to access each nodes HTTP API.
pub fn remote_nodes(&self) -> Result<Vec<RemoteBeaconNode<E>>, String> {
let beacon_nodes = self.beacon_nodes.read();
beacon_nodes
.iter()
.map(|beacon_node| beacon_node.remote_node())
.collect()
}
}

332
tests/beacon_chain_sim/src/main.rs
View File

@@ -1,131 +1,243 @@
use node_test_rig::{
environment::{Environment, EnvironmentBuilder, RuntimeContext},
testing_client_config, ClientConfig, ClientGenesis, LocalBeaconNode, LocalValidatorClient,
ProductionClient, ValidatorConfig,
};
use std::time::{SystemTime, UNIX_EPOCH};
use types::EthSpec;
//! This crate provides a simluation that creates `n` beacon node and validator clients, each with
//! `v` validators. A deposit contract is deployed at the start of the simulation using a local
//! `ganache-cli` instance (you must have `ganache-cli` installed and avaliable on your path). All
//! beacon nodes independently listen for genesis from the deposit contract, then start operating.
//!
//! As the simulation runs, there are checks made to ensure that all components are running
//! correctly. If any of these checks fail, the simulation will exit immediately.
//!
//! By default, the simulation will end as soon as all checks have finished. It may be configured
//! to run indefinitely by setting `end_after_checks = false`.
//!
//! ## Future works
//!
//! Presently all the beacon nodes and validator clients all log to stdout. Additionally, the
//! simulation uses `println` to communicate some info. It might be nice if the nodes logged to
//! easy-to-find files and stdout only contained info from the simulation.
//!
//! It would also be nice to add a CLI using `clap` so that the variables in `main()` can be
//! changed without a recompile.
pub type BeaconNode<E> = LocalBeaconNode<ProductionClient<E>>;
mod checks;
mod local_network;
use env_logger::{Builder, Env};
use eth1_test_rig::GanacheEth1Instance;
use futures::{future, stream, Future, Stream};
use local_network::LocalNetwork;
use node_test_rig::{
environment::EnvironmentBuilder, testing_client_config, ClientGenesis, ValidatorConfig,
};
use std::time::{Duration, Instant};
use tokio::timer::Interval;
use types::MinimalEthSpec;
pub type E = MinimalEthSpec;
fn main() {
let nodes = 4;
let validators_per_node = 64 / nodes;
// Debugging output for libp2p and external crates.
Builder::from_env(Env::default()).init();
match simulation(nodes, validators_per_node) {
let nodes = 4;
let validators_per_node = 20;
let log_level = "debug";
let speed_up_factor = 4;
let end_after_checks = true;
match async_sim(
nodes,
validators_per_node,
speed_up_factor,
log_level,
end_after_checks,
) {
Ok(()) => println!("Simulation exited successfully"),
Err(e) => println!("Simulation exited with error: {}", e),
Err(e) => {
eprintln!("Simulation exited with error: {}", e);
std::process::exit(1)
}
}
}
fn simulation(num_nodes: usize, validators_per_node: usize) -> Result<(), String> {
if num_nodes < 1 {
return Err("Must have at least one node".into());
}
fn async_sim(
node_count: usize,
validators_per_node: usize,
speed_up_factor: u64,
log_level: &str,
end_after_checks: bool,
) -> Result<(), String> {
let mut env = EnvironmentBuilder::minimal()
.async_logger("debug")?
.async_logger(log_level)?
.multi_threaded_tokio_runtime()?
.build()?;
let mut base_config = testing_client_config();
let eth1_block_time = Duration::from_millis(15_000 / speed_up_factor);
let now = SystemTime::now()
.duration_since(UNIX_EPOCH)
.expect("should get system time")
.as_secs();
base_config.genesis = ClientGenesis::Interop {
genesis_time: now,
validator_count: num_nodes * validators_per_node,
};
let spec = &mut env.eth2_config.spec;
let boot_node =
BeaconNode::production(env.service_context("boot_node".into()), base_config.clone());
spec.milliseconds_per_slot = spec.milliseconds_per_slot / speed_up_factor;
spec.eth1_follow_distance = 16;
spec.seconds_per_day = eth1_block_time.as_secs() * spec.eth1_follow_distance * 2;
spec.min_genesis_time = 0;
spec.min_genesis_active_validator_count = 64;
let mut nodes = (1..num_nodes)
.map(|i| {
let context = env.service_context(format!("node_{}", i));
new_with_bootnode_via_enr(context, &boot_node, base_config.clone())
let slot_duration = Duration::from_millis(spec.milliseconds_per_slot);
let initial_validator_count = spec.min_genesis_active_validator_count as usize;
let total_validator_count = validators_per_node * node_count;
let deposit_amount = env.eth2_config.spec.max_effective_balance;
let context = env.core_context();
let executor = context.executor.clone();
let future = GanacheEth1Instance::new()
/*
* Deploy the deposit contract, spawn tasks to keep creating new blocks and deposit
* validators.
*/
.map(move |ganache_eth1_instance| {
let deposit_contract = ganache_eth1_instance.deposit_contract;
let ganache = ganache_eth1_instance.ganache;
let eth1_endpoint = ganache.endpoint();
let deposit_contract_address = deposit_contract.address();
// Start a timer that produces eth1 blocks on an interval.
executor.spawn(
Interval::new(Instant::now(), eth1_block_time)
.map_err(|_| eprintln!("Eth1 block timer failed"))
.for_each(move |_| ganache.evm_mine().map_err(|_| ()))
.map_err(|_| eprintln!("Eth1 evm_mine failed"))
.map(|_| ()),
);
// Submit deposits to the deposit contract.
executor.spawn(
stream::unfold(0..total_validator_count, move |mut iter| {
iter.next().map(|i| {
println!("Submitting deposit for validator {}...", i);
deposit_contract
.deposit_deterministic_async::<E>(i, deposit_amount)
.map(|_| ((), iter))
})
})
.collect()
.map(|_| ())
.map_err(|e| eprintln!("Error submitting deposit: {}", e)),
);
let mut beacon_config = testing_client_config();
beacon_config.genesis = ClientGenesis::DepositContract;
beacon_config.eth1.endpoint = eth1_endpoint;
beacon_config.eth1.deposit_contract_address = deposit_contract_address;
beacon_config.eth1.deposit_contract_deploy_block = 0;
beacon_config.eth1.lowest_cached_block_number = 0;
beacon_config.eth1.follow_distance = 1;
beacon_config.dummy_eth1_backend = false;
beacon_config.sync_eth1_chain = true;
beacon_config
})
.collect::<Vec<_>>();
let _validators = nodes
.iter()
.enumerate()
.map(|(i, node)| {
let mut context = env.service_context(format!("validator_{}", i));
// Pull the spec from the beacon node's beacon chain, in case there were some changes
// to the spec after the node booted.
context.eth2_config.spec = node
.client
.beacon_chain()
.expect("should have beacon chain")
.spec
.clone();
let context = env.service_context(format!("validator_{}", i));
let indices =
(i * validators_per_node..(i + 1) * validators_per_node).collect::<Vec<_>>();
new_validator_client(
&mut env,
context,
node,
ValidatorConfig::default(),
&indices,
)
/*
* Create a new `LocalNetwork` with one beacon node.
*/
.and_then(move |beacon_config| {
LocalNetwork::new(context, beacon_config.clone())
.map(|network| (network, beacon_config))
})
.collect::<Vec<_>>();
/*
* One by one, add beacon nodes to the network.
*/
.and_then(move |(network, beacon_config)| {
let network_1 = network.clone();
nodes.insert(0, boot_node);
stream::unfold(0..node_count - 1, move |mut iter| {
iter.next().map(|_| {
network_1
.add_beacon_node(beacon_config.clone())
.map(|()| ((), iter))
})
})
.collect()
.map(|_| network)
})
/*
* One by one, add validator clients to the network. Each validator client is attached to
* a single corresponding beacon node.
*/
.and_then(move |network| {
let network_1 = network.clone();
env.block_until_ctrl_c()?;
// Note: presently the validator client future will only resolve once genesis time
// occurs. This is great for this scenario, but likely to change in the future.
//
// If the validator client future behaviour changes, we would need to add a new future
// that delays until genesis. Otherwise, all of the checks that start in the next
// future will start too early.
Ok(())
}
// TODO: this function does not result in nodes connecting to each other. This is a bug due to
// using a 0 port for discovery. Age is fixing it.
fn new_with_bootnode_via_enr<E: EthSpec>(
context: RuntimeContext<E>,
boot_node: &BeaconNode<E>,
base_config: ClientConfig,
) -> BeaconNode<E> {
let mut config = base_config;
config.network.boot_nodes.push(
boot_node
.client
.enr()
.expect("bootnode must have a network"),
);
BeaconNode::production(context, config)
}
// Note: this function will block until the validator can connect to the beaco node. It is
// recommended to ensure that the beacon node is running first.
fn new_validator_client<E: EthSpec>(
env: &mut Environment<E>,
context: RuntimeContext<E>,
beacon_node: &BeaconNode<E>,
base_config: ValidatorConfig,
keypair_indices: &[usize],
) -> LocalValidatorClient<E> {
let mut config = base_config;
let socket_addr = beacon_node
.client
.http_listen_addr()
.expect("Must have http started");
config.http_server = format!("http://{}:{}", socket_addr.ip(), socket_addr.port());
env.runtime()
.block_on(LocalValidatorClient::production_with_insecure_keypairs(
context,
config,
keypair_indices,
))
.expect("should start validator")
stream::unfold(0..node_count, move |mut iter| {
iter.next().map(|i| {
let indices = (i * validators_per_node..(i + 1) * validators_per_node)
.collect::<Vec<_>>();
network_1
.add_validator_client(ValidatorConfig::default(), i, indices)
.map(|()| ((), iter))
})
})
.collect()
.map(|_| network)
})
/*
* Start the processes that will run checks on the network as it runs.
*/
.and_then(move |network| {
// The `final_future` either completes immediately or never completes, depending on the value
// of `end_after_checks`.
let final_future: Box<dyn Future<Item = (), Error = String> + Send> =
if end_after_checks {
Box::new(future::ok(()).map_err(|()| "".to_string()))
} else {
Box::new(future::empty().map_err(|()| "".to_string()))
};
future::ok(())
// Check that the chain finalizes at the first given opportunity.
.join(checks::verify_first_finalization(
network.clone(),
slot_duration,
))
// Check that the chain starts with the expected validator count.
.join(checks::verify_initial_validator_count(
network.clone(),
slot_duration,
initial_validator_count,
))
// Check that validators greater than `spec.min_genesis_active_validator_count` are
// onboarded at the first possible opportunity.
.join(checks::verify_validator_onboarding(
network.clone(),
slot_duration,
total_validator_count,
))
// End now or run forever, depending on the `end_after_checks` flag.
.join(final_future)
.map(|_| network)
})
/*
* End the simulation by dropping the network. This will kill all running beacon nodes and
* validator clients.
*/
.map(|network| {
println!(
"Simulation complete. Finished with {} beacon nodes and {} validator clients",
network.beacon_node_count(),
network.validator_client_count()
);
// Be explicit about dropping the network, as this kills all the nodes. This ensures
// all the checks have adequate time to pass.
drop(network)
});
env.runtime().block_on(future)
}

118
tests/eth1_test_rig/src/lib.rs
View File

@@ -7,17 +7,17 @@
//! some initial issues.
mod ganache;
use deposit_contract::{eth1_tx_data, ABI, BYTECODE, CONTRACT_DEPLOY_GAS, DEPOSIT_GAS};
use futures::{stream, Future, IntoFuture, Stream};
use deposit_contract::{eth1_tx_data, testnet, ABI, BYTECODE, CONTRACT_DEPLOY_GAS, DEPOSIT_GAS};
use futures::{future, stream, Future, IntoFuture, Stream};
use ganache::GanacheInstance;
use std::time::{Duration, Instant};
use tokio::{runtime::Runtime, timer::Delay};
use types::DepositData;
use types::{EthSpec, Hash256, Keypair, Signature};
use types::{test_utils::generate_deterministic_keypair, EthSpec, Hash256, Keypair, Signature};
use web3::contract::{Contract, Options};
use web3::transports::Http;
use web3::types::{Address, TransactionRequest, U256};
use web3::{Transport, Web3};
use web3::Web3;
pub const DEPLOYER_ACCOUNTS_INDEX: usize = 0;
pub const DEPOSIT_ACCOUNTS_INDEX: usize = 0;
@@ -31,7 +31,7 @@ pub struct GanacheEth1Instance {
impl GanacheEth1Instance {
pub fn new() -> impl Future<Item = Self, Error = String> {
GanacheInstance::new().into_future().and_then(|ganache| {
DepositContract::deploy(ganache.web3.clone(), 0).map(|deposit_contract| Self {
DepositContract::deploy(ganache.web3.clone(), 0, None).map(|deposit_contract| Self {
ganache,
deposit_contract,
})
@@ -58,21 +58,52 @@ impl DepositContract {
pub fn deploy(
web3: Web3<Http>,
confirmations: usize,
password: Option<String>,
) -> impl Future<Item = Self, Error = String> {
Self::deploy_bytecode(web3, confirmations, BYTECODE, ABI, password)
}
pub fn deploy_testnet(
web3: Web3<Http>,
confirmations: usize,
password: Option<String>,
) -> impl Future<Item = Self, Error = String> {
Self::deploy_bytecode(
web3,
confirmations,
testnet::BYTECODE,
testnet::ABI,
password,
)
}
fn deploy_bytecode(
web3: Web3<Http>,
confirmations: usize,
bytecode: &[u8],
abi: &[u8],
password: Option<String>,
) -> impl Future<Item = Self, Error = String> {
let web3_1 = web3.clone();
deploy_deposit_contract(web3.clone(), confirmations)
.map_err(|e| {
format!(
"Failed to deploy contract: {}. Is scripts/ganache_tests_node.sh running?.",
e
)
})
.and_then(move |address| {
Contract::from_json(web3_1.eth(), address, ABI)
.map_err(|e| format!("Failed to init contract: {:?}", e))
})
.map(|contract| Self { contract, web3 })
deploy_deposit_contract(
web3.clone(),
confirmations,
bytecode.to_vec(),
abi.to_vec(),
password,
)
.map_err(|e| {
format!(
"Failed to deploy contract: {}. Is scripts/ganache_tests_node.sh running?.",
e
)
})
.and_then(move |address| {
Contract::from_json(web3_1.eth(), address, ABI)
.map_err(|e| format!("Failed to init contract: {:?}", e))
})
.map(|contract| Self { contract, web3 })
}
/// The deposit contract's address in `0x00ab...` format.
@@ -125,6 +156,25 @@ impl DepositContract {
.map_err(|e| format!("Deposit failed: {:?}", e))
}
pub fn deposit_deterministic_async<E: EthSpec>(
&self,
keypair_index: usize,
amount: u64,
) -> impl Future<Item = (), Error = String> {
let keypair = generate_deterministic_keypair(keypair_index);
let mut deposit = DepositData {
pubkey: keypair.pk.into(),
withdrawal_credentials: Hash256::zero(),
amount,
signature: Signature::empty_signature().into(),
};
deposit.signature = deposit.create_signature(&keypair.sk, &E::default_spec());
self.deposit_async(deposit)
}
/// Performs a non-blocking deposit.
pub fn deposit_async(
&self,
@@ -208,11 +258,15 @@ fn from_gwei(gwei: u64) -> U256 {
/// Deploys the deposit contract to the given web3 instance using the account with index
/// `DEPLOYER_ACCOUNTS_INDEX`.
fn deploy_deposit_contract<T: Transport>(
web3: Web3<T>,
fn deploy_deposit_contract(
web3: Web3<Http>,
confirmations: usize,
bytecode: Vec<u8>,
abi: Vec<u8>,
password_opt: Option<String>,
) -> impl Future<Item = Address, Error = String> {
let bytecode = String::from_utf8_lossy(&BYTECODE);
let bytecode = String::from_utf8(bytecode).expect("bytecode must be valid utf8");
let web3_1 = web3.clone();
web3.eth()
.accounts()
@@ -223,8 +277,30 @@ fn deploy_deposit_contract<T: Transport>(
.cloned()
.ok_or_else(|| "Insufficient accounts for deployer".to_string())
})
.and_then(move |from_address| {
let future: Box<dyn Future<Item = Address, Error = String> + Send> =
if let Some(password) = password_opt {
// Unlock for only a single transaction.
let duration = None;
let future = web3_1
.personal()
.unlock_account(from_address, &password, duration)
.then(move |result| match result {
Ok(true) => Ok(from_address),
Ok(false) => Err("Eth1 node refused to unlock account".to_string()),
Err(e) => Err(format!("Eth1 unlock request failed: {:?}", e)),
});
Box::new(future)
} else {
Box::new(future::ok(from_address))
};
future
})
.and_then(move |deploy_address| {
Contract::deploy(web3.eth(), &ABI)
Contract::deploy(web3.eth(), &abi)
.map_err(|e| format!("Unable to build contract deployer: {:?}", e))?
.confirmations(confirmations)
.options(Options {

29
tests/node_test_rig/src/lib.rs
View File

@@ -2,10 +2,9 @@
//!
//! Intended to be used for testing and simulation purposes. Not for production.
use beacon_node::{beacon_chain::BeaconChainTypes, Client, ProductionBeaconNode};
use beacon_node::ProductionBeaconNode;
use environment::RuntimeContext;
use futures::Future;
use remote_beacon_node::RemoteBeaconNode;
use std::path::PathBuf;
use std::time::{SystemTime, UNIX_EPOCH};
use tempdir::TempDir;
@@ -14,22 +13,26 @@ use validator_client::{KeySource, ProductionValidatorClient};
pub use beacon_node::{ClientConfig, ClientGenesis, ProductionClient};
pub use environment;
pub use remote_beacon_node::RemoteBeaconNode;
pub use validator_client::Config as ValidatorConfig;
/// Provids a beacon node that is running in the current process on a given tokio executor (it
/// is _local_ to this process).
///
/// Intended for use in testing and simulation. Not for production.
pub struct LocalBeaconNode<T> {
pub client: T,
pub struct LocalBeaconNode<E: EthSpec> {
pub client: ProductionClient<E>,
pub datadir: TempDir,
}
impl<E: EthSpec> LocalBeaconNode<ProductionClient<E>> {
impl<E: EthSpec> LocalBeaconNode<E> {
/// Starts a new, production beacon node on the tokio runtime in the given `context`.
///
/// The node created is using the same types as the node we use in production.
pub fn production(context: RuntimeContext<E>, mut client_config: ClientConfig) -> Self {
pub fn production(
context: RuntimeContext<E>,
mut client_config: ClientConfig,
) -> impl Future<Item = Self, Error = String> {
// Creates a temporary directory that will be deleted once this `TempDir` is dropped.
let datadir = TempDir::new("lighthouse_node_test_rig")
.expect("should create temp directory for client datadir");
@@ -37,19 +40,17 @@ impl<E: EthSpec> LocalBeaconNode<ProductionClient<E>> {
client_config.data_dir = datadir.path().into();
client_config.network.network_dir = PathBuf::from(datadir.path()).join("network");
let client = ProductionBeaconNode::new(context, client_config)
.wait()
.expect("should build production client")
.into_inner();
LocalBeaconNode { client, datadir }
ProductionBeaconNode::new(context, client_config).map(move |client| Self {
client: client.into_inner(),
datadir,
})
}
}
impl<T: BeaconChainTypes> LocalBeaconNode<Client<T>> {
impl<E: EthSpec> LocalBeaconNode<E> {
/// Returns a `RemoteBeaconNode` that can connect to `self`. Useful for testing the node as if
/// it were external this process.
pub fn remote_node(&self) -> Result<RemoteBeaconNode<T::EthSpec>, String> {
pub fn remote_node(&self) -> Result<RemoteBeaconNode<E>, String> {
let socket_addr = self
.client
.http_listen_addr()