gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-03 00:31:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Prepare for public testnet (#628)

Browse Source 
* Update to spec v0.9.0

* Update to v0.9.1

* Bump spec tags for v0.9.1

* Formatting, fix CI failures

* Resolve accidental KeyPair merge conflict

* Document new BeaconState functions

* Add `validator` changes from `validator-to-rest`

* Add initial (failing) REST api tests

* Fix signature parsing

* Add more tests

* Refactor http router

* Add working tests for publish beacon block

* Add validator duties tests

* Move account_manager under `lighthouse` binary

* Unify logfile handling in `environment` crate.

* Fix incorrect cache drops in `advance_caches`

* Update fork choice for v0.9.1

* Add `deposit_contract` crate

* Add progress on validator onboarding

* Add unfinished attesation code

* Update account manager CLI

* Write eth1 data file as hex string

* Integrate ValidatorDirectory with validator_client

* Move ValidatorDirectory into validator_client

* Clean up some FIXMEs

* Add beacon_chain_sim

* Fix a few docs/logs

* Expand `beacon_chain_sim`

* Fix spec for `beacon_chain_sim

* More testing for api

* Start work on attestation endpoint

* Reject empty attestations

* Allow attestations to genesis block

* Add working tests for `rest_api` validator endpoint

* Remove grpc from beacon_node

* Start heavy refactor of validator client

- Block production is working

* Prune old validator client files

* Start works on attestation service

* Add attestation service to validator client

* Use full pubkey for validator directories

* Add validator duties post endpoint

* Use par_iter for keypair generation

* Use bulk duties request in validator client

* Add version http endpoint tests

* Add interop keys and startup wait

* Ensure a prompt exit

* Add duties pruning

* Fix compile error in beacon node tests

* Add github workflow

* Modify rust.yaml

* Modify gitlab actions

* Add to CI file

* Add sudo to CI npm install

* Move cargo fmt to own job in tests

* Fix cargo fmt in CI

* Add rustup update before cargo fmt

* Change name of CI job

* Make other CI jobs require cargo fmt

* Add CI badge

* Remove gitlab and travis files

* Add different http timeout for debug

* Update docker file, use makefile in CI

* Use make in the dockerfile, skip the test

* Use the makefile for debug GI test

* Update book

* Tidy grpc and misc things

* Apply discv5 fixes

* Address other minor issues

* Fix warnings

* Attempt fix for addr parsing

* Tidy validator config, CLIs

* Tidy comments

* Tidy signing, reduce ForkService duplication

* Fail if skipping too many slots

* Set default recent genesis time to 0

* Add custom http timeout to validator

* Fix compile bug in node_test_rig

* Remove old bootstrap flag from val CLI

* Update docs

* Tidy val client

* Change val client log levels

* Add comments, more validity checks

* Fix compile error, add comments

* Undo changes to eth2-libp2p/src

* Reduce duplication of keypair generation

* Add more logging for validator duties

* Fix beacon_chain_sim, nitpicks

* Fix compile error, minor nits

* Update to use v0.9.2 version of deposit contract

* Add efforts to automate eth1 testnet deployment

* Fix lcli testnet deployer

* Modify bn CLI to parse eth2_testnet_dir

* Progress with account_manager deposit tools

* Make account manager submit deposits

* Add password option for submitting deposits

* Allow custom deposit amount

* Add long names to lcli clap

* Add password option to lcli deploy command

* Fix minor bugs whilst testing

* Address Michael's comments

* Add refund-deposit-contract to lcli

* Use time instead of skip count for denying long skips

* Improve logging for eth1

* Fix bug with validator services exiting on error

* Drop the block cache after genesis

* Modify eth1 testnet config

* Improve eth1 logging

* Make validator wait until genesis time

* Fix bug in eth1 voting

* Add more logging to eth1 voting

* Handle errors in eth1 http module

* Set SECONDS_PER_DAY to sensible minimum

* Shorten delay before testnet start

* Ensure eth1 block is produced without any votes

* Improve eth1 logging

* Fix broken tests in eth1

* Tidy code in rest_api

* Fix failing test in deposit_contract

* Make CLI args more consistent

* Change validator/duties endpoint

* Add time-based skip slot limiting

* Add new error type missed in previous commit

* Add log when waiting for genesis

* Refactor beacon node CLI

* Remove unused dep

* Add lcli eth1-genesis command

* Fix bug in master merge

* Apply clippy lints to beacon node

* Add support for YamlConfig in Eth2TestnetDir

* Upgrade tesnet deposit contract version

* Remove unnecessary logging and correct formatting

* Add a hardcoded eth2 testnet config

* Ensure http server flag works. Overwrite configs with flags.

* Ensure boot nodes are loaded from testnet dir

* Fix account manager CLI bugs

* Fix bugs with beacon node cli

* Allow testnet dir without boot nodes

* Write genesis state as SSZ

* Remove ---/n from the start of testnet_dir files

* Set default libp2p address

* Tidy account manager CLI, add logging

* Add check to see if testnet dir exists

* Apply reviewers suggestions

* Add HeadTracker struct

* Add fork choice persistence

* Shorten slot time for simulator

* Add the /beacon/heads API endpoint

* Update hardcoded testnet

* Add tests for BeaconChain persistence + fix bugs

* Extend BeaconChain persistence testing

* Ensure chain is finalized b4 persistence tests

* Ensure boot_enr.yaml is include in binary

* Refactor beacon_chain_sim

* Move files about in beacon sim

* Update beacon_chain_sim

* Fix bug with deposit inclusion

* Increase log in genesis service, fix todo

* Tidy sim, fix broken rest_api tests

* Fix more broken tests

* Update testnet

* Fix broken rest api test

* Tidy account manager CLI

* Use tempdir for account manager

* Stop hardcoded testnet dir from creating dir

* Rename Eth2TestnetDir to Eth2TestnetConfig

* Change hardcoded -> hard_coded

* Tidy account manager

* Add log to account manager

* Tidy, ensure head tracker is loaded from disk

* Tidy beacon chain builder

* Tidy eth1_chain

* Adds log support for simulator

* Revert "Adds log support for simulator"

This reverts commit ec77c66a05.

* Adds log support for simulator

* Tidy after self-review

* Change default log level

* Address Michael's delicious PR comments

* Fix off-by-one in tests
This commit is contained in:
Paul Hauner
2019-12-03 15:28:57 +11:00
committed by GitHub
parent 4b4bc6247d
commit a0549e3842
74 changed files with 3421 additions and 1298 deletions
Download Patch File Download Diff File Expand all files Collapse all files

846
beacon_node/src/config.rs
View File

@@ -1,18 +1,21 @@
use clap::ArgMatches;
use client::{ClientConfig, ClientGenesis, Eth2Config};
use eth2_config::{read_from_file, write_to_file};
use eth2_libp2p::{Enr, Multiaddr};
use eth2_testnet_config::Eth2TestnetConfig;
use genesis::recent_genesis_time;
use lighthouse_bootstrap::Bootstrapper;
use rand::{distributions::Alphanumeric, Rng};
use slog::{crit, info, warn, Logger};
use slog::{crit, info, Logger};
use ssz::Encode;
use std::fs;
use std::net::Ipv4Addr;
use std::path::{Path, PathBuf};
use types::{Address, Epoch, Fork};
use std::path::PathBuf;
use types::{Epoch, EthSpec, Fork};
pub const DEFAULT_DATA_DIR: &str = ".lighthouse";
pub const CLIENT_CONFIG_FILENAME: &str = "beacon-node.toml";
pub const ETH2_CONFIG_FILENAME: &str = "eth2-spec.toml";
pub const BEACON_NODE_DIR: &str = "beacon";
pub const NETWORK_DIR: &str = "network";
type Result<T> = std::result::Result<T, String>;
type Config = (ClientConfig, Eth2Config, Logger);
@@ -24,89 +27,391 @@ type Config = (ClientConfig, Eth2Config, Logger);
/// The output of this function depends primarily upon the given `cli_args`, however it's behaviour
/// may be influenced by other external services like the contents of the file system or the
/// response of some remote server.
pub fn get_configs(cli_args: &ArgMatches, core_log: Logger) -> Result<Config> {
pub fn get_configs<E: EthSpec>(
cli_args: &ArgMatches,
mut eth2_config: Eth2Config,
core_log: Logger,
) -> Result<Config> {
let log = core_log.clone();
let mut builder = ConfigBuilder::new(cli_args, core_log)?;
let mut client_config = ClientConfig::default();
// Read the `--datadir` flag.
//
// If it's not present, try and find the home directory (`~`) and push the default data
// directory onto it.
client_config.data_dir = cli_args
.value_of("datadir")
.map(PathBuf::from)
.or_else(|| dirs::home_dir().map(|home| home.join(".lighthouse").join(BEACON_NODE_DIR)))
.unwrap_or_else(|| PathBuf::from("."));
// Load the client config, if it exists .
let path = client_config.data_dir.join(CLIENT_CONFIG_FILENAME);
if path.exists() {
client_config = read_from_file(path.clone())
.map_err(|e| format!("Unable to parse {:?} file: {:?}", path, e))?
.ok_or_else(|| format!("{:?} file does not exist", path))?;
}
// Load the eth2 config, if it exists .
let path = client_config.data_dir.join(ETH2_CONFIG_FILENAME);
if path.exists() {
eth2_config = read_from_file(path.clone())
.map_err(|e| format!("Unable to parse {:?} file: {:?}", path, e))?
.ok_or_else(|| format!("{:?} file does not exist", path))?;
}
// Read the `--testnet-dir` flag.
if let Some(val) = cli_args.value_of("testnet-dir") {
client_config.testnet_dir = Some(PathBuf::from(val));
}
/*
* Networking
*/
// If a network dir has been specified, override the `datadir` definition.
if let Some(dir) = cli_args.value_of("network-dir") {
client_config.network.network_dir = PathBuf::from(dir);
} else {
client_config.network.network_dir = client_config.data_dir.join(NETWORK_DIR);
};
if let Some(listen_address_str) = cli_args.value_of("listen-address") {
let listen_address = listen_address_str
.parse()
.map_err(|_| format!("Invalid listen address: {:?}", listen_address_str))?;
client_config.network.listen_address = listen_address;
client_config.network.discovery_address = listen_address;
}
if let Some(max_peers_str) = cli_args.value_of("maxpeers") {
client_config.network.max_peers = max_peers_str
.parse::<usize>()
.map_err(|_| format!("Invalid number of max peers: {}", max_peers_str))?;
}
if let Some(port_str) = cli_args.value_of("port") {
let port = port_str
.parse::<u16>()
.map_err(|_| format!("Invalid port: {}", port_str))?;
client_config.network.libp2p_port = port;
client_config.network.discovery_port = port;
}
if let Some(boot_enr_str) = cli_args.value_of("boot-nodes") {
client_config.network.boot_nodes = boot_enr_str
.split(',')
.map(|enr| enr.parse().map_err(|_| format!("Invalid ENR: {}", enr)))
.collect::<Result<Vec<Enr>>>()?;
}
if let Some(libp2p_addresses_str) = cli_args.value_of("libp2p-addresses") {
client_config.network.libp2p_nodes = libp2p_addresses_str
.split(',')
.map(|multiaddr| {
multiaddr
.parse()
.map_err(|_| format!("Invalid Multiaddr: {}", multiaddr))
})
.collect::<Result<Vec<Multiaddr>>>()?;
}
if let Some(topics_str) = cli_args.value_of("topics") {
client_config.network.topics = topics_str.split(',').map(|s| s.into()).collect();
}
if let Some(discovery_address_str) = cli_args.value_of("discovery-address") {
client_config.network.discovery_address = discovery_address_str
.parse()
.map_err(|_| format!("Invalid discovery address: {:?}", discovery_address_str))?
}
if let Some(disc_port_str) = cli_args.value_of("disc-port") {
client_config.network.discovery_port = disc_port_str
.parse::<u16>()
.map_err(|_| format!("Invalid discovery port: {}", disc_port_str))?;
}
if let Some(p2p_priv_key) = cli_args.value_of("p2p-priv-key") {
client_config.network.secret_key_hex = Some(p2p_priv_key.to_string());
}
/*
* Http server
*/
if cli_args.is_present("http") {
client_config.rest_api.enabled = true;
}
if let Some(address) = cli_args.value_of("http-address") {
client_config.rest_api.listen_address = address
.parse::<Ipv4Addr>()
.map_err(|_| "http-address is not a valid IPv4 address.")?;
}
if let Some(port) = cli_args.value_of("http-port") {
client_config.rest_api.port = port
.parse::<u16>()
.map_err(|_| "http-port is not a valid u16.")?;
}
/*
* Websocket server
*/
if cli_args.is_present("ws") {
client_config.websocket_server.enabled = true;
}
if let Some(address) = cli_args.value_of("ws-address") {
client_config.websocket_server.listen_address = address
.parse::<Ipv4Addr>()
.map_err(|_| "ws-address is not a valid IPv4 address.")?;
}
if let Some(port) = cli_args.value_of("ws-port") {
client_config.websocket_server.port = port
.parse::<u16>()
.map_err(|_| "ws-port is not a valid u16.")?;
}
/*
* Eth1
*/
// When present, use an eth1 backend that generates deterministic junk.
//
// Useful for running testnets without the overhead of a deposit contract.
if cli_args.is_present("dummy-eth1") {
builder.client_config.dummy_eth1_backend = true;
client_config.dummy_eth1_backend = true;
}
// When present, attempt to sync to an eth1 node.
//
// Required for block production.
if cli_args.is_present("eth1") {
client_config.sync_eth1_chain = true;
}
// Defines the URL to reach the eth1 node.
if let Some(val) = cli_args.value_of("eth1-endpoint") {
builder.set_eth1_endpoint(val)
}
if let Some(val) = cli_args.value_of("deposit-contract") {
builder.set_deposit_contract(
val.parse::<Address>()
.map_err(|e| format!("Unable to parse deposit-contract address: {:?}", e))?,
)
}
if let Some(val) = cli_args.value_of("deposit-contract-deploy") {
builder.set_deposit_contract_deploy_block(
val.parse::<u64>()
.map_err(|e| format!("Unable to parse deposit-contract-deploy: {:?}", e))?,
)
}
if let Some(val) = cli_args.value_of("eth1-follow") {
builder.set_eth1_follow(
val.parse::<u64>()
.map_err(|e| format!("Unable to parse follow distance: {:?}", e))?,
)
client_config.eth1.endpoint = val.to_string();
}
match cli_args.subcommand() {
("testnet", Some(sub_cmd_args)) => {
process_testnet_subcommand(&mut builder, sub_cmd_args, &log)?
process_testnet_subcommand(&mut client_config, &mut eth2_config, sub_cmd_args)?
}
// No sub-command assumes a resume operation.
_ => {
info!(
log,
"Resuming from existing datadir";
"path" => format!("{:?}", builder.client_config.data_dir)
);
// If no primary subcommand was given, start the beacon chain from an existing
// database.
builder.set_genesis(ClientGenesis::Resume);
client_config.genesis = ClientGenesis::Resume;
// Whilst there is no large testnet or mainnet force the user to specify how they want
// to start a new chain (e.g., from a genesis YAML file, another node, etc).
if !builder.client_config.data_dir.exists() {
return Err(
"No datadir found. To start a new beacon chain, see `testnet --help`. \
Use `--datadir` to specify a different directory"
.into(),
if !client_config.data_dir.exists() {
info!(
log,
"Starting from an empty database";
"data_dir" => format!("{:?}", client_config.data_dir)
);
init_new_client::<E>(&mut client_config, &mut eth2_config)?
} else {
info!(
log,
"Resuming from existing datadir";
"data_dir" => format!("{:?}", client_config.data_dir)
);
// If the `testnet` command was not provided, attempt to load an existing datadir and
// continue with an existing chain.
load_from_datadir(&mut client_config)?
}
// If the `testnet` command was not provided, attempt to load an existing datadir and
// continue with an existing chain.
builder.load_from_datadir()?;
}
};
builder.build(cli_args)
if let Some(freezer_dir) = cli_args.value_of("freezer-dir") {
client_config.freezer_db_path = Some(PathBuf::from(freezer_dir));
}
if eth2_config.spec_constants != client_config.spec_constants {
crit!(log, "Specification constants do not match.";
"client_config" => client_config.spec_constants.to_string(),
"eth2_config" => eth2_config.spec_constants.to_string()
);
return Err("Specification constant mismatch".into());
}
/*
* Zero-ports
*
* Replaces previously set flags.
*/
if cli_args.is_present("zero-ports") {
client_config.network.libp2p_port = 0;
client_config.network.discovery_port = 0;
client_config.rest_api.port = 0;
client_config.websocket_server.port = 0;
}
Ok((client_config, eth2_config, log))
}
/// Load from an existing database.
fn load_from_datadir(client_config: &mut ClientConfig) -> Result<()> {
// Check to ensure the datadir exists.
//
// For now we return an error. In the future we may decide to boot a default (e.g.,
// public testnet or mainnet).
if !client_config.get_data_dir().map_or(false, |d| d.exists()) {
return Err(
"No datadir found. Either create a new testnet or specify a different `--datadir`."
.into(),
);
}
// If there is a path to a database in the config, ensure it exists.
if !client_config
.get_db_path()
.map_or(false, |path| path.exists())
{
return Err(
"No database found in datadir. Use 'testnet -f' to overwrite the existing \
datadir, or specify a different `--datadir`."
.into(),
);
}
client_config.genesis = ClientGenesis::Resume;
Ok(())
}
/// Create a new client with the default configuration.
fn init_new_client<E: EthSpec>(
client_config: &mut ClientConfig,
eth2_config: &mut Eth2Config,
) -> Result<()> {
let eth2_testnet_config: Eth2TestnetConfig<E> =
if let Some(testnet_dir) = &client_config.testnet_dir {
Eth2TestnetConfig::load(testnet_dir.clone())
.map_err(|e| format!("Unable to open testnet dir at {:?}: {}", testnet_dir, e))?
} else {
Eth2TestnetConfig::hard_coded()
.map_err(|e| format!("Unable to load hard-coded testnet dir: {}", e))?
};
eth2_config.spec = eth2_testnet_config
.yaml_config
.as_ref()
.ok_or_else(|| "The testnet directory must contain a spec config".to_string())?
.apply_to_chain_spec::<E>(&eth2_config.spec)
.ok_or_else(|| {
format!(
"The loaded config is not compatible with the {} spec",
&eth2_config.spec_constants
)
})?;
let spec = &mut eth2_config.spec;
client_config.eth1.deposit_contract_address =
format!("{:?}", eth2_testnet_config.deposit_contract_address()?);
client_config.eth1.deposit_contract_deploy_block =
eth2_testnet_config.deposit_contract_deploy_block;
client_config.eth1.follow_distance = spec.eth1_follow_distance / 2;
client_config.dummy_eth1_backend = false;
client_config.eth1.lowest_cached_block_number = client_config
.eth1
.deposit_contract_deploy_block
.saturating_sub(client_config.eth1.follow_distance * 2);
if let Some(boot_nodes) = eth2_testnet_config.boot_enr {
client_config
.network
.boot_nodes
.append(&mut boot_nodes.clone())
}
if let Some(genesis_state) = eth2_testnet_config.genesis_state {
// Note: re-serializing the genesis state is not so efficient, however it avoids adding
// trait bounds to the `ClientGenesis` enum. This would have significant flow-on
// effects.
client_config.genesis = ClientGenesis::SszBytes {
genesis_state_bytes: genesis_state.as_ssz_bytes(),
};
} else {
client_config.genesis = ClientGenesis::DepositContract;
}
create_new_datadir(&client_config, &eth2_config)?;
Ok(())
}
/// Writes the configs in `self` to `self.data_dir`.
///
/// Returns an error if `self.data_dir` already exists.
pub fn create_new_datadir(client_config: &ClientConfig, eth2_config: &Eth2Config) -> Result<()> {
if client_config.data_dir.exists() {
return Err(format!(
"Data dir already exists at {:?}",
client_config.data_dir
))?;
}
// Create `datadir` and any non-existing parent directories.
fs::create_dir_all(&client_config.data_dir)
.map_err(|e| format!("Failed to create data dir: {}", e))?;
macro_rules! write_to_file {
($file: ident, $variable: ident) => {
let file = client_config.data_dir.join($file);
if file.exists() {
return Err(format!("Datadir is not clean, {} exists.", $file));
} else {
// Write the onfig to a TOML file in the datadir.
write_to_file(client_config.data_dir.join($file), $variable)
.map_err(|e| format!("Unable to write {} file: {:?}", $file, e))?;
}
};
}
write_to_file!(CLIENT_CONFIG_FILENAME, client_config);
write_to_file!(ETH2_CONFIG_FILENAME, eth2_config);
Ok(())
}
/// Process the `testnet` CLI subcommand arguments, updating the `builder`.
fn process_testnet_subcommand(
builder: &mut ConfigBuilder,
client_config: &mut ClientConfig,
eth2_config: &mut Eth2Config,
cli_args: &ArgMatches,
log: &Logger,
) -> Result<()> {
// Specifies that a random datadir should be used.
if cli_args.is_present("random-datadir") {
builder.set_random_datadir()?;
client_config
.data_dir
.push(format!("random_{}", random_string(6)));
client_config.network.network_dir = client_config.data_dir.join("network");
}
// Deletes the existing datadir.
if cli_args.is_present("force") {
builder.clean_datadir()?;
if client_config.data_dir.exists() {
fs::remove_dir_all(&client_config.data_dir)
.map_err(|e| format!("Unable to delete existing datadir: {:?}", e))?;
}
}
// Define a percentage of messages that should be propogated, useful for simulating bad network
// conditions.
//
// WARNING: setting this to anything less than 100 will cause bad behaviour.
if let Some(propagation_percentage_string) = cli_args.value_of("random-propagation") {
let percentage = propagation_percentage_string
.parse::<u8>()
@@ -114,72 +419,20 @@ fn process_testnet_subcommand(
if percentage > 100 {
return Err("Propagation percentage greater than 100".to_string());
}
builder.client_config.network.propagation_percentage = Some(percentage);
}
let is_bootstrap = cli_args.subcommand_name() == Some("bootstrap");
if let Some(path_string) = cli_args.value_of("eth2-config") {
if is_bootstrap {
return Err("Cannot supply --eth2-config when using bootstrap".to_string());
}
let path = path_string
.parse::<PathBuf>()
.map_err(|e| format!("Unable to parse eth2-config path: {:?}", e))?;
builder.load_eth2_config(path)?;
} else {
builder.update_spec_from_subcommand(&cli_args)?;
client_config.network.propagation_percentage = Some(percentage);
}
// Modify the `SECONDS_PER_SLOT` "constant".
if let Some(slot_time) = cli_args.value_of("slot-time") {
if is_bootstrap {
return Err("Cannot supply --slot-time flag whilst using bootstrap.".into());
}
let slot_time = slot_time
.parse::<u64>()
.map_err(|e| format!("Unable to parse slot-time: {:?}", e))?;
builder.set_slot_time(slot_time);
eth2_config.spec.milliseconds_per_slot = slot_time;
}
if let Some(path_string) = cli_args.value_of("client-config") {
let path = path_string
.parse::<PathBuf>()
.map_err(|e| format!("Unable to parse client config path: {:?}", e))?;
builder.load_client_config(path)?;
}
info!(
log,
"Creating new datadir";
"path" => format!("{:?}", builder.client_config.data_dir)
);
// When using the testnet command we listen on all addresses.
builder.set_listen_addresses("0.0.0.0".into())?;
warn!(log, "All services listening on 0.0.0.0");
// Start matching on the second subcommand (e.g., `testnet bootstrap ...`).
match cli_args.subcommand() {
("bootstrap", Some(cli_args)) => {
let server = cli_args
.value_of("server")
.ok_or_else(|| "No bootstrap server specified")?;
let port: Option<u16> = cli_args
.value_of("libp2p-port")
.and_then(|s| s.parse::<u16>().ok());
builder.import_bootstrap_libp2p_address(server, port)?;
builder.import_bootstrap_enr_address(server)?;
builder.import_bootstrap_eth2_config(server)?;
builder.set_genesis(ClientGenesis::RemoteNode {
server: server.to_string(),
port,
})
}
("recent", Some(cli_args)) => {
let validator_count = cli_args
.value_of("validator_count")
@@ -193,12 +446,12 @@ fn process_testnet_subcommand(
.parse::<u64>()
.map_err(|e| format!("Unable to parse minutes: {:?}", e))?;
builder.client_config.dummy_eth1_backend = true;
client_config.dummy_eth1_backend = true;
builder.set_genesis(ClientGenesis::Interop {
client_config.genesis = ClientGenesis::Interop {
validator_count,
genesis_time: recent_genesis_time(minutes),
})
};
}
("quick", Some(cli_args)) => {
let validator_count = cli_args
@@ -213,12 +466,12 @@ fn process_testnet_subcommand(
.parse::<u64>()
.map_err(|e| format!("Unable to parse genesis time: {:?}", e))?;
builder.client_config.dummy_eth1_backend = true;
client_config.dummy_eth1_backend = true;
builder.set_genesis(ClientGenesis::Interop {
client_config.genesis = ClientGenesis::Interop {
validator_count,
genesis_time,
})
};
}
("file", Some(cli_args)) => {
let path = cli_args
@@ -236,11 +489,10 @@ fn process_testnet_subcommand(
other => return Err(format!("Unknown genesis file format: {}", other)),
};
builder.set_genesis(start_method)
client_config.genesis = start_method;
}
("prysm", Some(_)) => {
let mut spec = &mut builder.eth2_config.spec;
let mut client_config = &mut builder.client_config;
let mut spec = &mut eth2_config.spec;
spec.min_deposit_amount = 100;
spec.max_effective_balance = 3_200_000_000;
@@ -259,7 +511,7 @@ fn process_testnet_subcommand(
client_config.eth1.follow_distance = 16;
client_config.dummy_eth1_backend = false;
builder.set_genesis(ClientGenesis::DepositContract)
client_config.genesis = ClientGenesis::DepositContract;
}
(cmd, Some(_)) => {
return Err(format!(
@@ -270,369 +522,11 @@ fn process_testnet_subcommand(
_ => return Err("No testnet method specified. See 'testnet --help'.".into()),
};
builder.write_configs_to_new_datadir()?;
create_new_datadir(&client_config, &eth2_config)?;
Ok(())
}
/// Allows for building a set of configurations based upon `clap` arguments.
struct ConfigBuilder {
log: Logger,
pub eth2_config: Eth2Config,
pub client_config: ClientConfig,
}
impl ConfigBuilder {
/// Create a new builder with default settings.
pub fn new(cli_args: &ArgMatches, log: Logger) -> Result<Self> {
// Read the `--datadir` flag.
//
// If it's not present, try and find the home directory (`~`) and push the default data
// directory onto it.
let data_dir: PathBuf = cli_args
.value_of("datadir")
.map(PathBuf::from)
.or_else(|| {
dirs::home_dir().map(|mut home| {
home.push(DEFAULT_DATA_DIR);
home
})
})
.ok_or_else(|| "Unable to find a home directory for the datadir".to_string())?;
let mut client_config = ClientConfig::default();
client_config.data_dir = data_dir;
Ok(Self {
log,
eth2_config: Eth2Config::minimal(),
client_config,
})
}
/// Clears any configuration files that would interfere with writing new configs.
///
/// Moves the following files in `data_dir` into a backup directory:
///
/// - Client config
/// - Eth2 config
/// - All database directories
pub fn clean_datadir(&mut self) -> Result<()> {
let backup_dir = {
let mut s = String::from("backup_");
s.push_str(&random_string(6));
self.client_config.data_dir.join(s)
};
fs::create_dir_all(&backup_dir)
.map_err(|e| format!("Unable to create config backup dir: {:?}", e))?;
let move_to_backup_dir = |path: &Path| -> Result<()> {
let file_name = path
.file_name()
.ok_or_else(|| "Invalid path found during datadir clean (no filename).")?;
let mut new = path.to_path_buf();
new.pop();
new.push(backup_dir.clone());
new.push(file_name);
let _ = fs::rename(path, new);
Ok(())
};
move_to_backup_dir(&self.client_config.data_dir.join(CLIENT_CONFIG_FILENAME))?;
move_to_backup_dir(&self.client_config.data_dir.join(ETH2_CONFIG_FILENAME))?;
move_to_backup_dir(&self.client_config.create_db_path()?)?;
move_to_backup_dir(&self.client_config.create_freezer_db_path()?)?;
Ok(())
}
pub fn set_eth1_endpoint(&mut self, endpoint: &str) {
self.client_config.eth1.endpoint = endpoint.to_string();
}
pub fn set_deposit_contract(&mut self, deposit_contract: Address) {
self.client_config.eth1.deposit_contract_address = format!("{:?}", deposit_contract);
}
pub fn set_deposit_contract_deploy_block(&mut self, eth1_block_number: u64) {
self.client_config.eth1.deposit_contract_deploy_block = eth1_block_number;
}
pub fn set_eth1_follow(&mut self, distance: u64) {
self.client_config.eth1.follow_distance = distance;
}
pub fn set_genesis(&mut self, method: ClientGenesis) {
self.client_config.genesis = method;
}
/// Import the libp2p address for `server` into the list of libp2p nodes to connect with.
///
/// If `port` is `Some`, it is used as the port for the `Multiaddr`. If `port` is `None`,
/// attempts to connect to the `server` via HTTP and retrieve it's libp2p listen port.
pub fn import_bootstrap_libp2p_address(
&mut self,
server: &str,
port: Option<u16>,
) -> Result<()> {
let bootstrapper = Bootstrapper::connect(server.to_string(), &self.log)?;
if let Some(server_multiaddr) = bootstrapper.best_effort_multiaddr(port) {
info!(
self.log,
"Estimated bootstrapper libp2p address";
"multiaddr" => format!("{:?}", server_multiaddr)
);
self.client_config
.network
.libp2p_nodes
.push(server_multiaddr);
} else {
warn!(
self.log,
"Unable to estimate a bootstrapper libp2p address, this node may not find any peers."
);
};
Ok(())
}
/// Import the enr address for `server` into the list of initial enrs (boot nodes).
pub fn import_bootstrap_enr_address(&mut self, server: &str) -> Result<()> {
let bootstrapper = Bootstrapper::connect(server.to_string(), &self.log)?;
if let Ok(enr) = bootstrapper.enr() {
info!(
self.log,
"Loaded bootstrapper libp2p address";
"enr" => format!("{:?}", enr)
);
self.client_config.network.boot_nodes.push(enr);
} else {
warn!(
self.log,
"Unable to estimate a bootstrapper enr address, this node may not find any peers."
);
};
Ok(())
}
/// Set the config data_dir to be an random directory.
///
/// Useful for easily spinning up ephemeral testnets.
pub fn set_random_datadir(&mut self) -> Result<()> {
self.client_config
.data_dir
.push(format!("random_{}", random_string(6)));
self.client_config.network.network_dir = self.client_config.data_dir.join("network");
Ok(())
}
/// Imports an `Eth2Config` from `server`, returning an error if this fails.
pub fn import_bootstrap_eth2_config(&mut self, server: &str) -> Result<()> {
let bootstrapper = Bootstrapper::connect(server.to_string(), &self.log)?;
self.update_eth2_config(bootstrapper.eth2_config()?);
Ok(())
}
fn update_eth2_config(&mut self, eth2_config: Eth2Config) {
self.eth2_config = eth2_config;
}
fn set_slot_time(&mut self, milliseconds_per_slot: u64) {
self.eth2_config.spec.milliseconds_per_slot = milliseconds_per_slot;
}
/// Reads the subcommand and tries to update `self.eth2_config` based up on the `--spec` flag.
///
/// Returns an error if the `--spec` flag is not present in the given `cli_args`.
pub fn update_spec_from_subcommand(&mut self, cli_args: &ArgMatches) -> Result<()> {
// Re-initialise the `Eth2Config`.
//
// If a CLI parameter is set, overwrite any config file present.
// If a parameter is not set, use either the config file present or default to minimal.
let eth2_config = match cli_args.value_of("spec") {
Some("mainnet") => Eth2Config::mainnet(),
Some("minimal") => Eth2Config::minimal(),
Some("interop") => Eth2Config::interop(),
_ => return Err("Unable to determine specification type.".into()),
};
self.client_config.spec_constants = cli_args
.value_of("spec")
.expect("Guarded by prior match statement")
.to_string();
self.eth2_config = eth2_config;
Ok(())
}
/// Writes the configs in `self` to `self.data_dir`.
///
/// Returns an error if `self.data_dir` already exists.
pub fn write_configs_to_new_datadir(&mut self) -> Result<()> {
let db_exists = self
.client_config
.get_db_path()
.map(|d| d.exists())
.unwrap_or_else(|| false);
// Do not permit creating a new config when the datadir exists.
if db_exists {
return Err("Database already exists. See `-f` or `-r` in `testnet --help`".into());
}
// Create `datadir` and any non-existing parent directories.
fs::create_dir_all(&self.client_config.data_dir).map_err(|e| {
crit!(self.log, "Failed to initialize data dir"; "error" => format!("{}", e));
format!("{}", e)
})?;
let client_config_file = self.client_config.data_dir.join(CLIENT_CONFIG_FILENAME);
if client_config_file.exists() {
return Err(format!(
"Datadir is not clean, {} exists. See `-f` in `testnet --help`.",
CLIENT_CONFIG_FILENAME
));
} else {
// Write the onfig to a TOML file in the datadir.
write_to_file(
self.client_config.data_dir.join(CLIENT_CONFIG_FILENAME),
&self.client_config,
)
.map_err(|e| format!("Unable to write {} file: {:?}", CLIENT_CONFIG_FILENAME, e))?;
}
let eth2_config_file = self.client_config.data_dir.join(ETH2_CONFIG_FILENAME);
if eth2_config_file.exists() {
return Err(format!(
"Datadir is not clean, {} exists. See `-f` in `testnet --help`.",
ETH2_CONFIG_FILENAME
));
} else {
// Write the config to a TOML file in the datadir.
write_to_file(
self.client_config.data_dir.join(ETH2_CONFIG_FILENAME),
&self.eth2_config,
)
.map_err(|e| format!("Unable to write {} file: {:?}", ETH2_CONFIG_FILENAME, e))?;
}
Ok(())
}
/// Attempts to load the client and eth2 configs from `self.data_dir`.
///
/// Returns an error if any files are not found or are invalid.
pub fn load_from_datadir(&mut self) -> Result<()> {
// Check to ensure the datadir exists.
//
// For now we return an error. In the future we may decide to boot a default (e.g.,
// public testnet or mainnet).
if !self
.client_config
.get_data_dir()
.map_or(false, |d| d.exists())
{
return Err(
"No datadir found. Either create a new testnet or specify a different `--datadir`."
.into(),
);
}
// If there is a path to a database in the config, ensure it exists.
if !self
.client_config
.get_db_path()
.map_or(false, |path| path.exists())
{
return Err(
"No database found in datadir. Use 'testnet -f' to overwrite the existing \
datadir, or specify a different `--datadir`."
.into(),
);
}
self.load_eth2_config(self.client_config.data_dir.join(ETH2_CONFIG_FILENAME))?;
self.load_client_config(self.client_config.data_dir.join(CLIENT_CONFIG_FILENAME))?;
Ok(())
}
/// Attempts to load the client config from `path`.
///
/// Returns an error if any files are not found or are invalid.
pub fn load_client_config(&mut self, path: PathBuf) -> Result<()> {
self.client_config = read_from_file::<ClientConfig>(path.clone())
.map_err(|e| format!("Unable to parse {:?} file: {:?}", path, e))?
.ok_or_else(|| format!("{:?} file does not exist", path))?;
Ok(())
}
/// Attempts to load the eth2 config from `path`.
///
/// Returns an error if any files are not found or are invalid.
pub fn load_eth2_config(&mut self, path: PathBuf) -> Result<()> {
self.eth2_config = read_from_file::<Eth2Config>(path.clone())
.map_err(|e| format!("Unable to parse {:?} file: {:?}", path, e))?
.ok_or_else(|| format!("{:?} file does not exist", path))?;
Ok(())
}
/// Sets all listening addresses to the given `addr`.
pub fn set_listen_addresses(&mut self, addr: String) -> Result<()> {
let addr = addr
.parse::<Ipv4Addr>()
.map_err(|e| format!("Unable to parse default listen address: {:?}", e))?;
self.client_config.network.listen_address = addr.into();
self.client_config.rest_api.listen_address = addr;
Ok(())
}
/// Consumes self, returning the configs.
///
/// The supplied `cli_args` should be the base-level `clap` cli_args (i.e., not a subcommand
/// cli_args).
pub fn build(mut self, cli_args: &ArgMatches) -> Result<Config> {
self.client_config.apply_cli_args(cli_args, &self.log)?;
if let Some(bump) = cli_args.value_of("port-bump") {
let bump = bump
.parse::<u16>()
.map_err(|e| format!("Unable to parse port bump: {}", e))?;
self.client_config.network.libp2p_port += bump;
self.client_config.network.discovery_port += bump;
self.client_config.rest_api.port += bump;
self.client_config.websocket_server.port += bump;
}
if self.eth2_config.spec_constants != self.client_config.spec_constants {
crit!(self.log, "Specification constants do not match.";
"client_config" => self.client_config.spec_constants.to_string(),
"eth2_config" => self.eth2_config.spec_constants.to_string()
);
return Err("Specification constant mismatch".into());
}
Ok((self.client_config, self.eth2_config, self.log))
}
}
fn random_string(len: usize) -> String {
rand::thread_rng()
.sample_iter(&Alphanumeric)