gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-04-16 20:39:10 +00:00
Code Issues Packages Projects Releases Wiki Activity

Wallet-based, encrypted key management (#1138)

Browse Source 
* Update hashmap hashset to stable futures

* Adds panic test to hashset delay

* Port remote_beacon_node to stable futures

* Fix lcli merge conflicts

* Non rpc stuff compiles

* Remove padding

* Add error enum, zeroize more things

* Fix comment

* protocol.rs compiles

* Port websockets, timer and notifier to stable futures (#1035)

* Fix lcli

* Port timer to stable futures

* Fix timer

* Port websocket_server to stable futures

* Port notifier to stable futures

* Add TODOS

* Port remote_beacon_node to stable futures

* Partial eth2-libp2p stable future upgrade

* Finished first round of fighting RPC types

* Further progress towards porting eth2-libp2p adds caching to discovery

* Update behaviour

* Add keystore builder

* Remove keystore stuff from val client

* Add more tests, comments

* RPC handler to stable futures

* Update RPC to master libp2p

* Add more comments, test vectors

* Network service additions

* Progress on improving JSON validation

* More JSON verification

* Start moving JSON into own mod

* Remove old code

* Add more tests, reader/writers

* Tidy

* Move keystore into own file

* Move more logic into keystore file

* Tidy

* Tidy

* Fix the fallback transport construction (#1102)

* Allow for odd-character hex

* Correct warning

* Remove hashmap delay

* Compiling version of eth2-libp2p

* Update all crates versions

* Fix conversion function and add tests (#1113)

* Add more json missing field checks

* Use scrypt by default

* Tidy, address comments

* Test path and uuid in vectors

* Fix comment

* Add checks for kdf params

* Enforce empty kdf message

* Port validator_client to stable futures (#1114)

* Add PH & MS slot clock changes

* Account for genesis time

* Add progress on duties refactor

* Add simple is_aggregator bool to val subscription

* Start work on attestation_verification.rs

* Add progress on ObservedAttestations

* Progress with ObservedAttestations

* Fix tests

* Add observed attestations to the beacon chain

* Add attestation observation to processing code

* Add progress on attestation verification

* Add first draft of ObservedAttesters

* Add more tests

* Add observed attesters to beacon chain

* Add observers to attestation processing

* Add more attestation verification

* Create ObservedAggregators map

* Remove commented-out code

* Add observed aggregators into chain

* Add progress

* Finish adding features to attestation verification

* Ensure beacon chain compiles

* Link attn verification into chain

* Integrate new attn verification in chain

* Remove old attestation processing code

* Start trying to fix beacon_chain tests

* Split adding into pools into two functions

* Add aggregation to harness

* Get test harness working again

* Adjust the number of aggregators for test harness

* Fix edge-case in harness

* Integrate new attn processing in network

* Fix compile bug in validator_client

* Update validator API endpoints

* Fix aggreagation in test harness

* Fix enum thing

* Fix attestation observation bug:

* Patch failing API tests

* Start adding comments to attestation verification

* Remove unused attestation field

* Unify "is block known" logic

* Update comments

* Supress fork choice errors for network processing

* Add todos

* Tidy

* Add gossip attn tests

* Disallow test harness to produce old attns

* Comment out in-progress tests

* Partially address pruning tests

* Fix failing store test

* Add aggregate tests

* Add comments about which spec conditions we check

* Dont re-aggregate

* Split apart test harness attn production

* Fix compile error in network

* Make progress on commented-out test

* Fix skipping attestation test

* Add fork choice verification tests

* Tidy attn tests, remove dead code

* Remove some accidentally added code

* Fix clippy lint

* Rename test file

* Add block tests, add cheap block proposer check

* Rename block testing file

* Add observed_block_producers

* Tidy

* Switch around block signature verification

* Finish block testing

* Remove gossip from signature tests

* First pass of self review

* Fix deviation in spec

* Update test spec tags

* Start moving over to hashset

* Finish moving observed attesters to hashmap

* Move aggregation pool over to hashmap

* Make fc attn borrow again

* Fix rest_api compile error

* Fix missing comments

* Fix monster test

* Uncomment increasing slots test

* Address remaining comments

* Remove unsafe, use cfg test

* Remove cfg test flag

* Fix dodgy comment

* Revert "Update hashmap hashset to stable futures"

This reverts commit d432378a3c.

* Revert "Adds panic test to hashset delay"

This reverts commit 281502396f.

* Ported attestation_service

* Ported duties_service

* Ported fork_service

* More ports

* Port block_service

* Minor fixes

* VC compiles

* Update TODOS

* Borrow self where possible

* Ignore aggregates that are already known.

* Unify aggregator modulo logic

* Fix typo in logs

* Refactor validator subscription logic

* Avoid reproducing selection proof

* Skip HTTP call if no subscriptions

* Rename DutyAndState -> DutyAndProof

* Tidy logs

* Print root as dbg

* Fix compile errors in tests

* Fix compile error in test

* Re-Fix attestation and duties service

* Minor fixes

Co-authored-by: Paul Hauner <paul@paulhauner.com>

* Expose json_keystore mod

* First commits on path derivation

* Progress with implementation

* More progress

* Passing intermediate test vectors

* Tidy, add comments

* Add DerivedKey structs

* Move key derivation into own crate

* Add zeroize structs

* Return error for empty seed

* Add tests

* Tidy

* First commits on path derivation

* Progress with implementation

* Move key derivation into own crate

* Start defining JSON wallet

* Add progress

* Split out encrypt/decrypt

* First commits on path derivation

* Progress with implementation

* More progress

* Passing intermediate test vectors

* Tidy, add comments

* Add DerivedKey structs

* Move key derivation into own crate

* Add zeroize structs

* Return error for empty seed

* Add tests

* Tidy

* Add progress

* Replace some password usage with slice

* First commits on path derivation

* Progress with implementation

* More progress

* Passing intermediate test vectors

* Tidy, add comments

* Add DerivedKey structs

* Move key derivation into own crate

* Add zeroize structs

* Return error for empty seed

* Add tests

* Tidy

* Add progress

* Expose PlainText struct

* First commits on path derivation

* Progress with implementation

* More progress

* Passing intermediate test vectors

* Tidy, add comments

* Add DerivedKey structs

* Move key derivation into own crate

* Add zeroize structs

* Return error for empty seed

* Add tests

* Tidy

* Add builder

* Expose consts, remove Password

* Minor progress

* Expose SALT_SIZE

* First compiling version

* Add test vectors

* Network crate update to stable futures

* Move dbg assert statement

* Port account_manager to stable futures (#1121)

* Port account_manager to stable futures

* Run async fns in tokio environment

* Port rest_api crate to stable futures (#1118)

* Port rest_api lib to stable futures

* Reduce tokio features

* Update notifier to stable futures

* Builder update

* Further updates

* Add mnemonic, tidy

* Convert self referential async functions

* Tidy

* Add testing

* Add first attempt at validator_dir

* Present pubkey field

* stable futures fixes (#1124)

* Fix eth1 update functions

* Fix genesis and client

* Fix beacon node lib

* Return appropriate runtimes from environment

* Fix test rig

* Refactor eth1 service update

* Upgrade simulator to stable futures

* Lighthouse compiles on stable futures

* Add first pass of wallet manager

* Progress with CLI

* Remove println debugging statement

* Tidy output

* Tidy 600 perms

* Update libp2p service, start rpc test upgrade

* Add validator creation flow

* Update network crate for new libp2p

* Start tidying, adding comments

* Update tokio::codec to futures_codec (#1128)

* Further work towards RPC corrections

* Correct http timeout and network service select

* Add wallet mgr testing

* Shift LockedWallet into own file

* Add comments to fs

* Start integration into VC

* Use tokio runtime for libp2p

* Revert "Update tokio::codec to futures_codec (#1128)"

This reverts commit e57aea924a.

* Upgrade RPC libp2p tests

* Upgrade secio fallback test

* Add lcli keypair upgrade command

* Upgrade gossipsub examples

* Clean up RPC protocol

* Test fixes (#1133)

* Correct websocket timeout and run on os thread

* Fix network test

* Add --secrets-dir to VC

* Remove --legacy-keys from VC

* Clean up PR

* Correct tokio tcp move attestation service tests

* Upgrade attestation service tests

* Fix sim

* Correct network test

* Correct genesis test

* Start docs

* Add progress for validator generation

* Tidy error messages

* Test corrections

* Log info when block is received

* Modify logs and update attester service events

* Stable futures: fixes to vc, eth1 and account manager (#1142)

* Add local testnet scripts

* Remove whiteblock script

* Rename local testnet script

* Move spawns onto handle

* Fix VC panic

* Initial fix to block production issue

* Tidy block producer fix

* Tidy further

* Add local testnet clean script

* Run cargo fmt

* Tidy duties service

* Tidy fork service

* Tidy ForkService

* Tidy AttestationService

* Tidy notifier

* Ensure await is not suppressed in eth1

* Ensure await is not suppressed in account_manager

* Use .ok() instead of .unwrap_or(())

* RPC decoding test for proto

* Update discv5 and eth2-libp2p deps

* Run cargo fmt

* Pre-build keystores for sim

* Fix lcli double runtime issue (#1144)

* Handle stream termination and dialing peer errors

* Correct peer_info variant types

* Add progress on new deposit flow

* Remove unnecessary warnings

* Handle subnet unsubscription removal and improve logigng

* Add logs around ping

* Upgrade discv5 and improve logging

* Handle peer connection status for multiple connections

* Improve network service logging

* Add more incomplete progress

* Improve logging around peer manager

* Upgrade swarm poll centralise peer management

* Identify clients on error

* Fix `remove_peer` in sync (#1150)

* remove_peer removes from all chains

* Remove logs

* Fix early return from loop

* Improved logging, fix panic

* Partially correct tests

* Add deposit command

* Remove old validator directory

* Start adding AM tests

* Stable futures: Vc sync (#1149)

* Improve syncing heuristic

* Add comments

* Use safer method for tolerance

* Fix tests

* Binary testing progress

* Progress with CLI tests

* Use constants for flags

* More account manager testing

* Improve CLI tests

* Move upgrade-legacy-keypairs into account man

* Use rayon for VC key generation

* Add comments to `validator_dir`

* Add testing to validator_dir

* Add fix to eth1-sim

* Check errors in eth1-sim

* Fix mutability issue

* Ensure password file ends in .pass

* Add more tests to wallet manager

* Tidy deposit

* Tidy account manager

* Tidy account manager

* Remove panic

* Generate keypairs earlier in sim

* Tidy eth1-sime

* Try to fix eth1 sim

* Address review comments

* Fix typo in CLI command

* Update docs

* Disable eth1 sim

* Remove eth1 sim completely

Co-authored-by: Age Manning <Age@AgeManning.com>
Co-authored-by: pawanjay176 <pawandhananjay@gmail.com>
This commit is contained in:
Paul Hauner
2020-05-18 19:01:45 +10:00
committed by GitHub
parent a4b07a833c
commit c571afb8d8
55 changed files with 3761 additions and 1163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

91
account_manager/src/cli.rs
View File

@@ -1,91 +0,0 @@
use crate::deposits;
use clap::{App, Arg, SubCommand};
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new("account_manager")
.visible_aliases(&["a", "am", "account", "account_manager"])
.about("Utilities for generating and managing Ethereum 2.0 accounts.")
.subcommand(
SubCommand::with_name("validator")
.about("Generate or manage Ethereum 2.0 validators.")
.subcommand(deposits::cli_app())
.subcommand(
SubCommand::with_name("new")
.about("Create a new Ethereum 2.0 validator.")
.arg(
Arg::with_name("deposit-value")
.short("v")
.long("deposit-value")
.value_name("GWEI")
.takes_value(true)
.default_value("32000000000")
.help("The deposit amount in Gwei (not Wei). Default is 32 ETH."),
)
.arg(
Arg::with_name("send-deposits")
.long("send-deposits")
.help("If present, submit validator deposits to an eth1 endpoint /
defined by the --eth1-endpoint. Requires either the /
--deposit-contract or --testnet-dir flag.")
)
.arg(
Arg::with_name("eth1-endpoint")
.short("e")
.long("eth1-endpoint")
.value_name("HTTP_SERVER")
.takes_value(true)
.default_value("http://localhost:8545")
.help("The URL to the Eth1 JSON-RPC HTTP API (e.g., Geth/Parity-Ethereum)."),
)
.arg(
Arg::with_name("account-index")
.short("i")
.long("account-index")
.value_name("INDEX")
.takes_value(true)
.default_value("0")
.help("The eth1 accounts[] index which will send the transaction"),
)
.arg(
Arg::with_name("password")
.short("p")
.long("password")
.value_name("FILE")
.takes_value(true)
.help("The password file to unlock the eth1 account (see --index)"),
)
.subcommand(
SubCommand::with_name("insecure")
.about("Produce insecure, ephemeral validators. DO NOT USE TO STORE VALUE.")
.arg(
Arg::with_name("first")
.index(1)
.value_name("INDEX")
.help("Index of the first validator")
.takes_value(true)
.required(true),
)
.arg(
Arg::with_name("last")
.index(2)
.value_name("INDEX")
.help("Index of the last validator")
.takes_value(true)
.required(true),
),
)
.subcommand(
SubCommand::with_name("random")
.about("Produces public keys using entropy from the Rust 'rand' library.")
.arg(
Arg::with_name("validator_count")
.index(1)
.value_name("INTEGER")
.help("The number of new validators to generate.")
.takes_value(true)
.default_value("1"),
),
)
)
)
}

39
account_manager/src/common.rs Normal file
View File

@@ -0,0 +1,39 @@
use clap::ArgMatches;
use eth2_wallet::PlainText;
use rand::{distributions::Alphanumeric, Rng};
use std::fs::create_dir_all;
use std::path::{Path, PathBuf};
/// The `Alphanumeric` crate only generates a-z, A-Z, 0-9, therefore it has a range of 62
/// characters.
///
/// 62**48 is greater than 255**32, therefore this password has more bits of entropy than a byte
/// array of length 32.
const DEFAULT_PASSWORD_LEN: usize = 48;
pub fn random_password() -> PlainText {
rand::thread_rng()
.sample_iter(&Alphanumeric)
.take(DEFAULT_PASSWORD_LEN)
.collect::<String>()
.into_bytes()
.into()
}
pub fn ensure_dir_exists<P: AsRef<Path>>(path: P) -> Result<(), String> {
let path = path.as_ref();
if !path.exists() {
create_dir_all(path).map_err(|e| format!("Unable to create {:?}: {:?}", path, e))?;
}
Ok(())
}
pub fn base_wallet_dir(matches: &ArgMatches, arg: &'static str) -> Result<PathBuf, String> {
clap_utils::parse_path_with_default_in_home_dir(
matches,
arg,
PathBuf::new().join(".lighthouse").join("wallets"),
)
}

241
account_manager/src/deposits.rs
View File

@@ -1,241 +0,0 @@
use clap::{App, Arg, ArgMatches};
use clap_utils;
use environment::Environment;
use futures::compat::Future01CompatExt;
use slog::{info, Logger};
use std::fs;
use std::path::PathBuf;
use tokio::time::{delay_until, Duration, Instant};
use types::EthSpec;
use validator_client::validator_directory::ValidatorDirectoryBuilder;
use web3::{
transports::Ipc,
types::{Address, SyncInfo, SyncState},
Transport, Web3,
};
const SYNCING_STATE_RETRY_DELAY: Duration = Duration::from_secs(2);
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new("deposited")
.about("Creates new Lighthouse validator keys and directories. Each newly-created validator
will have a deposit transaction formed and submitted to the deposit contract via
--eth1-ipc. This application will only write each validator keys to disk if the deposit
transaction returns successfully from the eth1 node. The process exits immediately if any
Eth1 tx fails. Does not wait for Eth1 confirmation blocks, so there is no guarantee that a
deposit will be accepted in the Eth1 chain. Before key generation starts, this application
will wait until the eth1 indicates that it is not syncing via the eth_syncing endpoint")
.arg(
Arg::with_name("validator-dir")
.long("validator-dir")
.value_name("VALIDATOR_DIRECTORY")
.help("The path where the validator directories will be created. Defaults to ~/.lighthouse/validators")
.takes_value(true),
)
.arg(
Arg::with_name("eth1-ipc")
.long("eth1-ipc")
.value_name("ETH1_IPC_PATH")
.help("Path to an Eth1 JSON-RPC IPC endpoint")
.takes_value(true)
.required(true)
)
.arg(
Arg::with_name("from-address")
.long("from-address")
.value_name("FROM_ETH1_ADDRESS")
.help("The address that will submit the eth1 deposit. Must be unlocked on the node
at --eth1-ipc.")
.takes_value(true)
.required(true)
)
.arg(
Arg::with_name("deposit-gwei")
.long("deposit-gwei")
.value_name("DEPOSIT_GWEI")
.help("The GWEI value of the deposit amount. Defaults to the minimum amount
required for an active validator (MAX_EFFECTIVE_BALANCE.")
.takes_value(true),
)
.arg(
Arg::with_name("count")
.long("count")
.value_name("DEPOSIT_COUNT")
.help("The number of deposits to create, regardless of how many already exist")
.conflicts_with("limit")
.takes_value(true),
)
.arg(
Arg::with_name("at-most")
.long("at-most")
.value_name("VALIDATOR_COUNT")
.help("Observe the number of validators in --validator-dir, only creating enough to
ensure reach the given count. Never deletes an existing validator.")
.conflicts_with("count")
.takes_value(true),
)
}
pub fn cli_run<T: EthSpec>(
matches: &ArgMatches<'_>,
mut env: Environment<T>,
) -> Result<(), String> {
let spec = env.core_context().eth2_config.spec;
let log = env.core_context().log;
let validator_dir = clap_utils::parse_path_with_default_in_home_dir(
matches,
"validator_dir",
PathBuf::new().join(".lighthouse").join("validators"),
)?;
let eth1_ipc_path: PathBuf = clap_utils::parse_required(matches, "eth1-ipc")?;
let from_address: Address = clap_utils::parse_required(matches, "from-address")?;
let deposit_gwei = clap_utils::parse_optional(matches, "deposit-gwei")?
.unwrap_or_else(|| spec.max_effective_balance);
let count: Option<usize> = clap_utils::parse_optional(matches, "count")?;
let at_most: Option<usize> = clap_utils::parse_optional(matches, "at-most")?;
let starting_validator_count = existing_validator_count(&validator_dir)?;
let n = match (count, at_most) {
(Some(_), Some(_)) => Err("Cannot supply --count and --at-most".to_string()),
(None, None) => Err("Must supply either --count or --at-most".to_string()),
(Some(count), None) => Ok(count),
(None, Some(at_most)) => Ok(at_most.saturating_sub(starting_validator_count)),
}?;
if n == 0 {
info!(
log,
"No need to produce and validators, exiting";
"--count" => count,
"--at-most" => at_most,
"existing_validators" => starting_validator_count,
);
return Ok(());
}
let deposit_contract = env
.testnet
.as_ref()
.ok_or_else(|| "Unable to run account manager without a testnet dir".to_string())?
.deposit_contract_address()
.map_err(|e| format!("Unable to parse deposit contract address: {}", e))?;
if deposit_contract == Address::zero() {
return Err("Refusing to deposit to the zero address. Check testnet configuration.".into());
}
let (_event_loop_handle, transport) =
Ipc::new(eth1_ipc_path).map_err(|e| format!("Unable to connect to eth1 IPC: {:?}", e))?;
let web3 = Web3::new(transport);
env.runtime()
.block_on(poll_until_synced(web3.clone(), log.clone()))?;
for i in 0..n {
let tx_hash_log = log.clone();
env.runtime()
.block_on(async {
ValidatorDirectoryBuilder::default()
.spec(spec.clone())
.custom_deposit_amount(deposit_gwei)
.thread_random_keypairs()
.submit_eth1_deposit(web3.clone(), from_address, deposit_contract)
.await
.map(move |(builder, tx_hash)| {
info!(
tx_hash_log,
"Validator deposited";
"eth1_tx_hash" => format!("{:?}", tx_hash),
"index" => format!("{}/{}", i + 1, n),
);
builder
})
})?
.create_directory(validator_dir.clone())?
.write_keypair_files()?
.write_eth1_data_file()?
.build()?;
}
let ending_validator_count = existing_validator_count(&validator_dir)?;
let delta = ending_validator_count.saturating_sub(starting_validator_count);
info!(
log,
"Success";
"validators_created_and_deposited" => delta,
);
Ok(())
}
/// Returns the number of validators that exist in the given `validator_dir`.
///
/// This function just assumes any file is a validator directory, making it likely to return a
/// higher number than accurate but never a lower one.
fn existing_validator_count(validator_dir: &PathBuf) -> Result<usize, String> {
fs::read_dir(&validator_dir)
.map(|iter| iter.count())
.map_err(|e| format!("Unable to read {:?}: {}", validator_dir, e))
}
/// Run a poll on the `eth_syncing` endpoint, blocking until the node is synced.
async fn poll_until_synced<T>(web3: Web3<T>, log: Logger) -> Result<(), String>
where
T: Transport + Send + 'static,
<T as Transport>::Out: Send,
{
loop {
let sync_state = web3
.clone()
.eth()
.syncing()
.compat()
.await
.map_err(|e| format!("Unable to read syncing state from eth1 node: {:?}", e))?;
match sync_state {
SyncState::Syncing(SyncInfo {
current_block,
highest_block,
..
}) => {
info!(
log,
"Waiting for eth1 node to sync";
"est_highest_block" => format!("{}", highest_block),
"current_block" => format!("{}", current_block),
);
delay_until(Instant::now() + SYNCING_STATE_RETRY_DELAY).await;
}
SyncState::NotSyncing => {
let block_number = web3
.clone()
.eth()
.block_number()
.compat()
.await
.map_err(|e| format!("Unable to read block number from eth1 node: {:?}", e))?;
if block_number > 0.into() {
info!(
log,
"Eth1 node is synced";
"head_block" => format!("{}", block_number),
);
break;
} else {
delay_until(Instant::now() + SYNCING_STATE_RETRY_DELAY).await;
info!(
log,
"Waiting for eth1 node to sync";
"current_block" => 0,
);
}
}
}
}
Ok(())
}

475
account_manager/src/lib.rs
View File

@@ -1,457 +1,40 @@
mod cli;
mod deposits;
mod common;
pub mod upgrade_legacy_keypairs;
pub mod validator;
pub mod wallet;
use clap::App;
use clap::ArgMatches;
use deposit_contract::DEPOSIT_GAS;
use environment::{Environment, RuntimeContext};
use eth2_testnet_config::Eth2TestnetConfig;
use futures::compat::Future01CompatExt;
use futures::{FutureExt, StreamExt};
use rayon::prelude::*;
use slog::{error, info, Logger};
use std::fs;
use std::fs::File;
use std::io::Read;
use std::path::PathBuf;
use types::{ChainSpec, EthSpec};
use validator_client::validator_directory::{ValidatorDirectory, ValidatorDirectoryBuilder};
use web3::{
transports::Http,
types::{Address, TransactionRequest, U256},
Web3,
};
use environment::Environment;
use types::EthSpec;
pub use cli::cli_app;
pub const CMD: &str = "account_manager";
pub const SECRETS_DIR_FLAG: &str = "secrets-dir";
pub const VALIDATOR_DIR_FLAG: &str = "validator-dir";
pub const BASE_DIR_FLAG: &str = "base-dir";
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new(CMD)
.visible_aliases(&["a", "am", "account", CMD])
.about("Utilities for generating and managing Ethereum 2.0 accounts.")
.subcommand(wallet::cli_app())
.subcommand(validator::cli_app())
.subcommand(upgrade_legacy_keypairs::cli_app())
}
/// Run the account manager, returning an error if the operation did not succeed.
pub fn run<T: EthSpec>(matches: &ArgMatches<'_>, mut env: Environment<T>) -> Result<(), String> {
let context = env.core_context();
let log = context.log.clone();
// If the `datadir` was not provided, default to the home directory. If the home directory is
// not known, use the current directory.
let datadir = matches
.value_of("datadir")
.map(PathBuf::from)
.unwrap_or_else(|| {
dirs::home_dir()
.unwrap_or_else(|| PathBuf::from("."))
.join(".lighthouse")
.join("validators")
});
fs::create_dir_all(&datadir).map_err(|e| format!("Failed to create datadir: {}", e))?;
info!(
log,
"Located data directory";
"path" => format!("{:?}", datadir)
);
pub fn run<T: EthSpec>(matches: &ArgMatches<'_>, env: Environment<T>) -> Result<(), String> {
match matches.subcommand() {
("validator", Some(matches)) => match matches.subcommand() {
("deposited", Some(matches)) => deposits::cli_run(matches, env)?,
("new", Some(matches)) => run_new_validator_subcommand(matches, datadir, env)?,
_ => {
return Err("Invalid 'validator new' command. See --help.".to_string());
}
},
_ => {
return Err("Invalid 'validator' command. See --help.".to_string());
(wallet::CMD, Some(matches)) => wallet::cli_run(matches)?,
(validator::CMD, Some(matches)) => validator::cli_run(matches, env)?,
(upgrade_legacy_keypairs::CMD, Some(matches)) => upgrade_legacy_keypairs::cli_run(matches)?,
(unknown, _) => {
return Err(format!(
"{} is not a valid {} command. See --help.",
unknown, CMD
));
}
}
Ok(())
}
/// Describes the crypto key generation methods for a validator.
enum KeygenMethod {
/// Produce an insecure "deterministic" keypair. Used only for interop and testing.
Insecure(usize),
/// Generate a new key from the `rand` thread random RNG.
ThreadRandom,
}
/// Process the subcommand for creating new validators.
fn run_new_validator_subcommand<T: EthSpec>(
matches: &ArgMatches,
datadir: PathBuf,
mut env: Environment<T>,
) -> Result<(), String> {
let mut context = env.core_context();
let log = context.log.clone();
// Load the testnet configuration from disk, or use the default testnet.
let eth2_testnet_config: Eth2TestnetConfig<T> =
if let Some(testnet_dir_str) = matches.value_of("testnet-dir") {
let testnet_dir = testnet_dir_str
.parse::<PathBuf>()
.map_err(|e| format!("Unable to parse testnet-dir: {}", e))?;
if !testnet_dir.exists() {
return Err(format!(
"Testnet directory at {:?} does not exist",
testnet_dir
));
}
info!(
log,
"Loading deposit contract address";
"testnet_dir" => format!("{:?}", &testnet_dir)
);
Eth2TestnetConfig::load(testnet_dir.clone())
.map_err(|e| format!("Failed to load testnet dir at {:?}: {}", testnet_dir, e))?
} else {
info!(
log,
"Using Lighthouse testnet deposit contract";
);
Eth2TestnetConfig::hard_coded()
.map_err(|e| format!("Failed to load hard_coded testnet dir: {}", e))?
};
context.eth2_config.spec = eth2_testnet_config
.yaml_config
.as_ref()
.ok_or_else(|| "The testnet directory must contain a spec config".to_string())?
.apply_to_chain_spec::<T>(&context.eth2_config.spec)
.ok_or_else(|| {
format!(
"The loaded config is not compatible with the {} spec",
&context.eth2_config.spec_constants
)
})?;
let methods: Vec<KeygenMethod> = match matches.subcommand() {
("insecure", Some(matches)) => {
let first = matches
.value_of("first")
.ok_or_else(|| "No first index".to_string())?
.parse::<usize>()
.map_err(|e| format!("Unable to parse first index: {}", e))?;
let last = matches
.value_of("last")
.ok_or_else(|| "No last index".to_string())?
.parse::<usize>()
.map_err(|e| format!("Unable to parse first index: {}", e))?;
(first..last).map(KeygenMethod::Insecure).collect()
}
("random", Some(matches)) => {
let count = matches
.value_of("validator_count")
.ok_or_else(|| "No validator count".to_string())?
.parse::<usize>()
.map_err(|e| format!("Unable to parse validator count: {}", e))?;
(0..count).map(|_| KeygenMethod::ThreadRandom).collect()
}
_ => {
return Err("Invalid 'validator' command. See --help.".to_string());
}
};
let deposit_value = matches
.value_of("deposit-value")
.ok_or_else(|| "No deposit-value".to_string())?
.parse::<u64>()
.map_err(|e| format!("Unable to parse deposit-value: {}", e))?;
let validators = make_validators(
datadir.clone(),
&methods,
deposit_value,
&context.eth2_config.spec,
&log,
)?;
if matches.is_present("send-deposits") {
let eth1_endpoint = matches
.value_of("eth1-endpoint")
.ok_or_else(|| "No eth1-endpoint".to_string())?;
let account_index = matches
.value_of("account-index")
.ok_or_else(|| "No account-index".to_string())?
.parse::<usize>()
.map_err(|e| format!("Unable to parse account-index: {}", e))?;
// If supplied, load the eth1 account password from file.
let password = if let Some(password_path) = matches.value_of("password") {
Some(
File::open(password_path)
.map_err(|e| format!("Unable to open password file: {:?}", e))
.and_then(|mut file| {
let mut password = String::new();
file.read_to_string(&mut password)
.map_err(|e| format!("Unable to read password file to string: {:?}", e))
.map(|_| password)
})
.map(|password| {
// Trim the line feed from the end of the password file, if present.
if password.ends_with('\n') {
password[0..password.len() - 1].to_string()
} else {
password
}
})?,
)
} else {
None
};
info!(
log,
"Submitting validator deposits";
"eth1_node_http_endpoint" => eth1_endpoint
);
// Convert from `types::Address` to `web3::types::Address`.
let deposit_contract = Address::from_slice(
eth2_testnet_config
.deposit_contract_address()?
.as_fixed_bytes(),
);
if let Err(()) = env.runtime().block_on(deposit_validators(
context.clone(),
eth1_endpoint.to_string(),
deposit_contract,
validators.clone(),
account_index,
deposit_value,
password,
)) {
error!(
log,
"Created validators but could not submit deposits";
)
} else {
info!(
log,
"Validator deposits complete";
);
}
}
info!(
log,
"Generated validator directories";
"base_path" => format!("{:?}", datadir),
"count" => validators.len(),
);
Ok(())
}
/// Produces a validator directory for each of the key generation methods provided in `methods`.
fn make_validators(
datadir: PathBuf,
methods: &[KeygenMethod],
deposit_value: u64,
spec: &ChainSpec,
log: &Logger,
) -> Result<Vec<ValidatorDirectory>, String> {
methods
.par_iter()
.map(|method| {
let mut builder = ValidatorDirectoryBuilder::default()
.spec(spec.clone())
.custom_deposit_amount(deposit_value);
builder = match method {
KeygenMethod::Insecure(index) => builder.insecure_keypairs(*index),
KeygenMethod::ThreadRandom => builder.thread_random_keypairs(),
};
let validator = builder
.create_directory(datadir.clone())?
.write_keypair_files()?
.write_eth1_data_file()?
.build()?;
let pubkey = &validator
.voting_keypair
.as_ref()
.ok_or_else(|| "Generated validator must have voting keypair".to_string())?
.pk;
info!(
log,
"Saved new validator to disk";
"voting_pubkey" => format!("{:?}", pubkey)
);
Ok(validator)
})
.collect()
}
/// For each `ValidatorDirectory`, submit a deposit transaction to the `eth1_endpoint`.
///
/// Returns success as soon as the eth1 endpoint accepts the transaction (i.e., does not wait for
/// transaction success/revert).
async fn deposit_validators<E: EthSpec>(
context: RuntimeContext<E>,
eth1_endpoint: String,
deposit_contract: Address,
validators: Vec<ValidatorDirectory>,
account_index: usize,
deposit_value: u64,
password: Option<String>,
) -> Result<(), ()> {
let log_1 = context.log.clone();
let log_2 = context.log.clone();
let (event_loop, transport) = Http::new(&eth1_endpoint).map_err(move |e| {
error!(
log_1,
"Failed to start web3 HTTP transport";
"error" => format!("{:?}", e)
)
})?;
/*
* Loop through the validator directories and submit the deposits.
*/
let web3 = Web3::new(transport);
futures::stream::iter(validators)
.for_each(|validator| async {
let web3 = web3.clone();
let log = log_2.clone();
let password = password.clone();
let _ = deposit_validator(
web3,
deposit_contract,
validator,
deposit_value,
account_index,
password,
log,
)
.await;
})
.map(|_| event_loop)
// // Web3 gives errors if the event loop is dropped whilst performing requests.
.map(drop)
.await;
Ok(())
}
/// For the given `ValidatorDirectory`, submit a deposit transaction to the `web3` node.
///
/// Returns success as soon as the eth1 endpoint accepts the transaction (i.e., does not wait for
/// transaction success/revert).
async fn deposit_validator(
web3: Web3<Http>,
deposit_contract: Address,
validator: ValidatorDirectory,
deposit_amount: u64,
account_index: usize,
password_opt: Option<String>,
log: Logger,
) -> Result<(), ()> {
let voting_keypair = validator
.voting_keypair
.clone()
.ok_or_else(|| error!(log, "Validator does not have voting keypair"))?;
let deposit_data = validator
.deposit_data
.clone()
.ok_or_else(|| error!(log, "Validator does not have deposit data"))?;
let pubkey_1 = voting_keypair.pk.clone();
let pubkey_2 = voting_keypair.pk;
let log_1 = log.clone();
let log_2 = log.clone();
// TODO: creating a future to extract the Error type
// check if there's a better way
let future = async move {
let accounts = web3
.eth()
.accounts()
.compat()
.await
.map_err(|e| format!("Failed to get accounts: {:?}", e))?;
let from_address = accounts
.get(account_index)
.cloned()
.ok_or_else(|| "Insufficient accounts for deposit".to_string())?;
/*
* If a password was supplied, unlock the account.
*/
let from = if let Some(password) = password_opt {
// Unlock for only a single transaction.
let duration = None;
let result = web3
.personal()
.unlock_account(from_address, &password, duration)
.compat()
.await;
match result {
Ok(true) => from_address,
Ok(false) => {
return Err::<(), String>(
"Eth1 node refused to unlock account. Check password.".to_string(),
)
}
Err(e) => return Err::<(), String>(format!("Eth1 unlock request failed: {:?}", e)),
}
} else {
from_address
};
/*
* Submit the deposit transaction.
*/
let tx_request = TransactionRequest {
from,
to: Some(deposit_contract),
gas: Some(U256::from(DEPOSIT_GAS)),
gas_price: None,
value: Some(from_gwei(deposit_amount)),
data: Some(deposit_data.into()),
nonce: None,
condition: None,
};
let tx = web3
.eth()
.send_transaction(tx_request)
.compat()
.await
.map_err(|e| format!("Failed to call deposit fn: {:?}", e))?;
info!(
log_1,
"Validator deposit successful";
"eth1_tx_hash" => format!("{:?}", tx),
"validator_voting_pubkey" => format!("{:?}", pubkey_1)
);
Ok(())
};
future.await.map_err(move |e| {
error!(
log_2,
"Validator deposit_failed";
"error" => e,
"validator_voting_pubkey" => format!("{:?}", pubkey_2)
);
})?;
Ok(())
}
/// Converts gwei to wei.
fn from_gwei(gwei: u64) -> U256 {
U256::from(gwei) * U256::exp10(9)
}

149
account_manager/src/upgrade_legacy_keypairs.rs Normal file
View File

@@ -0,0 +1,149 @@
//! This command allows migrating from the old method of storing keys (unencrypted SSZ) to the
//! current method of using encrypted EIP-2335 keystores.
//!
//! This command should be completely removed once the `unencrypted_keys` feature is removed from
//! the `validator_dir` command. This should hopefully be in mid-June 2020.
//!
//! ## Example
//!
//! This command will upgrade all keypairs in the `--validators-dir`, storing the newly-generated
//! passwords in `--secrets-dir`.
//!
//! ```ignore
//! lighthouse am upgrade-legacy-keypairs \
//! --validators-dir ~/.lighthouse/validators \
//! --secrets-dir ~/.lighthouse/secrets
//! ```
use crate::{SECRETS_DIR_FLAG, VALIDATOR_DIR_FLAG};
use clap::{App, Arg, ArgMatches};
use clap_utils::parse_required;
use eth2_keystore::KeystoreBuilder;
use rand::{distributions::Alphanumeric, Rng};
use std::fs::{create_dir_all, read_dir, write, File};
use std::path::{Path, PathBuf};
use types::Keypair;
use validator_dir::{
unencrypted_keys::load_unencrypted_keypair, VOTING_KEYSTORE_FILE, WITHDRAWAL_KEYSTORE_FILE,
};
pub const CMD: &str = "upgrade-legacy-keypairs";
pub const VOTING_KEYPAIR_FILE: &str = "voting_keypair";
pub const WITHDRAWAL_KEYPAIR_FILE: &str = "withdrawal_keypair";
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new(CMD)
.about(
"Converts legacy unencrypted SSZ keypairs into encrypted keystores.",
)
.arg(
Arg::with_name(VALIDATOR_DIR_FLAG)
.long(VALIDATOR_DIR_FLAG)
.value_name("VALIDATORS_DIRECTORY")
.takes_value(true)
.required(true)
.help("The directory containing legacy validators. Generally ~/.lighthouse/validators"),
)
.arg(
Arg::with_name(SECRETS_DIR_FLAG)
.long(SECRETS_DIR_FLAG)
.value_name("SECRETS_DIRECTORY")
.takes_value(true)
.required(true)
.help("The directory where keystore passwords will be stored. Generally ~/.lighthouse/secrets"),
)
}
pub fn cli_run(matches: &ArgMatches) -> Result<(), String> {
let validators_dir: PathBuf = parse_required(matches, VALIDATOR_DIR_FLAG)?;
let secrets_dir: PathBuf = parse_required(matches, SECRETS_DIR_FLAG)?;
if !secrets_dir.exists() {
create_dir_all(&secrets_dir)
.map_err(|e| format!("Failed to create secrets dir {:?}: {:?}", secrets_dir, e))?;
}
read_dir(&validators_dir)
.map_err(|e| {
format!(
"Failed to read validators directory {:?}: {:?}",
validators_dir, e
)
})?
.try_for_each(|dir| {
let path = dir
.map_err(|e| format!("Unable to read dir: {}", e))?
.path();
if path.is_dir() {
if let Err(e) = upgrade_keypair(
&path,
&secrets_dir,
VOTING_KEYPAIR_FILE,
VOTING_KEYSTORE_FILE,
) {
println!("Validator {:?}: {:?}", path, e);
} else {
println!("Validator {:?} voting keys: success", path);
}
if let Err(e) = upgrade_keypair(
&path,
&secrets_dir,
WITHDRAWAL_KEYPAIR_FILE,
WITHDRAWAL_KEYSTORE_FILE,
) {
println!("Validator {:?}: {:?}", path, e);
} else {
println!("Validator {:?} withdrawal keys: success", path);
}
}
Ok(())
})
}
fn upgrade_keypair<P: AsRef<Path>>(
validator_dir: P,
secrets_dir: P,
input_filename: &str,
output_filename: &str,
) -> Result<(), String> {
let validator_dir = validator_dir.as_ref();
let secrets_dir = secrets_dir.as_ref();
let keypair: Keypair = load_unencrypted_keypair(validator_dir.join(input_filename))?.into();
let password = rand::thread_rng()
.sample_iter(&Alphanumeric)
.take(48)
.collect::<String>()
.into_bytes();
let keystore = KeystoreBuilder::new(&keypair, &password, "".into())
.map_err(|e| format!("Unable to create keystore builder: {:?}", e))?
.build()
.map_err(|e| format!("Unable to build keystore: {:?}", e))?;
let keystore_path = validator_dir.join(output_filename);
if keystore_path.exists() {
return Err(format!("{:?} already exists", keystore_path));
}
let mut file = File::create(&keystore_path).map_err(|e| format!("Cannot create: {:?}", e))?;
keystore
.to_json_writer(&mut file)
.map_err(|e| format!("Cannot write keystore to {:?}: {:?}", keystore_path, e))?;
let password_path = secrets_dir.join(format!("{}", keypair.pk.as_hex_string()));
if password_path.exists() {
return Err(format!("{:?} already exists", password_path));
}
write(&password_path, &password)
.map_err(|e| format!("Unable to write password to {:?}: {:?}", password_path, e))?;
Ok(())
}

203
account_manager/src/validator/create.rs Normal file
View File

@@ -0,0 +1,203 @@
use crate::{
common::{ensure_dir_exists, random_password},
SECRETS_DIR_FLAG, VALIDATOR_DIR_FLAG,
};
use clap::{App, Arg, ArgMatches};
use environment::Environment;
use eth2_wallet::PlainText;
use eth2_wallet_manager::WalletManager;
use std::fs;
use std::path::{Path, PathBuf};
use types::EthSpec;
use validator_dir::Builder as ValidatorDirBuilder;
pub const CMD: &str = "create";
pub const BASE_DIR_FLAG: &str = "base-dir";
pub const WALLET_NAME_FLAG: &str = "wallet-name";
pub const WALLET_PASSPHRASE_FLAG: &str = "wallet-passphrase";
pub const DEPOSIT_GWEI_FLAG: &str = "deposit-gwei";
pub const STORE_WITHDRAW_FLAG: &str = "store-withdrawal-keystore";
pub const COUNT_FLAG: &str = "count";
pub const AT_MOST_FLAG: &str = "at-most";
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new(CMD)
.about(
"Creates new validators from an existing EIP-2386 wallet using the EIP-2333 HD key \
derivation scheme.",
)
.arg(
Arg::with_name(WALLET_NAME_FLAG)
.long(WALLET_NAME_FLAG)
.value_name("WALLET_NAME")
.help("Use the wallet identified by this name")
.takes_value(true)
.required(true),
)
.arg(
Arg::with_name(WALLET_PASSPHRASE_FLAG)
.long(WALLET_PASSPHRASE_FLAG)
.value_name("WALLET_PASSWORD_PATH")
.help("A path to a file containing the password which will unlock the wallet.")
.takes_value(true)
.required(true),
)
.arg(
Arg::with_name(VALIDATOR_DIR_FLAG)
.long(VALIDATOR_DIR_FLAG)
.value_name("VALIDATOR_DIRECTORY")
.help(
"The path where the validator directories will be created. \
Defaults to ~/.lighthouse/validators",
)
.takes_value(true),
)
.arg(
Arg::with_name(SECRETS_DIR_FLAG)
.long(SECRETS_DIR_FLAG)
.value_name("SECRETS_DIR")
.help(
"The path where the validator keystore passwords will be stored. \
Defaults to ~/.lighthouse/secrets",
)
.takes_value(true),
)
.arg(
Arg::with_name(DEPOSIT_GWEI_FLAG)
.long(DEPOSIT_GWEI_FLAG)
.value_name("DEPOSIT_GWEI")
.help(
"The GWEI value of the deposit amount. Defaults to the minimum amount \
required for an active validator (MAX_EFFECTIVE_BALANCE)",
)
.takes_value(true),
)
.arg(
Arg::with_name(STORE_WITHDRAW_FLAG)
.long(STORE_WITHDRAW_FLAG)
.help(
"If present, the withdrawal keystore will be stored alongside the voting \
keypair. It is generally recommended to *not* store the withdrawal key and \
instead generate them from the wallet seed when required.",
),
)
.arg(
Arg::with_name(COUNT_FLAG)
.long(COUNT_FLAG)
.value_name("VALIDATOR_COUNT")
.help("The number of validators to create, regardless of how many already exist")
.conflicts_with("at-most")
.takes_value(true),
)
.arg(
Arg::with_name(AT_MOST_FLAG)
.long(AT_MOST_FLAG)
.value_name("AT_MOST_VALIDATORS")
.help(
"Observe the number of validators in --validator-dir, only creating enough to \
reach the given count. Never deletes an existing validator.",
)
.conflicts_with("count")
.takes_value(true),
)
}
pub fn cli_run<T: EthSpec>(
matches: &ArgMatches,
mut env: Environment<T>,
wallet_base_dir: PathBuf,
) -> Result<(), String> {
let spec = env.core_context().eth2_config.spec;
let name: String = clap_utils::parse_required(matches, WALLET_NAME_FLAG)?;
let wallet_password_path: PathBuf =
clap_utils::parse_required(matches, WALLET_PASSPHRASE_FLAG)?;
let validator_dir = clap_utils::parse_path_with_default_in_home_dir(
matches,
VALIDATOR_DIR_FLAG,
PathBuf::new().join(".lighthouse").join("validators"),
)?;
let secrets_dir = clap_utils::parse_path_with_default_in_home_dir(
matches,
SECRETS_DIR_FLAG,
PathBuf::new().join(".lighthouse").join("secrets"),
)?;
let deposit_gwei = clap_utils::parse_optional(matches, DEPOSIT_GWEI_FLAG)?
.unwrap_or_else(|| spec.max_effective_balance);
let count: Option<usize> = clap_utils::parse_optional(matches, COUNT_FLAG)?;
let at_most: Option<usize> = clap_utils::parse_optional(matches, AT_MOST_FLAG)?;
ensure_dir_exists(&validator_dir)?;
ensure_dir_exists(&secrets_dir)?;
let starting_validator_count = existing_validator_count(&validator_dir)?;
let n = match (count, at_most) {
(Some(_), Some(_)) => Err(format!(
"Cannot supply --{} and --{}",
COUNT_FLAG, AT_MOST_FLAG
)),
(None, None) => Err(format!(
"Must supply either --{} or --{}",
COUNT_FLAG, AT_MOST_FLAG
)),
(Some(count), None) => Ok(count),
(None, Some(at_most)) => Ok(at_most.saturating_sub(starting_validator_count)),
}?;
if n == 0 {
eprintln!(
"No validators to create. {}={:?}, {}={:?}",
COUNT_FLAG, count, AT_MOST_FLAG, at_most
);
return Ok(());
}
let wallet_password = fs::read(&wallet_password_path)
.map_err(|e| format!("Unable to read {:?}: {:?}", wallet_password_path, e))
.map(|bytes| PlainText::from(bytes))?;
let mgr = WalletManager::open(&wallet_base_dir)
.map_err(|e| format!("Unable to open --{}: {:?}", BASE_DIR_FLAG, e))?;
let mut wallet = mgr
.wallet_by_name(&name)
.map_err(|e| format!("Unable to open wallet: {:?}", e))?;
for i in 0..n {
let voting_password = random_password();
let withdrawal_password = random_password();
let keystores = wallet
.next_validator(
wallet_password.as_bytes(),
voting_password.as_bytes(),
withdrawal_password.as_bytes(),
)
.map_err(|e| format!("Unable to create validator keys: {:?}", e))?;
let voting_pubkey = keystores.voting.pubkey().to_string();
ValidatorDirBuilder::new(validator_dir.clone(), secrets_dir.clone())
.voting_keystore(keystores.voting, voting_password.as_bytes())
.withdrawal_keystore(keystores.withdrawal, withdrawal_password.as_bytes())
.create_eth1_tx_data(deposit_gwei, &spec)
.store_withdrawal_keystore(matches.is_present(STORE_WITHDRAW_FLAG))
.build()
.map_err(|e| format!("Unable to build validator directory: {:?}", e))?;
println!("{}/{}\t0x{}", i + 1, n, voting_pubkey);
}
Ok(())
}
/// Returns the number of validators that exist in the given `validator_dir`.
///
/// This function just assumes any file is a validator directory, making it likely to return a
/// higher number than accurate but never a lower one.
fn existing_validator_count<P: AsRef<Path>>(validator_dir: P) -> Result<usize, String> {
fs::read_dir(validator_dir.as_ref())
.map(|iter| iter.count())
.map_err(|e| format!("Unable to read {:?}: {}", validator_dir.as_ref(), e))
}

271
account_manager/src/validator/deposit.rs Normal file
View File

@@ -0,0 +1,271 @@
use crate::VALIDATOR_DIR_FLAG;
use clap::{App, Arg, ArgMatches};
use clap_utils;
use deposit_contract::DEPOSIT_GAS;
use environment::Environment;
use futures::compat::Future01CompatExt;
use slog::{info, Logger};
use std::path::PathBuf;
use tokio::time::{delay_until, Duration, Instant};
use types::EthSpec;
use validator_dir::Manager as ValidatorManager;
use web3::{
transports::Ipc,
types::{Address, SyncInfo, SyncState, TransactionRequest, U256},
Transport, Web3,
};
pub const CMD: &str = "deposit";
pub const VALIDATOR_FLAG: &str = "validator";
pub const ETH1_IPC_FLAG: &str = "eth1-ipc";
pub const FROM_ADDRESS_FLAG: &str = "from-address";
const GWEI: u64 = 1_000_000_000;
const SYNCING_STATE_RETRY_DELAY: Duration = Duration::from_secs(2);
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new("deposit")
.about(
"Submits a deposit to an Eth1 validator registration contract via an IPC endpoint \
of an Eth1 client (e.g., Geth, OpenEthereum, etc.). The validators must already \
have been created and exist on the file-system. The process will exit immediately \
with an error if any error occurs. After each deposit is submitted to the Eth1 \
node, a file will be saved in the validator directory with the transaction hash. \
The application does not wait for confirmations so there is not guarantee that \
the transaction is included in the Eth1 chain; use a block explorer and the \
transaction hash to check for confirmations. The deposit contract address will \
be determined by the --testnet-dir flag on the primary Lighthouse binary.",
)
.arg(
Arg::with_name(VALIDATOR_DIR_FLAG)
.long(VALIDATOR_DIR_FLAG)
.value_name("VALIDATOR_DIRECTORY")
.help(
"The path the validator client data directory. \
Defaults to ~/.lighthouse/validators",
)
.takes_value(true),
)
.arg(
Arg::with_name(VALIDATOR_FLAG)
.long(VALIDATOR_FLAG)
.value_name("VALIDATOR_NAME")
.help(
"The name of the directory in --data-dir for which to deposit. \
Set to 'all' to deposit all validators in the --data-dir.",
)
.takes_value(true)
.required(true),
)
.arg(
Arg::with_name(ETH1_IPC_FLAG)
.long(ETH1_IPC_FLAG)
.value_name("ETH1_IPC_PATH")
.help("Path to an Eth1 JSON-RPC IPC endpoint")
.takes_value(true)
.required(true),
)
.arg(
Arg::with_name(FROM_ADDRESS_FLAG)
.long(FROM_ADDRESS_FLAG)
.value_name("FROM_ETH1_ADDRESS")
.help(
"The address that will submit the eth1 deposit. \
Must be unlocked on the node at --eth1-ipc.",
)
.takes_value(true)
.required(true),
)
}
pub fn cli_run<T: EthSpec>(
matches: &ArgMatches<'_>,
mut env: Environment<T>,
) -> Result<(), String> {
let log = env.core_context().log;
let data_dir = clap_utils::parse_path_with_default_in_home_dir(
matches,
VALIDATOR_DIR_FLAG,
PathBuf::new().join(".lighthouse").join("validators"),
)?;
let validator: String = clap_utils::parse_required(matches, VALIDATOR_FLAG)?;
let eth1_ipc_path: PathBuf = clap_utils::parse_required(matches, ETH1_IPC_FLAG)?;
let from_address: Address = clap_utils::parse_required(matches, FROM_ADDRESS_FLAG)?;
let manager = ValidatorManager::open(&data_dir)
.map_err(|e| format!("Unable to read --{}: {:?}", VALIDATOR_DIR_FLAG, e))?;
let validators = match validator.as_ref() {
"all" => manager
.open_all_validators()
.map_err(|e| format!("Unable to read all validators: {:?}", e)),
name => {
let path = manager
.directory_names()
.map_err(|e| {
format!(
"Unable to read --{} directory names: {:?}",
VALIDATOR_DIR_FLAG, e
)
})?
.get(name)
.ok_or_else(|| format!("Unknown validator: {}", name))?
.clone();
manager
.open_validator(&path)
.map_err(|e| format!("Unable to open {}: {:?}", name, e))
.map(|v| vec![v])
}
}?;
let eth1_deposit_datas = validators
.into_iter()
.filter(|v| !v.eth1_deposit_tx_hash_exists())
.map(|v| match v.eth1_deposit_data() {
Ok(Some(data)) => Ok((v, data)),
Ok(None) => Err(format!(
"Validator is missing deposit data file: {:?}",
v.dir()
)),
Err(e) => Err(format!(
"Unable to read deposit data for {:?}: {:?}",
v.dir(),
e
)),
})
.collect::<Result<Vec<_>, _>>()?;
let total_gwei: u64 = eth1_deposit_datas
.iter()
.map(|(_, d)| d.deposit_data.amount)
.sum();
if eth1_deposit_datas.is_empty() {
info!(log, "No validators to deposit");
return Ok(());
}
info!(
log,
"Starting deposits";
"deposit_count" => eth1_deposit_datas.len(),
"total_eth" => total_gwei / GWEI,
);
let deposit_contract = env
.testnet
.as_ref()
.ok_or_else(|| "Unable to run account manager without a testnet dir".to_string())?
.deposit_contract_address()
.map_err(|e| format!("Unable to parse deposit contract address: {}", e))?;
if deposit_contract == Address::zero() {
return Err("Refusing to deposit to the zero address. Check testnet configuration.".into());
}
let (_event_loop_handle, transport) =
Ipc::new(eth1_ipc_path).map_err(|e| format!("Unable to connect to eth1 IPC: {:?}", e))?;
let web3 = Web3::new(transport);
let deposits_fut = async {
poll_until_synced(web3.clone(), log.clone()).await?;
for (mut validator_dir, eth1_deposit_data) in eth1_deposit_datas {
let tx_hash = web3
.eth()
.send_transaction(TransactionRequest {
from: from_address,
to: Some(deposit_contract),
gas: Some(DEPOSIT_GAS.into()),
gas_price: None,
value: Some(from_gwei(eth1_deposit_data.deposit_data.amount)),
data: Some(eth1_deposit_data.rlp.into()),
nonce: None,
condition: None,
})
.compat()
.await
.map_err(|e| format!("Failed to send transaction: {:?}", e))?;
validator_dir
.save_eth1_deposit_tx_hash(&format!("{:?}", tx_hash))
.map_err(|e| format!("Failed to save tx hash {:?} to disk: {:?}", tx_hash, e))?;
}
Ok::<(), String>(())
};
env.runtime().block_on(deposits_fut)?;
Ok(())
}
/// Converts gwei to wei.
fn from_gwei(gwei: u64) -> U256 {
U256::from(gwei) * U256::exp10(9)
}
/// Run a poll on the `eth_syncing` endpoint, blocking until the node is synced.
async fn poll_until_synced<T>(web3: Web3<T>, log: Logger) -> Result<(), String>
where
T: Transport + Send + 'static,
<T as Transport>::Out: Send,
{
loop {
let sync_state = web3
.clone()
.eth()
.syncing()
.compat()
.await
.map_err(|e| format!("Unable to read syncing state from eth1 node: {:?}", e))?;
match sync_state {
SyncState::Syncing(SyncInfo {
current_block,
highest_block,
..
}) => {
info!(
log,
"Waiting for eth1 node to sync";
"est_highest_block" => format!("{}", highest_block),
"current_block" => format!("{}", current_block),
);
delay_until(Instant::now() + SYNCING_STATE_RETRY_DELAY).await;
}
SyncState::NotSyncing => {
let block_number = web3
.clone()
.eth()
.block_number()
.compat()
.await
.map_err(|e| format!("Unable to read block number from eth1 node: {:?}", e))?;
if block_number > 0.into() {
info!(
log,
"Eth1 node is synced";
"head_block" => format!("{}", block_number),
);
break;
} else {
delay_until(Instant::now() + SYNCING_STATE_RETRY_DELAY).await;
info!(
log,
"Waiting for eth1 node to sync";
"current_block" => 0,
);
}
}
}
}
Ok(())
}

38
account_manager/src/validator/mod.rs Normal file
View File

@@ -0,0 +1,38 @@
pub mod create;
pub mod deposit;
use crate::common::base_wallet_dir;
use clap::{App, Arg, ArgMatches};
use environment::Environment;
use types::EthSpec;
pub const CMD: &str = "validator";
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new(CMD)
.about("Provides commands for managing Eth2 validators.")
.arg(
Arg::with_name("base-dir")
.long("base-dir")
.value_name("BASE_DIRECTORY")
.help("A path containing Eth2 EIP-2386 wallets. Defaults to ~/.lighthouse/wallets")
.takes_value(true),
)
.subcommand(create::cli_app())
.subcommand(deposit::cli_app())
}
pub fn cli_run<T: EthSpec>(matches: &ArgMatches, env: Environment<T>) -> Result<(), String> {
let base_wallet_dir = base_wallet_dir(matches, "base-dir")?;
match matches.subcommand() {
(create::CMD, Some(matches)) => create::cli_run::<T>(matches, env, base_wallet_dir),
(deposit::CMD, Some(matches)) => deposit::cli_run::<T>(matches, env),
(unknown, _) => {
return Err(format!(
"{} does not have a {} command. See --help",
CMD, unknown
));
}
}
}

163
account_manager/src/wallet/create.rs Normal file
View File

@@ -0,0 +1,163 @@
use crate::{common::random_password, BASE_DIR_FLAG};
use clap::{App, Arg, ArgMatches};
use eth2_wallet::{
bip39::{Language, Mnemonic, MnemonicType},
PlainText,
};
use eth2_wallet_manager::{WalletManager, WalletType};
use std::ffi::OsStr;
use std::fs::{self, File};
use std::io::prelude::*;
use std::os::unix::fs::PermissionsExt;
use std::path::{Path, PathBuf};
pub const CMD: &str = "create";
pub const HD_TYPE: &str = "hd";
pub const NAME_FLAG: &str = "name";
pub const PASSPHRASE_FLAG: &str = "passphrase-file";
pub const TYPE_FLAG: &str = "type";
pub const MNEMONIC_FLAG: &str = "mnemonic-output-path";
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new(CMD)
.about("Creates a new HD (hierarchical-deterministic) EIP-2386 wallet.")
.arg(
Arg::with_name(NAME_FLAG)
.long(NAME_FLAG)
.value_name("WALLET_NAME")
.help(
"The wallet will be created with this name. It is not allowed to \
create two wallets with the same name for the same --base-dir.",
)
.takes_value(true)
.required(true),
)
.arg(
Arg::with_name(PASSPHRASE_FLAG)
.long(PASSPHRASE_FLAG)
.value_name("WALLET_PASSWORD_PATH")
.help(
"A path to a file containing the password which will unlock the wallet. \
If the file does not exist, a random password will be generated and \
saved at that path. To avoid confusion, if the file does not already \
exist it must include a '.pass' suffix.",
)
.takes_value(true)
.required(true),
)
.arg(
Arg::with_name(TYPE_FLAG)
.long(TYPE_FLAG)
.value_name("WALLET_TYPE")
.help(
"The type of wallet to create. Only HD (hierarchical-deterministic) \
wallets are supported presently..",
)
.takes_value(true)
.possible_values(&[HD_TYPE])
.default_value(HD_TYPE),
)
.arg(
Arg::with_name(MNEMONIC_FLAG)
.long(MNEMONIC_FLAG)
.value_name("MNEMONIC_PATH")
.help(
"If present, the mnemonic will be saved to this file. DO NOT SHARE THE MNEMONIC.",
)
.takes_value(true)
)
}
pub fn cli_run(matches: &ArgMatches, base_dir: PathBuf) -> Result<(), String> {
let name: String = clap_utils::parse_required(matches, NAME_FLAG)?;
let wallet_password_path: PathBuf = clap_utils::parse_required(matches, PASSPHRASE_FLAG)?;
let mnemonic_output_path: Option<PathBuf> = clap_utils::parse_optional(matches, MNEMONIC_FLAG)?;
let type_field: String = clap_utils::parse_required(matches, TYPE_FLAG)?;
let wallet_type = match type_field.as_ref() {
HD_TYPE => WalletType::Hd,
unknown => return Err(format!("--{} {} is not supported", TYPE_FLAG, unknown)),
};
let mgr = WalletManager::open(&base_dir)
.map_err(|e| format!("Unable to open --{}: {:?}", BASE_DIR_FLAG, e))?;
// Create a new random mnemonic.
//
// The `tiny-bip39` crate uses `thread_rng()` for this entropy.
let mnemonic = Mnemonic::new(MnemonicType::Words12, Language::English);
// Create a random password if the file does not exist.
if !wallet_password_path.exists() {
// To prevent users from accidentally supplying their password to the PASSPHRASE_FLAG and
// create a file with that name, we require that the password has a .pass suffix.
if wallet_password_path.extension() != Some(&OsStr::new("pass")) {
return Err(format!(
"Only creates a password file if that file ends in .pass: {:?}",
wallet_password_path
));
}
create_with_600_perms(&wallet_password_path, random_password().as_bytes())
.map_err(|e| format!("Unable to write to {:?}: {:?}", wallet_password_path, e))?;
}
let wallet_password = fs::read(&wallet_password_path)
.map_err(|e| format!("Unable to read {:?}: {:?}", wallet_password_path, e))
.map(|bytes| PlainText::from(bytes))?;
let wallet = mgr
.create_wallet(name, wallet_type, &mnemonic, wallet_password.as_bytes())
.map_err(|e| format!("Unable to create wallet: {:?}", e))?;
if let Some(path) = mnemonic_output_path {
create_with_600_perms(&path, mnemonic.phrase().as_bytes())
.map_err(|e| format!("Unable to write mnemonic to {:?}: {:?}", path, e))?;
}
println!("Your wallet's 12-word BIP-39 mnemonic is:");
println!("");
println!("\t{}", mnemonic.phrase());
println!("");
println!("This mnemonic can be used to fully restore your wallet, should ");
println!("you lose the JSON file or your password. ");
println!("");
println!("It is very important that you DO NOT SHARE this mnemonic as it will ");
println!("reveal the private keys of all validators and keys generated with ");
println!("this wallet. That would be catastrophic.");
println!("");
println!("It is also import to store a backup of this mnemonic so you can ");
println!("recover your private keys in the case of data loss. Writing it on ");
println!("a piece of paper and storing it in a safe place would be prudent.");
println!("");
println!("Your wallet's UUID is:");
println!("");
println!("\t{}", wallet.wallet().uuid());
println!("");
println!("You do not need to backup your UUID or keep it secret.");
Ok(())
}
/// Creates a file with `600 (-rw-------)` permissions.
pub fn create_with_600_perms<P: AsRef<Path>>(path: P, bytes: &[u8]) -> Result<(), String> {
let path = path.as_ref();
let mut file =
File::create(&path).map_err(|e| format!("Unable to create {:?}: {}", path, e))?;
let mut perm = file
.metadata()
.map_err(|e| format!("Unable to get {:?} metadata: {}", path, e))?
.permissions();
perm.set_mode(0o600);
file.set_permissions(perm)
.map_err(|e| format!("Unable to set {:?} permissions: {}", path, e))?;
file.write_all(bytes)
.map_err(|e| format!("Unable to write to {:?}: {}", path, e))?;
Ok(())
}

24
account_manager/src/wallet/list.rs Normal file
View File

@@ -0,0 +1,24 @@
use crate::BASE_DIR_FLAG;
use clap::App;
use eth2_wallet_manager::WalletManager;
use std::path::PathBuf;
pub const CMD: &str = "list";
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new(CMD).about("Lists the names of all wallets.")
}
pub fn cli_run(base_dir: PathBuf) -> Result<(), String> {
let mgr = WalletManager::open(&base_dir)
.map_err(|e| format!("Unable to open --{}: {:?}", BASE_DIR_FLAG, e))?;
for (name, _uuid) in mgr
.wallets()
.map_err(|e| format!("Unable to list wallets: {:?}", e))?
{
println!("{}", name)
}
Ok(())
}

40
account_manager/src/wallet/mod.rs Normal file
View File

@@ -0,0 +1,40 @@
pub mod create;
pub mod list;
use crate::{
common::{base_wallet_dir, ensure_dir_exists},
BASE_DIR_FLAG,
};
use clap::{App, Arg, ArgMatches};
pub const CMD: &str = "wallet";
pub fn cli_app<'a, 'b>() -> App<'a, 'b> {
App::new(CMD)
.about("TODO")
.arg(
Arg::with_name(BASE_DIR_FLAG)
.long(BASE_DIR_FLAG)
.value_name("BASE_DIRECTORY")
.help("A path containing Eth2 EIP-2386 wallets. Defaults to ~/.lighthouse/wallets")
.takes_value(true),
)
.subcommand(create::cli_app())
.subcommand(list::cli_app())
}
pub fn cli_run(matches: &ArgMatches) -> Result<(), String> {
let base_dir = base_wallet_dir(matches, BASE_DIR_FLAG)?;
ensure_dir_exists(&base_dir)?;
match matches.subcommand() {
(create::CMD, Some(matches)) => create::cli_run(matches, base_dir),
(list::CMD, Some(_)) => list::cli_run(base_dir),
(unknown, _) => {
return Err(format!(
"{} does not have a {} command. See --help",
CMD, unknown
));
}
}
}