Files
lighthouse/tests/eth1_test_rig/src/lib.rs
Paul Hauner a0549e3842 Prepare for public testnet (#628)
* Update to spec v0.9.0

* Update to v0.9.1

* Bump spec tags for v0.9.1

* Formatting, fix CI failures

* Resolve accidental KeyPair merge conflict

* Document new BeaconState functions

* Add `validator` changes from `validator-to-rest`

* Add initial (failing) REST api tests

* Fix signature parsing

* Add more tests

* Refactor http router

* Add working tests for publish beacon block

* Add validator duties tests

* Move account_manager under `lighthouse` binary

* Unify logfile handling in `environment` crate.

* Fix incorrect cache drops in `advance_caches`

* Update fork choice for v0.9.1

* Add `deposit_contract` crate

* Add progress on validator onboarding

* Add unfinished attesation code

* Update account manager CLI

* Write eth1 data file as hex string

* Integrate ValidatorDirectory with validator_client

* Move ValidatorDirectory into validator_client

* Clean up some FIXMEs

* Add beacon_chain_sim

* Fix a few docs/logs

* Expand `beacon_chain_sim`

* Fix spec for `beacon_chain_sim

* More testing for api

* Start work on attestation endpoint

* Reject empty attestations

* Allow attestations to genesis block

* Add working tests for `rest_api` validator endpoint

* Remove grpc from beacon_node

* Start heavy refactor of validator client

- Block production is working

* Prune old validator client files

* Start works on attestation service

* Add attestation service to validator client

* Use full pubkey for validator directories

* Add validator duties post endpoint

* Use par_iter for keypair generation

* Use bulk duties request in validator client

* Add version http endpoint tests

* Add interop keys and startup wait

* Ensure a prompt exit

* Add duties pruning

* Fix compile error in beacon node tests

* Add github workflow

* Modify rust.yaml

* Modify gitlab actions

* Add to CI file

* Add sudo to CI npm install

* Move cargo fmt to own job in tests

* Fix cargo fmt in CI

* Add rustup update before cargo fmt

* Change name of CI job

* Make other CI jobs require cargo fmt

* Add CI badge

* Remove gitlab and travis files

* Add different http timeout for debug

* Update docker file, use makefile in CI

* Use make in the dockerfile, skip the test

* Use the makefile for debug GI test

* Update book

* Tidy grpc and misc things

* Apply discv5 fixes

* Address other minor issues

* Fix warnings

* Attempt fix for addr parsing

* Tidy validator config, CLIs

* Tidy comments

* Tidy signing, reduce ForkService duplication

* Fail if skipping too many slots

* Set default recent genesis time to 0

* Add custom http timeout to validator

* Fix compile bug in node_test_rig

* Remove old bootstrap flag from val CLI

* Update docs

* Tidy val client

* Change val client log levels

* Add comments, more validity checks

* Fix compile error, add comments

* Undo changes to eth2-libp2p/src

* Reduce duplication of keypair generation

* Add more logging for validator duties

* Fix beacon_chain_sim, nitpicks

* Fix compile error, minor nits

* Update to use v0.9.2 version of deposit contract

* Add efforts to automate eth1 testnet deployment

* Fix lcli testnet deployer

* Modify bn CLI to parse eth2_testnet_dir

* Progress with account_manager deposit tools

* Make account manager submit deposits

* Add password option for submitting deposits

* Allow custom deposit amount

* Add long names to lcli clap

* Add password option to lcli deploy command

* Fix minor bugs whilst testing

* Address Michael's comments

* Add refund-deposit-contract to lcli

* Use time instead of skip count for denying long skips

* Improve logging for eth1

* Fix bug with validator services exiting on error

* Drop the block cache after genesis

* Modify eth1 testnet config

* Improve eth1 logging

* Make validator wait until genesis time

* Fix bug in eth1 voting

* Add more logging to eth1 voting

* Handle errors in eth1 http module

* Set SECONDS_PER_DAY to sensible minimum

* Shorten delay before testnet start

* Ensure eth1 block is produced without any votes

* Improve eth1 logging

* Fix broken tests in eth1

* Tidy code in rest_api

* Fix failing test in deposit_contract

* Make CLI args more consistent

* Change validator/duties endpoint

* Add time-based skip slot limiting

* Add new error type missed in previous commit

* Add log when waiting for genesis

* Refactor beacon node CLI

* Remove unused dep

* Add lcli eth1-genesis command

* Fix bug in master merge

* Apply clippy lints to beacon node

* Add support for YamlConfig in Eth2TestnetDir

* Upgrade tesnet deposit contract version

* Remove unnecessary logging and correct formatting

* Add a hardcoded eth2 testnet config

* Ensure http server flag works. Overwrite configs with flags.

* Ensure boot nodes are loaded from testnet dir

* Fix account manager CLI bugs

* Fix bugs with beacon node cli

* Allow testnet dir without boot nodes

* Write genesis state as SSZ

* Remove ---/n from the start of testnet_dir files

* Set default libp2p address

* Tidy account manager CLI, add logging

* Add check to see if testnet dir exists

* Apply reviewers suggestions

* Add HeadTracker struct

* Add fork choice persistence

* Shorten slot time for simulator

* Add the /beacon/heads API endpoint

* Update hardcoded testnet

* Add tests for BeaconChain persistence + fix bugs

* Extend BeaconChain persistence testing

* Ensure chain is finalized b4 persistence tests

* Ensure boot_enr.yaml is include in binary

* Refactor beacon_chain_sim

* Move files about in beacon sim

* Update beacon_chain_sim

* Fix bug with deposit inclusion

* Increase log in genesis service, fix todo

* Tidy sim, fix broken rest_api tests

* Fix more broken tests

* Update testnet

* Fix broken rest api test

* Tidy account manager CLI

* Use tempdir for account manager

* Stop hardcoded testnet dir from creating dir

* Rename Eth2TestnetDir to Eth2TestnetConfig

* Change hardcoded -> hard_coded

* Tidy account manager

* Add log to account manager

* Tidy, ensure head tracker is loaded from disk

* Tidy beacon chain builder

* Tidy eth1_chain

* Adds log support for simulator

* Revert "Adds log support for simulator"

This reverts commit ec77c66a05.

* Adds log support for simulator

* Tidy after self-review

* Change default log level

* Address Michael's delicious PR comments

* Fix off-by-one in tests
2019-12-03 15:28:57 +11:00

319 lines
11 KiB
Rust

//! Provides utilities for deploying and manipulating the eth2 deposit contract on the eth1 chain.
//!
//! Presently used with [`ganache-cli`](https://github.com/trufflesuite/ganache-cli) to simulate
//! the deposit contract for testing beacon node eth1 integration.
//!
//! Not tested to work with actual clients (e.g., geth). It should work fine, however there may be
//! some initial issues.
mod ganache;
use deposit_contract::{eth1_tx_data, testnet, ABI, BYTECODE, CONTRACT_DEPLOY_GAS, DEPOSIT_GAS};
use futures::{future, stream, Future, IntoFuture, Stream};
use ganache::GanacheInstance;
use std::time::{Duration, Instant};
use tokio::{runtime::Runtime, timer::Delay};
use types::DepositData;
use types::{test_utils::generate_deterministic_keypair, EthSpec, Hash256, Keypair, Signature};
use web3::contract::{Contract, Options};
use web3::transports::Http;
use web3::types::{Address, TransactionRequest, U256};
use web3::Web3;
pub const DEPLOYER_ACCOUNTS_INDEX: usize = 0;
pub const DEPOSIT_ACCOUNTS_INDEX: usize = 0;
/// Provides a dedicated ganache-cli instance with the deposit contract already deployed.
pub struct GanacheEth1Instance {
pub ganache: GanacheInstance,
pub deposit_contract: DepositContract,
}
impl GanacheEth1Instance {
pub fn new() -> impl Future<Item = Self, Error = String> {
GanacheInstance::new().into_future().and_then(|ganache| {
DepositContract::deploy(ganache.web3.clone(), 0, None).map(|deposit_contract| Self {
ganache,
deposit_contract,
})
})
}
pub fn endpoint(&self) -> String {
self.ganache.endpoint()
}
pub fn web3(&self) -> Web3<Http> {
self.ganache.web3.clone()
}
}
/// Deploys and provides functions for the eth2 deposit contract, deployed on the eth1 chain.
#[derive(Clone, Debug)]
pub struct DepositContract {
web3: Web3<Http>,
contract: Contract<Http>,
}
impl DepositContract {
pub fn deploy(
web3: Web3<Http>,
confirmations: usize,
password: Option<String>,
) -> impl Future<Item = Self, Error = String> {
Self::deploy_bytecode(web3, confirmations, BYTECODE, ABI, password)
}
pub fn deploy_testnet(
web3: Web3<Http>,
confirmations: usize,
password: Option<String>,
) -> impl Future<Item = Self, Error = String> {
Self::deploy_bytecode(
web3,
confirmations,
testnet::BYTECODE,
testnet::ABI,
password,
)
}
fn deploy_bytecode(
web3: Web3<Http>,
confirmations: usize,
bytecode: &[u8],
abi: &[u8],
password: Option<String>,
) -> impl Future<Item = Self, Error = String> {
let web3_1 = web3.clone();
deploy_deposit_contract(
web3.clone(),
confirmations,
bytecode.to_vec(),
abi.to_vec(),
password,
)
.map_err(|e| {
format!(
"Failed to deploy contract: {}. Is scripts/ganache_tests_node.sh running?.",
e
)
})
.and_then(move |address| {
Contract::from_json(web3_1.eth(), address, ABI)
.map_err(|e| format!("Failed to init contract: {:?}", e))
})
.map(|contract| Self { contract, web3 })
}
/// The deposit contract's address in `0x00ab...` format.
pub fn address(&self) -> String {
format!("0x{:x}", self.contract.address())
}
/// A helper to return a fully-formed `DepositData`. Does not submit the deposit data to the
/// smart contact.
pub fn deposit_helper<E: EthSpec>(
&self,
keypair: Keypair,
withdrawal_credentials: Hash256,
amount: u64,
) -> DepositData {
let mut deposit = DepositData {
pubkey: keypair.pk.into(),
withdrawal_credentials,
amount,
signature: Signature::empty_signature().into(),
};
deposit.signature = deposit.create_signature(&keypair.sk, &E::default_spec());
deposit
}
/// Creates a random, valid deposit and submits it to the deposit contract.
///
/// The keypairs are created randomly and destroyed.
pub fn deposit_random<E: EthSpec>(&self, runtime: &mut Runtime) -> Result<(), String> {
let keypair = Keypair::random();
let mut deposit = DepositData {
pubkey: keypair.pk.into(),
withdrawal_credentials: Hash256::zero(),
amount: 32_000_000_000,
signature: Signature::empty_signature().into(),
};
deposit.signature = deposit.create_signature(&keypair.sk, &E::default_spec());
self.deposit(runtime, deposit)
}
/// Perfoms a blocking deposit.
pub fn deposit(&self, runtime: &mut Runtime, deposit_data: DepositData) -> Result<(), String> {
runtime
.block_on(self.deposit_async(deposit_data))
.map_err(|e| format!("Deposit failed: {:?}", e))
}
pub fn deposit_deterministic_async<E: EthSpec>(
&self,
keypair_index: usize,
amount: u64,
) -> impl Future<Item = (), Error = String> {
let keypair = generate_deterministic_keypair(keypair_index);
let mut deposit = DepositData {
pubkey: keypair.pk.into(),
withdrawal_credentials: Hash256::zero(),
amount,
signature: Signature::empty_signature().into(),
};
deposit.signature = deposit.create_signature(&keypair.sk, &E::default_spec());
self.deposit_async(deposit)
}
/// Performs a non-blocking deposit.
pub fn deposit_async(
&self,
deposit_data: DepositData,
) -> impl Future<Item = (), Error = String> {
let contract = self.contract.clone();
let web3_1 = self.web3.clone();
self.web3
.eth()
.accounts()
.map_err(|e| format!("Failed to get accounts: {:?}", e))
.and_then(|accounts| {
accounts
.get(DEPOSIT_ACCOUNTS_INDEX)
.cloned()
.ok_or_else(|| "Insufficient accounts for deposit".to_string())
})
.and_then(move |from| {
let tx_request = TransactionRequest {
from,
to: Some(contract.address()),
gas: Some(U256::from(DEPOSIT_GAS)),
gas_price: None,
value: Some(from_gwei(deposit_data.amount)),
// Note: the reason we use this `TransactionRequest` instead of just using the
// function in `self.contract` is so that the `eth1_tx_data` function gets used
// during testing.
//
// It's important that `eth1_tx_data` stays correct and does not suffer from
// code-rot.
data: eth1_tx_data(&deposit_data).map(Into::into).ok(),
nonce: None,
condition: None,
};
web3_1
.eth()
.send_transaction(tx_request)
.map_err(|e| format!("Failed to call deposit fn: {:?}", e))
})
.map(|_| ())
}
/// Peforms many deposits, each preceded by a delay.
pub fn deposit_multiple(
&self,
deposits: Vec<DelayThenDeposit>,
) -> impl Future<Item = (), Error = String> {
let s = self.clone();
stream::unfold(deposits.into_iter(), move |mut deposit_iter| {
let s = s.clone();
match deposit_iter.next() {
Some(deposit) => Some(
Delay::new(Instant::now() + deposit.delay)
.map_err(|e| format!("Failed to execute delay: {:?}", e))
.and_then(move |_| s.deposit_async(deposit.deposit))
.map(move |yielded| (yielded, deposit_iter)),
),
None => None,
}
})
.collect()
.map(|_| ())
}
}
/// Describes a deposit and a delay that should should precede it's submission to the deposit
/// contract.
#[derive(Clone)]
pub struct DelayThenDeposit {
/// Wait this duration ...
pub delay: Duration,
/// ... then submit this deposit.
pub deposit: DepositData,
}
fn from_gwei(gwei: u64) -> U256 {
U256::from(gwei) * U256::exp10(9)
}
/// Deploys the deposit contract to the given web3 instance using the account with index
/// `DEPLOYER_ACCOUNTS_INDEX`.
fn deploy_deposit_contract(
web3: Web3<Http>,
confirmations: usize,
bytecode: Vec<u8>,
abi: Vec<u8>,
password_opt: Option<String>,
) -> impl Future<Item = Address, Error = String> {
let bytecode = String::from_utf8(bytecode).expect("bytecode must be valid utf8");
let web3_1 = web3.clone();
web3.eth()
.accounts()
.map_err(|e| format!("Failed to get accounts: {:?}", e))
.and_then(|accounts| {
accounts
.get(DEPLOYER_ACCOUNTS_INDEX)
.cloned()
.ok_or_else(|| "Insufficient accounts for deployer".to_string())
})
.and_then(move |from_address| {
let future: Box<dyn Future<Item = Address, Error = String> + Send> =
if let Some(password) = password_opt {
// Unlock for only a single transaction.
let duration = None;
let future = web3_1
.personal()
.unlock_account(from_address, &password, duration)
.then(move |result| match result {
Ok(true) => Ok(from_address),
Ok(false) => Err("Eth1 node refused to unlock account".to_string()),
Err(e) => Err(format!("Eth1 unlock request failed: {:?}", e)),
});
Box::new(future)
} else {
Box::new(future::ok(from_address))
};
future
})
.and_then(move |deploy_address| {
Contract::deploy(web3.eth(), &abi)
.map_err(|e| format!("Unable to build contract deployer: {:?}", e))?
.confirmations(confirmations)
.options(Options {
gas: Some(U256::from(CONTRACT_DEPLOY_GAS)),
..Options::default()
})
.execute(bytecode, (), deploy_address)
.map_err(|e| format!("Failed to execute deployment: {:?}", e))
})
.and_then(|pending_contract| {
pending_contract
.map(|contract| contract.address())
.map_err(|e| format!("Unable to resolve pending contract: {:?}", e))
})
}