Introduce validator definition file for VC (#1357)

## Issue Addressed

NA

## Proposed Changes

- Introduces the `valdiator_definitions.yml` file which serves as an explicit list of validators that should be run by the validator client.
  - Removes `--strict` flag, split into `--strict-lockfiles` and `--disable-auto-discover`  
  - Adds a "Validator Management" page to the book.
- Adds the `common/account_utils` crate which contains some logic that was starting to duplicate across the codebase.

The new docs for this feature are the best description of it (apart from the code, I guess): 9cb87e93ce/book/src/validator-management.md

## API Changes

This change should be transparent for *most* existing users. If the `valdiator_definitions.yml` doesn't exist then it will be automatically generated using a method that will detect all the validators in their `validators_dir`.

Users will have issues if they are:

1. Using `--strict`.
1. Have keystores in their `~/.lighthouse/validators` directory that weren't being detected by the current keystore discovery method.

For users with (1), the VC will refuse to start because the `--strict` flag has been removed. They will be forced to review `--help` and choose an equivalent flag.

For users with (2), this seems fairly unlikely and since we're only in testnets there's no *real* value on the line here. I'm happy to take the risk, it would be a different case for mainnet.

## Additional Info

This PR adds functionality we will need for #1347.

## TODO

- [x] Reconsider flags
- [x] Move doc into a more reasonable chapter.
- [x] Check for compile warnings.
This commit is contained in:
Paul Hauner
2020-07-22 09:34:55 +00:00
parent 393782f632
commit e26da35cbf
19 changed files with 1117 additions and 153 deletions

View File

@@ -0,0 +1,15 @@
[package]
name = "account_utils"
version = "0.1.0"
authors = ["Paul Hauner <paul@paulhauner.com>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
rand = "0.7.2"
eth2_wallet = { path = "../../crypto/eth2_wallet" }
eth2_keystore = { path = "../../crypto/eth2_keystore" }
zeroize = { version = "1.0.0", features = ["zeroize_derive"] }
serde = "1.0.110"
serde_derive = "1.0.110"

View File

@@ -0,0 +1,151 @@
//! Provides functions that are used for key/account management across multiple crates in the
//! Lighthouse project.
use eth2_keystore::Keystore;
use eth2_wallet::Wallet;
use rand::{distributions::Alphanumeric, Rng};
use serde_derive::{Deserialize, Serialize};
use std::fs::{self, File};
use std::io;
use std::io::prelude::*;
use std::os::unix::fs::PermissionsExt;
use std::path::{Path, PathBuf};
use zeroize::Zeroize;
pub use eth2_wallet::PlainText;
/// The `Alphanumeric` crate only generates a-z, A-Z, 0-9, therefore it has a range of 62
/// characters.
///
/// 62**48 is greater than 255**32, therefore this password has more bits of entropy than a byte
/// array of length 32.
const DEFAULT_PASSWORD_LEN: usize = 48;
/// Returns the "default" path where a wallet should store its password file.
pub fn default_wallet_password_path<P: AsRef<Path>>(wallet_name: &str, secrets_dir: P) -> PathBuf {
secrets_dir.as_ref().join(format!("{}.pass", wallet_name))
}
/// Returns a password for a wallet, where that password is loaded from the "default" path.
pub fn default_wallet_password<P: AsRef<Path>>(
wallet: &Wallet,
secrets_dir: P,
) -> Result<PlainText, io::Error> {
let path = default_wallet_password_path(wallet.name(), secrets_dir);
fs::read(path).map(|bytes| PlainText::from(strip_off_newlines(bytes)))
}
/// Returns the "default" path where a keystore should store its password file.
pub fn default_keystore_password_path<P: AsRef<Path>>(
keystore: &Keystore,
secrets_dir: P,
) -> PathBuf {
secrets_dir
.as_ref()
.join(format!("0x{}", keystore.pubkey()))
}
/// Reads a password file into a Zeroize-ing `PlainText` struct, with new-lines removed.
pub fn read_password<P: AsRef<Path>>(path: P) -> Result<PlainText, io::Error> {
fs::read(path).map(strip_off_newlines).map(Into::into)
}
/// Creates a file with `600 (-rw-------)` permissions.
pub fn create_with_600_perms<P: AsRef<Path>>(path: P, bytes: &[u8]) -> Result<(), io::Error> {
let path = path.as_ref();
let mut file = File::create(&path)?;
let mut perm = file.metadata()?.permissions();
perm.set_mode(0o600);
file.set_permissions(perm)?;
file.write_all(bytes)?;
Ok(())
}
/// Generates a random alphanumeric password of length `DEFAULT_PASSWORD_LEN`.
pub fn random_password() -> PlainText {
rand::thread_rng()
.sample_iter(&Alphanumeric)
.take(DEFAULT_PASSWORD_LEN)
.collect::<String>()
.into_bytes()
.into()
}
/// Remove any number of newline or carriage returns from the end of a vector of bytes.
pub fn strip_off_newlines(mut bytes: Vec<u8>) -> Vec<u8> {
let mut strip_off = 0;
for (i, byte) in bytes.iter().rev().enumerate() {
if *byte == b'\n' || *byte == b'\r' {
strip_off = i + 1;
} else {
break;
}
}
bytes.truncate(bytes.len() - strip_off);
bytes
}
/// Provides a new-type wrapper around `String` that is zeroized on `Drop`.
///
/// Useful for ensuring that password memory is zeroed-out on drop.
#[derive(Clone, Serialize, Deserialize, Zeroize)]
#[zeroize(drop)]
#[serde(transparent)]
pub struct ZeroizeString(String);
impl From<String> for ZeroizeString {
fn from(s: String) -> Self {
Self(s)
}
}
impl AsRef<[u8]> for ZeroizeString {
fn as_ref(&self) -> &[u8] {
self.0.as_bytes()
}
}
#[cfg(test)]
mod test {
use super::strip_off_newlines;
#[test]
fn test_strip_off() {
let expected = "hello world".as_bytes().to_vec();
assert_eq!(
strip_off_newlines("hello world\n".as_bytes().to_vec()),
expected
);
assert_eq!(
strip_off_newlines("hello world\n\n\n\n".as_bytes().to_vec()),
expected
);
assert_eq!(
strip_off_newlines("hello world\r".as_bytes().to_vec()),
expected
);
assert_eq!(
strip_off_newlines("hello world\r\r\r\r\r".as_bytes().to_vec()),
expected
);
assert_eq!(
strip_off_newlines("hello world\r\n".as_bytes().to_vec()),
expected
);
assert_eq!(
strip_off_newlines("hello world\r\n\r\n".as_bytes().to_vec()),
expected
);
assert_eq!(
strip_off_newlines("hello world".as_bytes().to_vec()),
expected
);
}
}