gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-20 21:34:46 +00:00
Code Issues Packages Projects Releases Wiki Activity

Introduce validator definition file for VC (#1357)

Browse Source 
## Issue Addressed

NA

## Proposed Changes

- Introduces the `valdiator_definitions.yml` file which serves as an explicit list of validators that should be run by the validator client.
  - Removes `--strict` flag, split into `--strict-lockfiles` and `--disable-auto-discover`  
  - Adds a "Validator Management" page to the book.
- Adds the `common/account_utils` crate which contains some logic that was starting to duplicate across the codebase.

The new docs for this feature are the best description of it (apart from the code, I guess): 9cb87e93ce/book/src/validator-management.md

## API Changes

This change should be transparent for *most* existing users. If the `valdiator_definitions.yml` doesn't exist then it will be automatically generated using a method that will detect all the validators in their `validators_dir`.

Users will have issues if they are:

1. Using `--strict`.
1. Have keystores in their `~/.lighthouse/validators` directory that weren't being detected by the current keystore discovery method.

For users with (1), the VC will refuse to start because the `--strict` flag has been removed. They will be forced to review `--help` and choose an equivalent flag.

For users with (2), this seems fairly unlikely and since we're only in testnets there's no *real* value on the line here. I'm happy to take the risk, it would be a different case for mainnet.

## Additional Info

This PR adds functionality we will need for #1347.

## TODO

- [x] Reconsider flags
- [x] Move doc into a more reasonable chapter.
- [x] Check for compile warnings.
This commit is contained in:
Paul Hauner
2020-07-22 09:34:55 +00:00
parent 393782f632
commit e26da35cbf
19 changed files with 1117 additions and 153 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
book/src/SUMMARY.md
View File

@@ -10,6 +10,7 @@
* [Key Management](./key-managment.md)
* [Create a wallet](./wallet-create.md)
* [Create a validator](./validator-create.md)
* [Validator Management](./validator-management.md)
* [Local Testnets](./local-testnets.md)
* [API](./api.md)
* [HTTP (RESTful JSON)](./http.md)

187
book/src/validator-management.md Normal file
View File

@@ -0,0 +1,187 @@
# Validator Management
The `lighthouse vc` command starts a *validator client* instance which connects
to a beacon node performs the duties of a staked validator.
This document provides information on how the validator client discovers the
validators it will act for and how it should obtain their cryptographic
signatures.
Users that create validators using the `lighthouse account` tool in the
standard directories and do not start their `lighthouse vc` with the
`--disable-auto-discover` flag should not need to understand the contents of
this document. However, users with more complex needs may find this document
useful.
## Introducing the `validator_definitions.yml` file
The `validator_definitions.yml` file is located in the `validator-dir`, which
defaults to `~/.lighthouse/validators`. It is a
[YAML](https://en.wikipedia.org/wiki/YAML) encoded file defining exactly which
validators the validator client will (and won't) act for.
### Example
Here's an example file with two validators:
```yaml
---
- enabled: true
voting_public_key: "0x87a580d31d7bc69069b55f5a01995a610dd391a26dc9e36e81057a17211983a79266800ab8531f21f1083d7d84085007"
type: local_keystore
voting_keystore_path: /home/paul/.lighthouse/validators/0x87a580d31d7bc69069b55f5a01995a610dd391a26dc9e36e81057a17211983a79266800ab8531f21f1083d7d84085007/voting-keystore.json
voting_keystore_password_path: /home/paul/.lighthouse/secrets/0x87a580d31d7bc69069b55f5a01995a610dd391a26dc9e36e81057a17211983a79266800ab8531f21f1083d7d84085007
- enabled: false
voting_public_key: "0xa5566f9ec3c6e1fdf362634ebec9ef7aceb0e460e5079714808388e5d48f4ae1e12897fed1bea951c17fa389d511e477"
type: local_keystore
voting_keystore_path: /home/paul/.lighthouse/validators/0xa5566f9ec3c6e1fdf362634ebec9ef7aceb0e460e5079714808388e5d48f4ae1e12897fed1bea951c17fa389d511e477/voting-keystore.json
voting_keystore_password: myStrongpa55word123&$
```
In this example we can see two validators:
- A validator identified by the `0x87a5...` public key which is enabled.
- Another validator identified by the `0x0xa556...` public key which is **not** enabled.
### Fields
Each permitted field of the file is listed below for reference:
- `enabled`: A `true`/`false` indicating if the validator client should consider this
validator "enabled".
- `voting_public_key`: A validator public key.
- `type`: How the validator signs messages (currently restricted to `local_keystore`).
- `voting_keystore_path`: The path to a EIP-2335 keystore.
- `voting_keystore_password_path`: The path to the password for the EIP-2335 keystore.
- `voting_keystore_password`: The password to the EIP-2335 keystore.
> **Note**: Either `voting_keystore_password_path` or `voting_keystore_password` *must* be
> supplied. If both are supplied, `voting_keystore_password_path` is ignored.
## Populating the `validator_definitions.yml` file
When validator client starts and the `validator_definitions.yml` file doesn't
exist, a new file will be created. If the `--disable-auto-discover` flag is
provided, the new file will be empty and the validator client will not start
any validators. If the `--disable-auto-discover` flag is **not** provided, an
*automatic validator discovery* routine will start (more on that later). To
recap:
- `lighthouse vc`: validators are automatically discovered.
- `lighthouse vc --disable-auto-discover`: validators are **not** automatically discovered.
### Automatic validator discovery
When the `--disable-auto-discover` flag is **not** provided, the validator will search the
`validator-dir` for validators and add any *new* validators to the
`validator_definitions.yml` with `enabled: true`.
The routine for this search begins in the `validator-dir`, where it obtains a
list of all files in that directory and all sub-directories (i.e., recursive
directory-tree search). For each file named `voting-keystore.json` it creates a
new validator definition by the following process:
1. Set `enabled` to `true`.
1. Set `voting_public_key` to the `pubkey` value from the `voting-keystore.json`.
1. Set `type` to `local_keystore`.
1. Set `voting_keystore_path` to the full path of the discovered keystore.
1. Set `voting_keystore_password_path` to be a file in the `secrets-dir` with a
name identical to the `voting_public_key` value.
#### Discovery Example
Lets assume the following directory structure:
```
~/.lighthouse/validators
├── john
│   └── voting-keystore.json
├── sally
│   ├── one
│   │   └── voting-keystore.json
│   ├── three
│   │   └── my-voting-keystore.json
│   └── two
│   └── voting-keystore.json
└── slashing_protection.sqlite
```
There is no `validator_definitions.yml` file present, so we can run `lighthouse
vc` (**without** `--disable-auto-discover`) and it will create the following `validator_definitions.yml`:
```yaml
---
- enabled: true
voting_public_key: "0xa5566f9ec3c6e1fdf362634ebec9ef7aceb0e460e5079714808388e5d48f4ae1e12897fed1bea951c17fa389d511e477"
type: local_keystore
voting_keystore_path: /home/paul/.lighthouse/validators/sally/one/voting-keystore.json
voting_keystore_password_path: /home/paul/.lighthouse/secrets/0xa5566f9ec3c6e1fdf362634ebec9ef7aceb0e460e5079714808388e5d48f4ae1e12897fed1bea951c17fa389d511e477
- enabled: true
voting_public_key: "0xaa440c566fcf34dedf233baf56cf5fb05bb420d9663b4208272545608c27c13d5b08174518c758ecd814f158f2b4a337"
type: local_keystore
voting_keystore_path: /home/paul/.lighthouse/validators/sally/two/voting-keystore.json
voting_keystore_password_path: /home/paul/.lighthouse/secrets/0xaa440c566fcf34dedf233baf56cf5fb05bb420d9663b4208272545608c27c13d5b08174518c758ecd814f158f2b4a337
- enabled: true
voting_public_key: "0x87a580d31d7bc69069b55f5a01995a610dd391a26dc9e36e81057a17211983a79266800ab8531f21f1083d7d84085007"
type: local_keystore
voting_keystore_path: /home/paul/.lighthouse/validators/john/voting-keystore.json
voting_keystore_password_path: /home/paul/.lighthouse/secrets/0x87a580d31d7bc69069b55f5a01995a610dd391a26dc9e36e81057a17211983a79266800ab8531f21f1083d7d84085007
```
All `voting-keystore.json` files have been detected and added to the file.
Notably, the `sally/three/my-voting-keystore.json` file was *not* added to the
file, since the file name is not exactly `voting-keystore.json`.
In order for the validator client to decrypt the validators, they will need to
ensure their `secrets-dir` is organised as below:
```
~/.lighthouse/secrets
├── 0xa5566f9ec3c6e1fdf362634ebec9ef7aceb0e460e5079714808388e5d48f4ae1e12897fed1bea951c17fa389d511e477
├── 0xaa440c566fcf34dedf233baf56cf5fb05bb420d9663b4208272545608c27c13d5b08174518c758ecd814f158f2b4a337
└── 0x87a580d31d7bc69069b55f5a01995a610dd391a26dc9e36e81057a17211983a79266800ab8531f21f1083d7d84085007
```
### Manual configuration
The automatic validator discovery process works out-of-the-box with validators
that are created using the `lighthouse account validator new` command. The
details of this process are only interesting to those who are using keystores
generated with another tool or have a non-standard requirements.
If you are one of these users, manually edit the `validator_definitions.yml`
file to suit your requirements. If the file is poorly formatted or any one of
the validators is unable to be initialized, the validator client will refuse to
start.
## How the `validator_definitions.yml` file is processed
If a validator client were to start using the [first example
`validator_definitions.yml` file](#example) it would print the following log,
acknowledging there there are two validators and one is disabled:
```
INFO Initialized validators enabled: 1, disabled: 1
```
The validator client will simply ignore the disabled validator. However, for
the active validator, the validator client will:
1. Load an EIP-2335 keystore from the `voting_keystore_path`.
1. If the `voting_keystore_password` field is present, use it as the keystore
password. Otherwise, attempt to read the file at
`voting_keystore_password_path` and use the contents as the keystore
password.
1. Use the keystore password to decrypt the keystore and obtain a BLS keypair.
1. Verify that the decrypted BLS keypair matches the `voting_public_key`.
1. Create a `voting-keystore.json.lock` file adjacent to the
`voting_keystore_path`, indicating that the voting keystore is in-use and
should not be opened by another process.
1. Proceed to act for that validator, creating blocks and attestations if/when required.
If there is an error during any of these steps (e.g., a file is missing or
corrupt) the validator client will log an error and continue to attempt to
process other validators.
When the validator client exits (or the validator is deactivated) it will
remove the `voting-keystore.json.lock` to indicate that the keystore is free for use again.