gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-05-08 09:16:00 +00:00
Code Issues Packages Projects Releases Wiki Activity

Validator client refactor (#618)

Browse Source 
* Update to spec v0.9.0

* Update to v0.9.1

* Bump spec tags for v0.9.1

* Formatting, fix CI failures

* Resolve accidental KeyPair merge conflict

* Document new BeaconState functions

* Add `validator` changes from `validator-to-rest`

* Add initial (failing) REST api tests

* Fix signature parsing

* Add more tests

* Refactor http router

* Add working tests for publish beacon block

* Add validator duties tests

* Move account_manager under `lighthouse` binary

* Unify logfile handling in `environment` crate.

* Fix incorrect cache drops in `advance_caches`

* Update fork choice for v0.9.1

* Add `deposit_contract` crate

* Add progress on validator onboarding

* Add unfinished attesation code

* Update account manager CLI

* Write eth1 data file as hex string

* Integrate ValidatorDirectory with validator_client

* Move ValidatorDirectory into validator_client

* Clean up some FIXMEs

* Add beacon_chain_sim

* Fix a few docs/logs

* Expand `beacon_chain_sim`

* Fix spec for `beacon_chain_sim

* More testing for api

* Start work on attestation endpoint

* Reject empty attestations

* Allow attestations to genesis block

* Add working tests for `rest_api` validator endpoint

* Remove grpc from beacon_node

* Start heavy refactor of validator client

- Block production is working

* Prune old validator client files

* Start works on attestation service

* Add attestation service to validator client

* Use full pubkey for validator directories

* Add validator duties post endpoint

* Use par_iter for keypair generation

* Use bulk duties request in validator client

* Add version http endpoint tests

* Add interop keys and startup wait

* Ensure a prompt exit

* Add duties pruning

* Fix compile error in beacon node tests

* Add github workflow

* Modify rust.yaml

* Modify gitlab actions

* Add to CI file

* Add sudo to CI npm install

* Move cargo fmt to own job in tests

* Fix cargo fmt in CI

* Add rustup update before cargo fmt

* Change name of CI job

* Make other CI jobs require cargo fmt

* Add CI badge

* Remove gitlab and travis files

* Add different http timeout for debug

* Update docker file, use makefile in CI

* Use make in the dockerfile, skip the test

* Use the makefile for debug GI test

* Update book

* Tidy grpc and misc things

* Apply discv5 fixes

* Address other minor issues

* Fix warnings

* Attempt fix for addr parsing

* Tidy validator config, CLIs

* Tidy comments

* Tidy signing, reduce ForkService duplication

* Fail if skipping too many slots

* Set default recent genesis time to 0

* Add custom http timeout to validator

* Fix compile bug in node_test_rig

* Remove old bootstrap flag from val CLI

* Update docs

* Tidy val client

* Change val client log levels

* Add comments, more validity checks

* Fix compile error, add comments

* Undo changes to eth2-libp2p/src

* Reduce duplication of keypair generation

* Add more logging for validator duties

* Fix beacon_chain_sim, nitpicks

* Fix compile error, minor nits

* Address Michael's comments
This commit is contained in:
Paul Hauner
2019-11-25 15:48:24 +11:00
committed by GitHub
parent 3ca63cfa83
commit 78d82d9193
100 changed files with 4571 additions and 4032 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
book/src/SUMMARY.md
View File

@@ -1,6 +1,8 @@
# Summary
* [Introduction](./intro.md)
* [Installation](./installation.md)
* [Docker](./docker.md)
* [CLI](./cli.md)
* [Testnets](./testnets.md)
* [Simple Local Testnet](./simple-testnet.md)
@@ -9,4 +11,3 @@
* [WebSocket](./websockets.md)
* [Contributing](./contributing.md)
* [Development Environment](./setup.md)
* [CI & Testing](./ci.md)

33
book/src/ci.md
View File

@@ -1,33 +0,0 @@
# Contiguous Integration (CI) and Testing
Lighthouse uses a self-hosted Gitlab CI server to run tests and deploy docs.
For security reasons, **CI will only be run automatically for Lighthouse
maintainers.** Contributors without maintainer privileges will need to have CI
triggered for them prior to a PR being merged.
You can see the full set of tests we run in the
[gitlab-ci.yml](https://github.com/sigp/lighthouse/blob/master/.gitlab-ci.yml)
file. The following two commands should complete successfully before CI can
pass:
```bash
$ cargo test --all --all-features
$ cargo fmt --all --check
```
_Note: Travis CI is also used, however it does not run the full test suite._
### Ethereum 2.0 Spec Tests
The
[ethereum/eth2.0-spec-tests](https://github.com/ethereum/eth2.0-spec-tests/)
repository contains a large set of tests that verify Lighthouse behaviour
against the Ethereum Foundation specifications.
These tests are quite large (100's of MB), so we don't download them by
default. Developers should ensure they have downloaded these tests using the
`Makefile` in
[tests/ef_tests](https://github.com/sigp/lighthouse/tree/master/tests/ef_tests).
**Failures in these tests should prevent CI from passing.**

19
book/src/docker.md Normal file
View File

@@ -0,0 +1,19 @@
# Docker Guide
This repository has a `Dockerfile` in the root which builds an image with the
`lighthouse` binary installed.
To use the image, first build it (this will likely take several minutes):
```bash
$ docker build . -t lighthouse
```
Once it's built, run it with:
```bash
$ docker run lighthouse lighthouse --help
```
_Note: the first `lighthouse` is the name of the tag we created earlier. The
second `lighthouse` refers to the binary installed in the image._

48
book/src/installation.md Normal file
View File

@@ -0,0 +1,48 @@
# 📦 Installation
Lighthouse runs on Linux, MacOS and Windows. Installation should be easy. In
fact, if you already have Rust installed all you need is:
- `$ git clone https://github.com/sigp/lighthouse.git`
- `$ cd lighthouse`
- `$ make`
If this doesn't work or is not clear enough, see the [Detailed Instructions](#detailed-instructions). If you have further issues, see [Troubleshooting](#troubleshooting). If you'd prefer to use Docker, see the [Docker Guide](./docker.md).
## Detailed Instructions
1. Install Rust and Cargo with [rustup](https://rustup.rs/).
- Use the `stable` toolchain (it's the default).
1. Clone the Lighthouse repository.
- Run `$ git clone https://github.com/sigp/lighthouse.git`
- Change into the newly created directory with `$ cd lighthouse`
1. Build Lighthouse with `$ make`.
1. Installation was successful if `$ lighthouse --help` displays the
command-line documentation.
> First time compilation may take several minutes. If you experience any
> failures, please reach out on [discord](https://discord.gg/cyAszAh) or
> [create an issue](https://github.com/sigp/lighthouse/issues/new).
## Troubleshooting
### Command is not found
Lighthouse will be installed to `CARGO_HOME` or `$HOME/.cargo`. This directory
needs to be on your `PATH` before you can run `$ lighthouse`.
See ["Configuring the `PATH` environment variable"
(rust-lang.org)](https://www.rust-lang.org/tools/install) for more information.
### OpenSSL
If you get a build failure relating to OpenSSL, try installing `openssl-dev` or
`libssl-dev` using your OS package manager.
- Ubuntu: `$ apt-get install libssl-dev`.
- Amazon Linux: `$ yum install openssl-devel`.
### Perl for Windows
Perl may also be required to build Lighthouse. You can install [Strawberry
Perl](http://strawberryperl.com/), or alternatively if you're using the [Chocolatey](https://chocolatey.org/) package manager for Windows, use the following choco install command: `choco install strawberryperl`.

3
book/src/intro.md
View File

@@ -19,7 +19,8 @@ We implement the specification as defined in the
You may read this book from start to finish, or jump to some of these topics:
- Get started with [development environment setup](./setup.md).
- Follow the [Installation Guide](./installation.md) to install Lighthouse.
- Get hacking with the [Development Environment Guide](./setup.md).
- Utilize the whole stack by starting a [simple local testnet](./simple-testnet.md).
- Query the [RESTful HTTP API](./http.md) using `curl`.
- Listen to events with the [JSON WebSocket API](./websockets.md).

70
book/src/setup.md
View File

@@ -1,41 +1,49 @@
# Development Environment Setup
# Development Environment
## Linux, MacOS & Windows
Most Lighthouse developers work on Linux or MacOS, however Windows should still
be suitable.
1. Install Rust and Cargo with [rustup](https://rustup.rs/).
- Use the `stable` toolchain (it's the default).
1. Install build dependencies using your package manager.
- `clang`, `protobuf`, `libssl-dev`, `cmake`
1. Clone the [github.com/sigp/lighthouse](https://github.com/sigp/lighthouse)
repository.
1. Run `$ make` to build Lighthouse.
1. Run `$ make test` to run the test suite
- If you experience any failures, please reach out on
[discord](https://discord.gg/cyAszAh).
- Developers use `$ make test-full` to ensure you have the full set of
test vectors.
First, follow the [`Installation Guide`](./installation.md) to install
Lighthouse. This will install Lighthouse to your `PATH`, which is not
particularly useful for development but still a good way to ensure you have the
base dependencies.
> - The `beacon_node`, `validator_client` and other binaries are created in
> `target/release` directory.
> - First-time compilation may take several minutes.
The only additional requirement for developers is
[`ganache-cli`](https://github.com/trufflesuite/ganache-cli). This is used to
simulate the Eth1 chain during tests. You'll get failures during tests if you
don't have `ganache-cli` available on your `PATH`.
### Installing to `PATH`
## Testing
Use `cargo install --path lighthouse` from the root of the repository to
install the compiled binary to `CARGO_HOME` or `$HOME/.cargo`. If this
directory is on your `PATH`, you can run `$ lighthouse ..` from anywhere.
As with most other Rust projects, Lighthouse uses `cargo test` for unit and
integration tests. For example, to test the `ssz` crate run:
See ["Configuring the `PATH` environment
variable" (rust-lang.org)](https://www.rust-lang.org/tools/install) for more information.
```bash
cd eth2/utils/ssz
cargo test
```
> If you _don't_ install `lighthouse` to the path, you'll need to run the
> binaries directly from the `target` directory or using `cargo run ...`.
We also wrap some of these commands and expose them via the `Makefile` in the
project root for the benefit of CI/CD. We list some of these commands below so
you can run them locally and avoid CI failures:
### Windows
- `$ make cargo-fmt`: (fast) runs a Rust code linter.
- `$ make test`: (medium) runs unit tests across the whole project.
- `$ make test-ef`: (medium) runs the Ethereum Foundation test vectors.
- `$ make test-full`: (slow) runs the full test suite (including all previous
commands). This is approximately everything
that is required to pass CI.
Perl may also be required to build Lighthouse. You can install [Strawberry
Perl](http://strawberryperl.com/), or alternatively if you're using the [Chocolatey](https://chocolatey.org/) package manager for Windows, use the following choco install command: `choco install strawberryperl`.
_The lighthouse test suite is quite extensive, running the whole suite may take 30+ minutes._
Additionally, the dependency `protoc-grpcio v0.3.1` is reported to have issues
compiling in Windows. You can specify a known working version by editing
version in `protos/Cargo.toml` section to `protoc-grpcio = "<=0.3.0"`.
### Ethereum 2.0 Spec Tests
The
[ethereum/eth2.0-spec-tests](https://github.com/ethereum/eth2.0-spec-tests/)
repository contains a large set of tests that verify Lighthouse behaviour
against the Ethereum Foundation specifications.
These tests are quite large (100's of MB) so they're only downloaded if you run
`$ make test-ef` (or anything that run it). You may want to avoid
downloading these tests if you're on a slow or metered Internet connection. CI
will require them to pass, though.