gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-03 00:31:50 +00:00
Code Issues Packages Projects Releases Wiki Activity

Implement standard eth2.0 API (#1569)

Browse Source 
- Resolves #1550
- Resolves #824
- Resolves #825
- Resolves #1131
- Resolves #1411
- Resolves #1256
- Resolve #1177

- Includes the `ShufflingId` struct initially defined in #1492. That PR is now closed and the changes are included here, with significant bug fixes.
- Implement the https://github.com/ethereum/eth2.0-APIs in a new `http_api` crate using `warp`. This replaces the `rest_api` crate.
- Add a new `common/eth2` crate which provides a wrapper around `reqwest`, providing the HTTP client that is used by the validator client and for testing. This replaces the `common/remote_beacon_node` crate.
- Create a `http_metrics` crate which is a dedicated server for Prometheus metrics (they are no longer served on the same port as the REST API). We now have flags for `--metrics`, `--metrics-address`, etc.
- Allow the `subnet_id` to be an optional parameter for `VerifiedUnaggregatedAttestation::verify`. This means it does not need to be provided unnecessarily by the validator client.
- Move `fn map_attestation_committee` in `mod beacon_chain::attestation_verification` to a new `fn with_committee_cache` on the `BeaconChain` so the same cache can be used for obtaining validator duties.
- Add some other helpers to `BeaconChain` to assist with common API duties (e.g., `block_root_at_slot`, `head_beacon_block_root`).
- Change the `NaiveAggregationPool` so it can index attestations by `hash_tree_root(attestation.data)`. This is a requirement of the API.
- Add functions to `BeaconChainHarness` to allow it to create slashings and exits.
- Allow for `eth1::Eth1NetworkId` to go to/from a `String`.
- Add functions to the `OperationPool` to allow getting all objects in the pool.
- Add function to `BeaconState` to check if a committee cache is initialized.
- Fix bug where `seconds_per_eth1_block` was not transferring over from `YamlConfig` to `ChainSpec`.
- Add the `deposit_contract_address` to `YamlConfig` and `ChainSpec`. We needed to be able to return it in an API response.
- Change some uses of serde `serialize_with` and `deserialize_with` to a single use of `with` (code quality).
- Impl `Display` and `FromStr` for several BLS fields.
- Check for clock discrepancy when VC polls BN for sync state (with +/- 1 slot tolerance). This is not intended to be comprehensive, it was just easy to do.

- See #1434 for a per-endpoint overview.
- Seeking clarity here: https://github.com/ethereum/eth2.0-APIs/issues/75

- [x] Add docs for prom port to close #1256
- [x] Follow up on this #1177
- [x] ~~Follow up with #1424~~ Will fix in future PR.
- [x] Follow up with #1411
- [x] ~~Follow up with  #1260~~ Will fix in future PR.
- [x] Add quotes to all integers.
- [x] Remove `rest_types`
- [x] Address missing beacon block error. (#1629)
- [x] ~~Add tests for lighthouse/peers endpoints~~ Wontfix
- [x] ~~Follow up with validator status proposal~~ Tracked in #1434
- [x] Unify graffiti structs
- [x] ~~Start server when waiting for genesis?~~ Will fix in future PR.
- [x] TODO in http_api tests
- [x] Move lighthouse endpoints off /eth/v1
- [x] Update docs to link to standard

- ~~Blocked on #1586~~

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
This commit is contained in:
Paul Hauner
2020-09-29 03:46:54 +00:00
parent 8e20176337
commit cdec3cec18
156 changed files with 8862 additions and 8916 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
common/eth2/Cargo.toml Normal file
View File

@@ -0,0 +1,25 @@
[package]
name = "eth2"
version = "0.1.0"
authors = ["Paul Hauner <paul@paulhauner.com>"]
edition = "2018"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
serde = { version = "1.0.110", features = ["derive"] }
serde_json = "1.0.52"
types = { path = "../../consensus/types" }
hex = "0.4.2"
reqwest = { version = "0.10.8", features = ["json"] }
eth2_libp2p = { path = "../../beacon_node/eth2_libp2p" }
proto_array = { path = "../../consensus/proto_array", optional = true }
serde_utils = { path = "../../consensus/serde_utils" }
[target.'cfg(target_os = "linux")'.dependencies]
psutil = { version = "3.1.0", optional = true }
procinfo = { version = "0.4.2", optional = true }
[features]
default = ["lighthouse"]
lighthouse = ["proto_array", "psutil", "procinfo"]

784
common/eth2/src/lib.rs Normal file
View File

@@ -0,0 +1,784 @@
//! This crate provides two major things:
//!
//! 1. The types served by the `http_api` crate.
//! 2. A wrapper around `reqwest` that forms a HTTP client, able of consuming the endpoints served
//! by the `http_api` crate.
//!
//! Eventually it would be ideal to publish this crate on crates.io, however we have some local
//! dependencies preventing this presently.
#[cfg(feature = "lighthouse")]
pub mod lighthouse;
pub mod types;
use self::types::*;
use reqwest::{IntoUrl, Response};
use serde::{de::DeserializeOwned, Serialize};
use std::convert::TryFrom;
use std::fmt;
pub use reqwest;
pub use reqwest::{StatusCode, Url};
#[derive(Debug)]
pub enum Error {
/// The `reqwest` client raised an error.
Reqwest(reqwest::Error),
/// The server returned an error message where the body was able to be parsed.
ServerMessage(ErrorMessage),
/// The server returned an error message where the body was unable to be parsed.
StatusCode(StatusCode),
/// The supplied URL is badly formatted. It should look something like `http://127.0.0.1:5052`.
InvalidUrl(Url),
}
impl Error {
/// If the error has a HTTP status code, return it.
pub fn status(&self) -> Option<StatusCode> {
match self {
Error::Reqwest(error) => error.status(),
Error::ServerMessage(msg) => StatusCode::try_from(msg.code).ok(),
Error::StatusCode(status) => Some(*status),
Error::InvalidUrl(_) => None,
}
}
}
impl fmt::Display for Error {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{:?}", self)
}
}
/// A wrapper around `reqwest::Client` which provides convenience methods for interfacing with a
/// Lighthouse Beacon Node HTTP server (`http_api`).
#[derive(Clone)]
pub struct BeaconNodeHttpClient {
client: reqwest::Client,
server: Url,
}
impl BeaconNodeHttpClient {
pub fn new(server: Url) -> Self {
Self {
client: reqwest::Client::new(),
server,
}
}
pub fn from_components(server: Url, client: reqwest::Client) -> Self {
Self { client, server }
}
/// Return the path with the standard `/eth1/v1` prefix applied.
fn eth_path(&self) -> Result<Url, Error> {
let mut path = self.server.clone();
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("eth")
.push("v1");
Ok(path)
}
/// Perform a HTTP GET request.
async fn get<T: DeserializeOwned, U: IntoUrl>(&self, url: U) -> Result<T, Error> {
let response = self.client.get(url).send().await.map_err(Error::Reqwest)?;
ok_or_error(response)
.await?
.json()
.await
.map_err(Error::Reqwest)
}
/// Perform a HTTP GET request, returning `None` on a 404 error.
async fn get_opt<T: DeserializeOwned, U: IntoUrl>(&self, url: U) -> Result<Option<T>, Error> {
let response = self.client.get(url).send().await.map_err(Error::Reqwest)?;
match ok_or_error(response).await {
Ok(resp) => resp.json().await.map(Option::Some).map_err(Error::Reqwest),
Err(err) => {
if err.status() == Some(StatusCode::NOT_FOUND) {
Ok(None)
} else {
Err(err)
}
}
}
}
/// Perform a HTTP POST request.
async fn post<T: Serialize, U: IntoUrl>(&self, url: U, body: &T) -> Result<(), Error> {
let response = self
.client
.post(url)
.json(body)
.send()
.await
.map_err(Error::Reqwest)?;
ok_or_error(response).await?;
Ok(())
}
/// `GET beacon/genesis`
///
/// ## Errors
///
/// May return a `404` if beacon chain genesis has not yet occurred.
pub async fn get_beacon_genesis(&self) -> Result<GenericResponse<GenesisData>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("genesis");
self.get(path).await
}
/// `GET beacon/states/{state_id}/root`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_states_root(
&self,
state_id: StateId,
) -> Result<Option<GenericResponse<RootData>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("states")
.push(&state_id.to_string())
.push("root");
self.get_opt(path).await
}
/// `GET beacon/states/{state_id}/fork`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_states_fork(
&self,
state_id: StateId,
) -> Result<Option<GenericResponse<Fork>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("states")
.push(&state_id.to_string())
.push("fork");
self.get_opt(path).await
}
/// `GET beacon/states/{state_id}/finality_checkpoints`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_states_finality_checkpoints(
&self,
state_id: StateId,
) -> Result<Option<GenericResponse<FinalityCheckpointsData>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("states")
.push(&state_id.to_string())
.push("finality_checkpoints");
self.get_opt(path).await
}
/// `GET beacon/states/{state_id}/validators`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_states_validators(
&self,
state_id: StateId,
) -> Result<Option<GenericResponse<Vec<ValidatorData>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("states")
.push(&state_id.to_string())
.push("validators");
self.get_opt(path).await
}
/// `GET beacon/states/{state_id}/committees?slot,index`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_states_committees(
&self,
state_id: StateId,
epoch: Epoch,
slot: Option<Slot>,
index: Option<u64>,
) -> Result<Option<GenericResponse<Vec<CommitteeData>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("states")
.push(&state_id.to_string())
.push("committees")
.push(&epoch.to_string());
if let Some(slot) = slot {
path.query_pairs_mut()
.append_pair("slot", &slot.to_string());
}
if let Some(index) = index {
path.query_pairs_mut()
.append_pair("index", &index.to_string());
}
self.get_opt(path).await
}
/// `GET beacon/states/{state_id}/validators/{validator_id}`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_states_validator_id(
&self,
state_id: StateId,
validator_id: &ValidatorId,
) -> Result<Option<GenericResponse<ValidatorData>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("states")
.push(&state_id.to_string())
.push("validators")
.push(&validator_id.to_string());
self.get_opt(path).await
}
/// `GET beacon/headers?slot,parent_root`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_headers(
&self,
slot: Option<Slot>,
parent_root: Option<Hash256>,
) -> Result<Option<GenericResponse<Vec<BlockHeaderData>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("headers");
if let Some(slot) = slot {
path.query_pairs_mut()
.append_pair("slot", &slot.to_string());
}
if let Some(root) = parent_root {
path.query_pairs_mut()
.append_pair("parent_root", &format!("{:?}", root));
}
self.get_opt(path).await
}
/// `GET beacon/headers/{block_id}`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_headers_block_id(
&self,
block_id: BlockId,
) -> Result<Option<GenericResponse<BlockHeaderData>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("headers")
.push(&block_id.to_string());
self.get_opt(path).await
}
/// `POST beacon/blocks`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn post_beacon_blocks<T: EthSpec>(
&self,
block: &SignedBeaconBlock<T>,
) -> Result<(), Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("blocks");
self.post(path, block).await?;
Ok(())
}
/// `GET beacon/blocks`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_blocks<T: EthSpec>(
&self,
block_id: BlockId,
) -> Result<Option<GenericResponse<SignedBeaconBlock<T>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("blocks")
.push(&block_id.to_string());
self.get_opt(path).await
}
/// `GET beacon/blocks/{block_id}/root`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_blocks_root(
&self,
block_id: BlockId,
) -> Result<Option<GenericResponse<RootData>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("blocks")
.push(&block_id.to_string())
.push("root");
self.get_opt(path).await
}
/// `GET beacon/blocks/{block_id}/attestations`
///
/// Returns `Ok(None)` on a 404 error.
pub async fn get_beacon_blocks_attestations<T: EthSpec>(
&self,
block_id: BlockId,
) -> Result<Option<GenericResponse<Vec<Attestation<T>>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("blocks")
.push(&block_id.to_string())
.push("attestations");
self.get_opt(path).await
}
/// `POST beacon/pool/attestations`
pub async fn post_beacon_pool_attestations<T: EthSpec>(
&self,
attestation: &Attestation<T>,
) -> Result<(), Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("pool")
.push("attestations");
self.post(path, attestation).await?;
Ok(())
}
/// `GET beacon/pool/attestations`
pub async fn get_beacon_pool_attestations<T: EthSpec>(
&self,
) -> Result<GenericResponse<Vec<Attestation<T>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("pool")
.push("attestations");
self.get(path).await
}
/// `POST beacon/pool/attester_slashings`
pub async fn post_beacon_pool_attester_slashings<T: EthSpec>(
&self,
slashing: &AttesterSlashing<T>,
) -> Result<(), Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("pool")
.push("attester_slashings");
self.post(path, slashing).await?;
Ok(())
}
/// `GET beacon/pool/attester_slashings`
pub async fn get_beacon_pool_attester_slashings<T: EthSpec>(
&self,
) -> Result<GenericResponse<Vec<AttesterSlashing<T>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("pool")
.push("attester_slashings");
self.get(path).await
}
/// `POST beacon/pool/proposer_slashings`
pub async fn post_beacon_pool_proposer_slashings(
&self,
slashing: &ProposerSlashing,
) -> Result<(), Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("pool")
.push("proposer_slashings");
self.post(path, slashing).await?;
Ok(())
}
/// `GET beacon/pool/proposer_slashings`
pub async fn get_beacon_pool_proposer_slashings(
&self,
) -> Result<GenericResponse<Vec<ProposerSlashing>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("pool")
.push("proposer_slashings");
self.get(path).await
}
/// `POST beacon/pool/voluntary_exits`
pub async fn post_beacon_pool_voluntary_exits(
&self,
exit: &SignedVoluntaryExit,
) -> Result<(), Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("pool")
.push("voluntary_exits");
self.post(path, exit).await?;
Ok(())
}
/// `GET beacon/pool/voluntary_exits`
pub async fn get_beacon_pool_voluntary_exits(
&self,
) -> Result<GenericResponse<Vec<SignedVoluntaryExit>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("beacon")
.push("pool")
.push("voluntary_exits");
self.get(path).await
}
/// `GET config/fork_schedule`
pub async fn get_config_fork_schedule(&self) -> Result<GenericResponse<Vec<Fork>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("config")
.push("fork_schedule");
self.get(path).await
}
/// `GET config/fork_schedule`
pub async fn get_config_spec(&self) -> Result<GenericResponse<YamlConfig>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("config")
.push("spec");
self.get(path).await
}
/// `GET config/deposit_contract`
pub async fn get_config_deposit_contract(
&self,
) -> Result<GenericResponse<DepositContractData>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("config")
.push("deposit_contract");
self.get(path).await
}
/// `GET node/version`
pub async fn get_node_version(&self) -> Result<GenericResponse<VersionData>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("node")
.push("version");
self.get(path).await
}
/// `GET node/syncing`
pub async fn get_node_syncing(&self) -> Result<GenericResponse<SyncingData>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("node")
.push("syncing");
self.get(path).await
}
/// `GET debug/beacon/states/{state_id}`
pub async fn get_debug_beacon_states<T: EthSpec>(
&self,
state_id: StateId,
) -> Result<Option<GenericResponse<BeaconState<T>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("debug")
.push("beacon")
.push("states")
.push(&state_id.to_string());
self.get_opt(path).await
}
/// `GET debug/beacon/heads`
pub async fn get_debug_beacon_heads(
&self,
) -> Result<GenericResponse<Vec<ChainHeadData>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("debug")
.push("beacon")
.push("heads");
self.get(path).await
}
/// `GET validator/duties/attester/{epoch}?index`
///
/// ## Note
///
/// The `index` query parameter accepts a list of validator indices.
pub async fn get_validator_duties_attester(
&self,
epoch: Epoch,
index: Option<&[u64]>,
) -> Result<GenericResponse<Vec<AttesterData>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("validator")
.push("duties")
.push("attester")
.push(&epoch.to_string());
if let Some(index) = index {
let string = index
.iter()
.map(|i| i.to_string())
.collect::<Vec<_>>()
.join(",");
path.query_pairs_mut().append_pair("index", &string);
}
self.get(path).await
}
/// `GET validator/duties/proposer/{epoch}`
pub async fn get_validator_duties_proposer(
&self,
epoch: Epoch,
) -> Result<GenericResponse<Vec<ProposerData>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("validator")
.push("duties")
.push("proposer")
.push(&epoch.to_string());
self.get(path).await
}
/// `GET validator/duties/attester/{epoch}?index`
///
/// ## Note
///
/// The `index` query parameter accepts a list of validator indices.
pub async fn get_validator_blocks<T: EthSpec>(
&self,
slot: Slot,
randao_reveal: SignatureBytes,
graffiti: Option<&Graffiti>,
) -> Result<GenericResponse<BeaconBlock<T>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("validator")
.push("blocks")
.push(&slot.to_string());
path.query_pairs_mut()
.append_pair("randao_reveal", &randao_reveal.to_string());
if let Some(graffiti) = graffiti {
path.query_pairs_mut()
.append_pair("graffiti", &graffiti.to_string());
}
self.get(path).await
}
/// `GET validator/attestation_data?slot,committee_index`
pub async fn get_validator_attestation_data(
&self,
slot: Slot,
committee_index: CommitteeIndex,
) -> Result<GenericResponse<AttestationData>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("validator")
.push("attestation_data");
path.query_pairs_mut()
.append_pair("slot", &slot.to_string())
.append_pair("committee_index", &committee_index.to_string());
self.get(path).await
}
/// `GET validator/attestation_attestation?slot,attestation_data_root`
pub async fn get_validator_aggregate_attestation<T: EthSpec>(
&self,
slot: Slot,
attestation_data_root: Hash256,
) -> Result<Option<GenericResponse<Attestation<T>>>, Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("validator")
.push("aggregate_attestation");
path.query_pairs_mut()
.append_pair("slot", &slot.to_string())
.append_pair(
"attestation_data_root",
&format!("{:?}", attestation_data_root),
);
self.get_opt(path).await
}
/// `POST validator/aggregate_and_proofs`
pub async fn post_validator_aggregate_and_proof<T: EthSpec>(
&self,
aggregate: &SignedAggregateAndProof<T>,
) -> Result<(), Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("validator")
.push("aggregate_and_proofs");
self.post(path, aggregate).await?;
Ok(())
}
/// `POST validator/beacon_committee_subscriptions`
pub async fn post_validator_beacon_committee_subscriptions(
&self,
subscriptions: &[BeaconCommitteeSubscription],
) -> Result<(), Error> {
let mut path = self.eth_path()?;
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("validator")
.push("beacon_committee_subscriptions");
self.post(path, &subscriptions).await?;
Ok(())
}
}
/// Returns `Ok(response)` if the response is a `200 OK` response. Otherwise, creates an
/// appropriate error message.
async fn ok_or_error(response: Response) -> Result<Response, Error> {
let status = response.status();
if status == StatusCode::OK {
Ok(response)
} else if let Ok(message) = response.json().await {
Err(Error::ServerMessage(message))
} else {
Err(Error::StatusCode(status))
}
}

224
common/eth2/src/lighthouse.rs Normal file
View File

@@ -0,0 +1,224 @@
//! This module contains endpoints that are non-standard and only available on Lighthouse servers.
use crate::{
types::{Epoch, EthSpec, GenericResponse, ValidatorId},
BeaconNodeHttpClient, Error,
};
use proto_array::core::ProtoArray;
use serde::{Deserialize, Serialize};
pub use eth2_libp2p::{types::SyncState, PeerInfo};
/// Information returned by `peers` and `connected_peers`.
// TODO: this should be deserializable..
#[derive(Debug, Clone, Serialize)]
#[serde(bound = "T: EthSpec")]
pub struct Peer<T: EthSpec> {
/// The Peer's ID
pub peer_id: String,
/// The PeerInfo associated with the peer.
pub peer_info: PeerInfo<T>,
}
/// The results of validators voting during an epoch.
///
/// Provides information about the current and previous epochs.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct GlobalValidatorInclusionData {
/// The total effective balance of all active validators during the _current_ epoch.
pub current_epoch_active_gwei: u64,
/// The total effective balance of all active validators during the _previous_ epoch.
pub previous_epoch_active_gwei: u64,
/// The total effective balance of all validators who attested during the _current_ epoch.
pub current_epoch_attesting_gwei: u64,
/// The total effective balance of all validators who attested during the _current_ epoch and
/// agreed with the state about the beacon block at the first slot of the _current_ epoch.
pub current_epoch_target_attesting_gwei: u64,
/// The total effective balance of all validators who attested during the _previous_ epoch.
pub previous_epoch_attesting_gwei: u64,
/// The total effective balance of all validators who attested during the _previous_ epoch and
/// agreed with the state about the beacon block at the first slot of the _previous_ epoch.
pub previous_epoch_target_attesting_gwei: u64,
/// The total effective balance of all validators who attested during the _previous_ epoch and
/// agreed with the state about the beacon block at the time of attestation.
pub previous_epoch_head_attesting_gwei: u64,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ValidatorInclusionData {
/// True if the validator has been slashed, ever.
pub is_slashed: bool,
/// True if the validator can withdraw in the current epoch.
pub is_withdrawable_in_current_epoch: bool,
/// True if the validator was active in the state's _current_ epoch.
pub is_active_in_current_epoch: bool,
/// True if the validator was active in the state's _previous_ epoch.
pub is_active_in_previous_epoch: bool,
/// The validator's effective balance in the _current_ epoch.
pub current_epoch_effective_balance_gwei: u64,
/// True if the validator had an attestation included in the _current_ epoch.
pub is_current_epoch_attester: bool,
/// True if the validator's beacon block root attestation for the first slot of the _current_
/// epoch matches the block root known to the state.
pub is_current_epoch_target_attester: bool,
/// True if the validator had an attestation included in the _previous_ epoch.
pub is_previous_epoch_attester: bool,
/// True if the validator's beacon block root attestation for the first slot of the _previous_
/// epoch matches the block root known to the state.
pub is_previous_epoch_target_attester: bool,
/// True if the validator's beacon block root attestation in the _previous_ epoch at the
/// attestation's slot (`attestation_data.slot`) matches the block root known to the state.
pub is_previous_epoch_head_attester: bool,
}
#[cfg(target_os = "linux")]
use {procinfo::pid, psutil::process::Process};
/// Reports on the health of the Lighthouse instance.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct Health {
/// The pid of this process.
pub pid: u32,
/// The number of threads used by this pid.
pub pid_num_threads: i32,
/// The total resident memory used by this pid.
pub pid_mem_resident_set_size: u64,
/// The total virtual memory used by this pid.
pub pid_mem_virtual_memory_size: u64,
/// Total virtual memory on the system
pub sys_virt_mem_total: u64,
/// Total virtual memory available for new processes.
pub sys_virt_mem_available: u64,
/// Total virtual memory used on the system
pub sys_virt_mem_used: u64,
/// Total virtual memory not used on the system
pub sys_virt_mem_free: u64,
/// Percentage of virtual memory used on the system
pub sys_virt_mem_percent: f32,
/// System load average over 1 minute.
pub sys_loadavg_1: f64,
/// System load average over 5 minutes.
pub sys_loadavg_5: f64,
/// System load average over 15 minutes.
pub sys_loadavg_15: f64,
}
impl Health {
#[cfg(not(target_os = "linux"))]
pub fn observe() -> Result<Self, String> {
Err("Health is only available on Linux".into())
}
#[cfg(target_os = "linux")]
pub fn observe() -> Result<Self, String> {
let process =
Process::current().map_err(|e| format!("Unable to get current process: {:?}", e))?;
let process_mem = process
.memory_info()
.map_err(|e| format!("Unable to get process memory info: {:?}", e))?;
let stat = pid::stat_self().map_err(|e| format!("Unable to get stat: {:?}", e))?;
let vm = psutil::memory::virtual_memory()
.map_err(|e| format!("Unable to get virtual memory: {:?}", e))?;
let loadavg =
psutil::host::loadavg().map_err(|e| format!("Unable to get loadavg: {:?}", e))?;
Ok(Self {
pid: process.pid(),
pid_num_threads: stat.num_threads,
pid_mem_resident_set_size: process_mem.rss(),
pid_mem_virtual_memory_size: process_mem.vms(),
sys_virt_mem_total: vm.total(),
sys_virt_mem_available: vm.available(),
sys_virt_mem_used: vm.used(),
sys_virt_mem_free: vm.free(),
sys_virt_mem_percent: vm.percent(),
sys_loadavg_1: loadavg.one,
sys_loadavg_5: loadavg.five,
sys_loadavg_15: loadavg.fifteen,
})
}
}
impl BeaconNodeHttpClient {
/// `GET lighthouse/health`
pub async fn get_lighthouse_health(&self) -> Result<GenericResponse<Health>, Error> {
let mut path = self.server.clone();
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("lighthouse")
.push("health");
self.get(path).await
}
/// `GET lighthouse/syncing`
pub async fn get_lighthouse_syncing(&self) -> Result<GenericResponse<SyncState>, Error> {
let mut path = self.server.clone();
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("lighthouse")
.push("syncing");
self.get(path).await
}
/*
* Note:
*
* The `lighthouse/peers` endpoints do not have functions here. We are yet to implement
* `Deserialize` on the `PeerInfo` struct since it contains use of `Instant`. This could be
* fairly simply achieved, if desired.
*/
/// `GET lighthouse/proto_array`
pub async fn get_lighthouse_proto_array(&self) -> Result<GenericResponse<ProtoArray>, Error> {
let mut path = self.server.clone();
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("lighthouse")
.push("proto_array");
self.get(path).await
}
/// `GET lighthouse/validator_inclusion/{epoch}/global`
pub async fn get_lighthouse_validator_inclusion_global(
&self,
epoch: Epoch,
) -> Result<GenericResponse<GlobalValidatorInclusionData>, Error> {
let mut path = self.server.clone();
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("lighthouse")
.push("validator_inclusion")
.push(&epoch.to_string())
.push("global");
self.get(path).await
}
/// `GET lighthouse/validator_inclusion/{epoch}/{validator_id}`
pub async fn get_lighthouse_validator_inclusion(
&self,
epoch: Epoch,
validator_id: ValidatorId,
) -> Result<GenericResponse<Option<ValidatorInclusionData>>, Error> {
let mut path = self.server.clone();
path.path_segments_mut()
.map_err(|()| Error::InvalidUrl(self.server.clone()))?
.push("lighthouse")
.push("validator_inclusion")
.push(&epoch.to_string())
.push(&validator_id.to_string());
self.get(path).await
}
}

432
common/eth2/src/types.rs Normal file
View File

@@ -0,0 +1,432 @@
//! This module exposes a superset of the `types` crate. It adds additional types that are only
//! required for the HTTP API.
use eth2_libp2p::{Enr, Multiaddr};
use serde::{Deserialize, Serialize};
use std::convert::TryFrom;
use std::fmt;
use std::str::FromStr;
pub use types::*;
/// An API error serializable to JSON.
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ErrorMessage {
pub code: u16,
pub message: String,
#[serde(default)]
pub stacktraces: Vec<String>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct GenesisData {
#[serde(with = "serde_utils::quoted_u64")]
pub genesis_time: u64,
pub genesis_validators_root: Hash256,
#[serde(with = "serde_utils::bytes_4_hex")]
pub genesis_fork_version: [u8; 4],
}
#[derive(Debug, Copy, Clone, PartialEq)]
pub enum BlockId {
Head,
Genesis,
Finalized,
Justified,
Slot(Slot),
Root(Hash256),
}
impl FromStr for BlockId {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s {
"head" => Ok(BlockId::Head),
"genesis" => Ok(BlockId::Genesis),
"finalized" => Ok(BlockId::Finalized),
"justified" => Ok(BlockId::Justified),
other => {
if other.starts_with("0x") {
Hash256::from_str(&s[2..])
.map(BlockId::Root)
.map_err(|e| format!("{} cannot be parsed as a root", e))
} else {
u64::from_str(s)
.map(Slot::new)
.map(BlockId::Slot)
.map_err(|_| format!("{} cannot be parsed as a parameter", s))
}
}
}
}
}
impl fmt::Display for BlockId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
BlockId::Head => write!(f, "head"),
BlockId::Genesis => write!(f, "genesis"),
BlockId::Finalized => write!(f, "finalized"),
BlockId::Justified => write!(f, "justified"),
BlockId::Slot(slot) => write!(f, "{}", slot),
BlockId::Root(root) => write!(f, "{:?}", root),
}
}
}
#[derive(Debug, Copy, Clone, PartialEq)]
pub enum StateId {
Head,
Genesis,
Finalized,
Justified,
Slot(Slot),
Root(Hash256),
}
impl FromStr for StateId {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
match s {
"head" => Ok(StateId::Head),
"genesis" => Ok(StateId::Genesis),
"finalized" => Ok(StateId::Finalized),
"justified" => Ok(StateId::Justified),
other => {
if other.starts_with("0x") {
Hash256::from_str(&s[2..])
.map(StateId::Root)
.map_err(|e| format!("{} cannot be parsed as a root", e))
} else {
u64::from_str(s)
.map(Slot::new)
.map(StateId::Slot)
.map_err(|_| format!("{} cannot be parsed as a slot", s))
}
}
}
}
}
impl fmt::Display for StateId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
StateId::Head => write!(f, "head"),
StateId::Genesis => write!(f, "genesis"),
StateId::Finalized => write!(f, "finalized"),
StateId::Justified => write!(f, "justified"),
StateId::Slot(slot) => write!(f, "{}", slot),
StateId::Root(root) => write!(f, "{:?}", root),
}
}
}
#[derive(Debug, PartialEq, Clone, Serialize, Deserialize)]
#[serde(bound = "T: Serialize + serde::de::DeserializeOwned")]
pub struct GenericResponse<T: Serialize + serde::de::DeserializeOwned> {
pub data: T,
}
impl<T: Serialize + serde::de::DeserializeOwned> From<T> for GenericResponse<T> {
fn from(data: T) -> Self {
Self { data }
}
}
#[derive(Debug, PartialEq, Clone, Serialize)]
#[serde(bound = "T: Serialize")]
pub struct GenericResponseRef<'a, T: Serialize> {
pub data: &'a T,
}
impl<'a, T: Serialize> From<&'a T> for GenericResponseRef<'a, T> {
fn from(data: &'a T) -> Self {
Self { data }
}
}
#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize)]
pub struct RootData {
pub root: Hash256,
}
impl From<Hash256> for RootData {
fn from(root: Hash256) -> Self {
Self { root }
}
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct FinalityCheckpointsData {
pub previous_justified: Checkpoint,
pub current_justified: Checkpoint,
pub finalized: Checkpoint,
}
#[derive(Debug, Clone, PartialEq)]
pub enum ValidatorId {
PublicKey(PublicKeyBytes),
Index(u64),
}
impl FromStr for ValidatorId {
type Err = String;
fn from_str(s: &str) -> Result<Self, Self::Err> {
if s.starts_with("0x") {
PublicKeyBytes::from_str(s)
.map(ValidatorId::PublicKey)
.map_err(|e| format!("{} cannot be parsed as a public key: {}", s, e))
} else {
u64::from_str(s)
.map(ValidatorId::Index)
.map_err(|e| format!("{} cannot be parsed as a slot: {}", s, e))
}
}
}
impl fmt::Display for ValidatorId {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
match self {
ValidatorId::PublicKey(pubkey) => write!(f, "{:?}", pubkey),
ValidatorId::Index(index) => write!(f, "{}", index),
}
}
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ValidatorData {
#[serde(with = "serde_utils::quoted_u64")]
pub index: u64,
#[serde(with = "serde_utils::quoted_u64")]
pub balance: u64,
pub status: ValidatorStatus,
pub validator: Validator,
}
// TODO: This does not currently match the spec, but I'm going to try and change the spec using
// this proposal:
//
// https://hackmd.io/bQxMDRt1RbS1TLno8K4NPg?view
#[derive(Debug, Clone, Copy, PartialEq, Serialize, Deserialize)]
pub enum ValidatorStatus {
Unknown,
WaitingForEligibility,
WaitingForFinality,
WaitingInQueue,
StandbyForActive(Epoch),
Active,
ActiveAwaitingVoluntaryExit(Epoch),
ActiveAwaitingSlashedExit(Epoch),
ExitedVoluntarily(Epoch),
ExitedSlashed(Epoch),
Withdrawable,
Withdrawn,
}
impl ValidatorStatus {
pub fn from_validator(
validator_opt: Option<&Validator>,
epoch: Epoch,
finalized_epoch: Epoch,
far_future_epoch: Epoch,
) -> Self {
if let Some(validator) = validator_opt {
if validator.is_withdrawable_at(epoch) {
ValidatorStatus::Withdrawable
} else if validator.is_exited_at(epoch) {
if validator.slashed {
ValidatorStatus::ExitedSlashed(validator.withdrawable_epoch)
} else {
ValidatorStatus::ExitedVoluntarily(validator.withdrawable_epoch)
}
} else if validator.is_active_at(epoch) {
if validator.exit_epoch < far_future_epoch {
if validator.slashed {
ValidatorStatus::ActiveAwaitingSlashedExit(validator.exit_epoch)
} else {
ValidatorStatus::ActiveAwaitingVoluntaryExit(validator.exit_epoch)
}
} else {
ValidatorStatus::Active
}
} else if validator.activation_epoch < far_future_epoch {
ValidatorStatus::StandbyForActive(validator.activation_epoch)
} else if validator.activation_eligibility_epoch < far_future_epoch {
if finalized_epoch < validator.activation_eligibility_epoch {
ValidatorStatus::WaitingForFinality
} else {
ValidatorStatus::WaitingInQueue
}
} else {
ValidatorStatus::WaitingForEligibility
}
} else {
ValidatorStatus::Unknown
}
}
}
#[derive(Serialize, Deserialize)]
pub struct CommitteesQuery {
pub slot: Option<Slot>,
pub index: Option<u64>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct CommitteeData {
#[serde(with = "serde_utils::quoted_u64")]
pub index: u64,
pub slot: Slot,
#[serde(with = "serde_utils::quoted_u64_vec")]
pub validators: Vec<u64>,
}
#[derive(Serialize, Deserialize)]
pub struct HeadersQuery {
pub slot: Option<Slot>,
pub parent_root: Option<Hash256>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct BlockHeaderAndSignature {
pub message: BeaconBlockHeader,
pub signature: SignatureBytes,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct BlockHeaderData {
pub root: Hash256,
pub canonical: bool,
pub header: BlockHeaderAndSignature,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct DepositContractData {
#[serde(with = "serde_utils::quoted_u64")]
pub chain_id: u64,
pub address: Address,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ChainHeadData {
pub slot: Slot,
pub root: Hash256,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct IdentityData {
pub peer_id: String,
pub enr: Enr,
pub p2p_addresses: Vec<Multiaddr>,
// TODO: missing the following fields:
//
// - discovery_addresses
// - metadata
//
// Tracked here: https://github.com/sigp/lighthouse/issues/1434
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct VersionData {
pub version: String,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct SyncingData {
pub is_syncing: bool,
pub head_slot: Slot,
pub sync_distance: Slot,
}
#[derive(Clone, PartialEq, Debug, Deserialize)]
#[serde(try_from = "String", bound = "T: FromStr")]
pub struct QueryVec<T: FromStr>(pub Vec<T>);
impl<T: FromStr> TryFrom<String> for QueryVec<T> {
type Error = String;
fn try_from(string: String) -> Result<Self, Self::Error> {
if string == "" {
return Ok(Self(vec![]));
}
string
.split(',')
.map(|s| s.parse().map_err(|_| "unable to parse".to_string()))
.collect::<Result<Vec<T>, String>>()
.map(Self)
}
}
#[derive(Clone, Deserialize)]
pub struct ValidatorDutiesQuery {
pub index: Option<QueryVec<u64>>,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct AttesterData {
pub pubkey: PublicKeyBytes,
#[serde(with = "serde_utils::quoted_u64")]
pub validator_index: u64,
#[serde(with = "serde_utils::quoted_u64")]
pub committees_at_slot: u64,
#[serde(with = "serde_utils::quoted_u64")]
pub committee_index: CommitteeIndex,
#[serde(with = "serde_utils::quoted_u64")]
pub committee_length: u64,
#[serde(with = "serde_utils::quoted_u64")]
pub validator_committee_index: u64,
pub slot: Slot,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct ProposerData {
pub pubkey: PublicKeyBytes,
pub slot: Slot,
}
#[derive(Clone, Serialize, Deserialize)]
pub struct ValidatorBlocksQuery {
pub randao_reveal: SignatureBytes,
pub graffiti: Option<Graffiti>,
}
#[derive(Clone, Serialize, Deserialize)]
pub struct ValidatorAttestationDataQuery {
pub slot: Slot,
pub committee_index: CommitteeIndex,
}
#[derive(Clone, Serialize, Deserialize)]
pub struct ValidatorAggregateAttestationQuery {
pub attestation_data_root: Hash256,
pub slot: Slot,
}
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct BeaconCommitteeSubscription {
#[serde(with = "serde_utils::quoted_u64")]
pub validator_index: u64,
#[serde(with = "serde_utils::quoted_u64")]
pub committee_index: u64,
#[serde(with = "serde_utils::quoted_u64")]
pub committees_at_slot: u64,
pub slot: Slot,
pub is_aggregator: bool,
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn query_vec() {
assert_eq!(
QueryVec::try_from("0,1,2".to_string()).unwrap(),
QueryVec(vec![0_u64, 1, 2])
);
}
}