Bootstrap (#501)

* Renamed fork_choice::process_attestation_from_block

* Processing attestation in fork choice

* Retrieving state from store and checking signature

* Looser check on beacon state validity.

* Cleaned up get_attestation_state

* Expanded fork choice api to provide latest validator message.

* Checking if the an attestation contains a latest message

* Correct process_attestation error handling.

* Copy paste error in comment fixed.

* Tidy ancestor iterators

* Getting attestation slot via helper method

* Refactored attestation creation in test utils

* Revert "Refactored attestation creation in test utils"

This reverts commit 4d277fe4239a7194758b18fb5c00dfe0b8231306.

* Integration tests for free attestation processing

* Implicit conflicts resolved.

* formatting

* Do first pass on Grants code

* Add another attestation processing test

* Tidy attestation processing

* Remove old code fragment

* Add non-compiling half finished changes

* Simplify, fix bugs, add tests for chain iters

* Remove attestation processing from op pool

* Fix bug with fork choice, tidy

* Fix overly restrictive check in fork choice.

* Ensure committee cache is build during attn proc

* Ignore unknown blocks at fork choice

* Various minor fixes

* Make fork choice write lock in to read lock

* Remove unused method

* Tidy comments

* Fix attestation prod. target roots change

* Fix compile error in store iters

* Reject any attestation prior to finalization

* Begin metrics refactor

* Move beacon_chain to new metrics structure.

* Make metrics not panic if already defined

* Use global prometheus gather at rest api

* Unify common metric fns into a crate

* Add heavy metering to block processing

* Remove hypen from prometheus metric name

* Add more beacon chain metrics

* Add beacon chain persistence metric

* Prune op pool on finalization

* Add extra prom beacon chain metrics

* Prefix BeaconChain metrics with "beacon_"

* Add more store metrics

* Add basic metrics to libp2p

* Add metrics to HTTP server

* Remove old `http_server` crate

* Update metrics names to be more like standard

* Fix broken beacon chain metrics, add slot clock metrics

* Add lighthouse_metrics gather fn

* Remove http args

* Fix wrong state given to op pool prune

* Make prom metric names more consistent

* Add more metrics, tidy existing metrics

* Fix store block read metrics

* Tidy attestation metrics

* Fix minor PR comments

* Fix minor PR comments

* Remove duplicated attestation finalization check

* Remove awkward `let` statement

* Add first attempts at HTTP bootstrap

* Add beacon_block methods to rest api

* Fix serde for block.body.grafitti

* Allow travis failures on beta (see desc)

There's a non-backward compatible change in `cargo fmt`. Stable and beta
do not agree.

* Add network routes to API

* Fix rustc warnings

* Add best_slot method

* Add --bootstrap arg to beacon node

* Get bootstrapper working for ENR address

* Store intermediate states during block processing

* Allow bootstrapper to scrape libp2p address

* Update bootstrapper libp2p address finding

* Add comments

* Tidy API to be more consistent with recent decisions

* Address some review comments

* Make BeaconChainTypes Send + Sync + 'static

* Add `/network/listen_port` API endpoint

* Abandon starting the node if libp2p doesn't start

* Update bootstrapper for API changes

* Remove unnecessary trait bounds

beacon_node/eth2-libp2p/src/behaviour.rs

@@ -78,6 +78,10 @@ impl<TSubstream: AsyncRead + AsyncWrite> Behaviour<TSubstream> {
             log: behaviour_log,
         })
     }
+
+    pub fn discovery(&self) -> &Discovery<TSubstream> {
+        &self.discovery
+    }
 }
 
 // Implement the NetworkBehaviourEventProcess trait so that we can derive NetworkBehaviour for Behaviour

beacon_node/eth2-libp2p/src/discovery.rs

@@ -103,6 +103,10 @@ impl<TSubstream> Discovery<TSubstream> {
         })
     }
 
+    pub fn local_enr(&self) -> &Enr {
+        self.discovery.local_enr()
+    }
+
     /// Manually search for peers. This restarts the discovery round, sparking multiple rapid
     /// queries.
     pub fn discover_peers(&mut self) {
@@ -120,6 +124,11 @@ impl<TSubstream> Discovery<TSubstream> {
         self.connected_peers.len()
     }
 
+    /// The current number of connected libp2p peers.
+    pub fn connected_peer_set(&self) -> &HashSet<PeerId> {
+        &self.connected_peers
+    }
+
     /// Search for new peers using the underlying discovery mechanism.
     fn find_peers(&mut self) {
         // pick a random NodeId

beacon_node/eth2-libp2p/src/lib.rs

@@ -17,12 +17,13 @@ pub use behaviour::PubsubMessage;
 pub use config::{
     Config as NetworkConfig, BEACON_ATTESTATION_TOPIC, BEACON_BLOCK_TOPIC, SHARD_TOPIC_PREFIX,
 };
+pub use libp2p::enr::Enr;
 pub use libp2p::gossipsub::{Topic, TopicHash};
 pub use libp2p::multiaddr;
 pub use libp2p::Multiaddr;
 pub use libp2p::{
     gossipsub::{GossipsubConfig, GossipsubConfigBuilder},
-    PeerId,
+    PeerId, Swarm,
 };
 pub use rpc::RPCEvent;
 pub use service::Libp2pEvent;

beacon_node/eth2-libp2p/src/service.rs

@@ -16,7 +16,7 @@ use libp2p::core::{
     upgrade::{InboundUpgradeExt, OutboundUpgradeExt},
 };
 use libp2p::{core, secio, PeerId, Swarm, Transport};
-use slog::{debug, info, trace, warn};
+use slog::{crit, debug, info, trace, warn};
 use std::fs::File;
 use std::io::prelude::*;
 use std::io::{Error, ErrorKind};
@@ -33,7 +33,7 @@ pub struct Service {
     //TODO: Make this private
     pub swarm: Swarm<Libp2pStream, Libp2pBehaviour>,
     /// This node's PeerId.
-    _local_peer_id: PeerId,
+    pub local_peer_id: PeerId,
     /// The libp2p logger handle.
     pub log: slog::Logger,
 }
@@ -69,10 +69,15 @@ impl Service {
                 log_address.push(Protocol::P2p(local_peer_id.clone().into()));
                 info!(log, "Listening on: {}", log_address);
             }
-            Err(err) => warn!(
-                log,
-                "Cannot listen on: {} because: {:?}", listen_multiaddr, err
-            ),
+            Err(err) => {
+                crit!(
+                    log,
+                    "Unable to listen on libp2p address";
+                    "error" => format!("{:?}", err),
+                    "listen_multiaddr" => format!("{}", listen_multiaddr),
+                );
+                return Err("Libp2p was unable to listen on the given listen address.".into());
+            }
         };
 
         // attempt to connect to user-input libp2p nodes
@@ -113,7 +118,7 @@ impl Service {
         info!(log, "Subscribed to topics: {:?}", subscribed_topics);
 
         Ok(Service {
-            _local_peer_id: local_peer_id,
+            local_peer_id,
             swarm,
             log,
         })