Refactor Peerdb and PeerManager (#2660)

## Proposed Changes

This is a refactor of the PeerDB and PeerManager. A number of bugs have been surfacing around the connection state of peers and their interaction with the score state. 

This refactor tightens the mutability properties of peers such that only specific modules are able to modify the state of peer information preventing inadvertant state changes that can lead to our local peer manager db being out of sync with libp2p. 

Further, the logic around connection and scoring was quite convoluted and the distinction between the PeerManager and Peerdb was not well defined. Although these issues are not fully resolved, this PR is step to cleaning up this logic. The peerdb solely manages most mutability operations of peers leaving high-order logic to the peer manager. 

A single `update_connection_state()` function has been added to the peer-db making it solely responsible for modifying the peer's connection state. The way the peer's scores can be modified have been reduced to three simple functions (`update_scores()`, `update_gossipsub_scores()` and `report_peer()`). This prevents any add-hoc modifications of scores and only natural processes of score modification is allowed which simplifies the reasoning of score and state changes.

beacon_node/network/src/sync/manager.rs

@@ -654,15 +654,21 @@ impl<T: BeaconChainTypes> SyncManager<T> {
         // NOTE: here we are gracefully handling two race conditions: Receiving the status message
         // of a peer that is 1) disconnected 2) not in the PeerDB.
 
-        if let Some(peer_info) = self.network_globals.peers.write().peer_info_mut(peer_id) {
-            let new_state = sync_type.as_sync_status(remote_sync_info);
-            let rpr = new_state.as_str();
-            let was_updated = peer_info.sync_status.update(new_state.clone());
+        let new_state = sync_type.as_sync_status(remote_sync_info);
+        let rpr = new_state.as_str();
+        // Drop the write lock
+        let update_sync_status = self
+            .network_globals
+            .peers
+            .write()
+            .update_sync_status(peer_id, new_state.clone());
+        if let Some(was_updated) = update_sync_status {
+            let is_connected = self.network_globals.peers.read().is_connected(peer_id);
             if was_updated {
                 debug!(self.log, "Peer transitioned sync state"; "peer_id" => %peer_id, "new_state" => rpr,
                     "our_head_slot" => local_sync_info.head_slot, "out_finalized_epoch" => local_sync_info.finalized_epoch,
                     "their_head_slot" => remote_sync_info.head_slot, "their_finalized_epoch" => remote_sync_info.finalized_epoch,
-                    "is_connected" => peer_info.is_connected());
+                    "is_connected" => is_connected);
 
                 // A peer has transitioned its sync state. If the new state is "synced" we
                 // inform the backfill sync that a new synced peer has joined us.
@@ -670,7 +676,7 @@ impl<T: BeaconChainTypes> SyncManager<T> {
                     self.backfill_sync.fully_synced_peer_joined();
                 }
             }
-            peer_info.is_connected()
+            is_connected
         } else {
             error!(self.log, "Status'd peer is unknown"; "peer_id" => %peer_id);
             false

beacon_node/network/src/sync/network_context.rs

@@ -55,7 +55,7 @@ impl<T: EthSpec> SyncNetworkContext<T> {
             .peers
             .read()
             .peer_info(peer_id)
-            .map(|info| info.client.clone())
+            .map(|info| info.client().clone())
             .unwrap_or_default()
     }
 

beacon_node/network/src/sync/peer_sync_info.rs

@@ -1,6 +1,6 @@
 use super::manager::SLOT_IMPORT_TOLERANCE;
 use beacon_chain::{BeaconChain, BeaconChainTypes};
-use eth2_libp2p::{PeerSyncStatus, SyncInfo};
+use eth2_libp2p::{SyncInfo, SyncStatus as PeerSyncStatus};
 use std::cmp::Ordering;
 
 /// The type of peer relative to our current state.