Prepare proposer (#3043)

## Issue Addressed

Resolves #2936

## Proposed Changes

Adds functionality for calling [`validator/prepare_beacon_proposer`](https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Validator/prepareBeaconProposer) in advance.

There is a `BeaconChain::prepare_beacon_proposer` method which, which called, computes the proposer for the next slot. If that proposer has been registered via the `validator/prepare_beacon_proposer` API method, then the `beacon_chain.execution_layer` will be provided the `PayloadAttributes` for us in all future forkchoiceUpdated calls. An artificial forkchoiceUpdated call will be created 4s before each slot, when the head updates and when a validator updates their information.

Additionally, I added strict ordering for calls from the `BeaconChain` to the `ExecutionLayer`. I'm not certain the `ExecutionLayer` will always maintain this ordering, but it's a good start to have consistency from the `BeaconChain`. There are some deadlock opportunities introduced, they are documented in the code.

## Additional Info

- ~~Blocked on #2837~~

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

beacon_node/network/src/sync/manager.rs

@@ -209,6 +209,9 @@ pub struct SyncManager<T: BeaconChainTypes> {
     /// A multi-threaded, non-blocking processor for applying messages to the beacon chain.
     beacon_processor_send: mpsc::Sender<BeaconWorkEvent<T>>,
 
+    /// Used for spawning tasks.
+    executor: task_executor::TaskExecutor,
+
     /// The logger for the import manager.
     log: Logger,
 }
@@ -269,6 +272,7 @@ pub fn spawn<T: BeaconChainTypes>(
         failed_chains: LRUCache::new(500),
         single_block_lookups: FnvHashMap::default(),
         beacon_processor_send,
+        executor: executor.clone(),
         log: log.clone(),
     };
 
@@ -475,19 +479,27 @@ impl<T: BeaconChainTypes> SyncManager<T> {
                             );
                             info!(self.log, "Processed block"; "block" => %block_root);
 
-                            match self.chain.fork_choice() {
-                                Ok(()) => trace!(
-                                    self.log,
-                                    "Fork choice success";
-                                    "location" => "single block"
-                                ),
-                                Err(e) => error!(
-                                    self.log,
-                                    "Fork choice failed";
-                                    "error" => ?e,
-                                    "location" => "single block"
-                                ),
-                            }
+                            // Spawn `BeaconChain::fork_choice` in a blocking task. It's
+                            // potentially long-running and it might panic if run from an async
+                            // context.
+                            let chain = self.chain.clone();
+                            let log = self.log.clone();
+                            self.executor.spawn_blocking(
+                                move || match chain.fork_choice() {
+                                    Ok(()) => trace!(
+                                        log,
+                                        "Fork choice success";
+                                        "location" => "single block"
+                                    ),
+                                    Err(e) => error!(
+                                        log,
+                                        "Fork choice failed";
+                                        "error" => ?e,
+                                        "location" => "single block"
+                                    ),
+                                },
+                                "sync_manager_fork_choice",
+                            );
                         }
                         Err(BlockError::ParentUnknown { .. }) => {
                             // We don't know of the blocks parent, begin a parent lookup search