Gloas payload cache (#9209)

In Gloas, beacon blocks are imported into fork choice immediately - the payload envelope and data columns arrive
separately. KZG commitments moved from the column sidecar into the execution payload bid, so the existing
`DataAvailabilityChecker` (which assumes block and data are coupled) can't be used for Gloas.


  * Introduced `PendingPayloadCache` to keep track of payload and data columns per block root.
* Added gossip column verification
* Added support for Gloas data column reconstruction
* Payload envelope verification simplified: removed `MaybeAvailableEnvelope`, `ExecutedEnvelope`, `EnvelopeImportData`

Not yet implemented (tracked with TODOs):
- Proper lookup sync for Gloas columns arriving before blocks
- Partial column merging for Gloas
- Moving `load_gloas_payload_bid` disk reads off the async runtime
- Backfill/range sync for Gloas

Based on @eserilev's PR and work in progress. See also #9202 for verification.


Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>

Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>

Co-Authored-By: Daniel Knopik <daniel@dknopik.de>

Co-Authored-By: Daniel Knopik <107140945+dknopik@users.noreply.github.com>

Co-Authored-By: dapplion <35266934+dapplion@users.noreply.github.com>

Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>

beacon_node/http_api/src/beacon/execution_payload_envelope.rs

@@ -7,6 +7,7 @@ use crate::version::{
     execution_optimistic_finalized_beacon_response,
 };
 use beacon_chain::data_column_verification::{GossipDataColumnError, GossipVerifiedDataColumn};
+use beacon_chain::payload_envelope_verification::EnvelopeError;
 use beacon_chain::{BeaconChain, BeaconChainTypes, NotifyExecutionLayer};
 use bytes::Bytes;
 use eth2::types as api_types;
@@ -148,7 +149,7 @@ pub async fn publish_execution_payload_envelope<T: BeaconChainTypes>(
             PubsubMessage::ExecutionPayload(Box::new(envelope_for_gossip)),
         )
         .map_err(|_| {
-            beacon_chain::payload_envelope_verification::EnvelopeError::BeaconChainError(Arc::new(
+            EnvelopeError::BeaconChainError(Arc::new(
                 beacon_chain::BeaconChainError::UnableToPublish,
             ))
         })
@@ -272,7 +273,7 @@ fn build_gloas_data_columns<T: BeaconChainTypes>(
             let index = *col.index();
             match GossipVerifiedDataColumn::new_for_block_publishing(col, chain) {
                 Ok(verified) => Some(verified),
-                Err(GossipDataColumnError::PriorKnownUnpublished) => None,
+                Err(GossipDataColumnError::PriorKnown { .. }) => None,
                 Err(e) => {
                     warn!(
                         %slot,

beacon_node/http_api/src/publish_blocks.rs

@@ -246,7 +246,7 @@ pub async fn publish_block<T: BeaconChainTypes, B: IntoGossipVerifiedBlock<T>>(
             if let Err(e) =
                 Box::pin(chain.process_gossip_data_columns(sampling_columns, publish_fn)).await
             {
-                let msg = format!("Invalid data column: {e}");
+                let msg = format!("Invalid data column: {e:?}");
                 return if let BroadcastValidation::Gossip = validation_level {
                     Err(warp_utils::reject::broadcast_without_import(msg))
                 } else {

beacon_node/http_api/tests/interactive_tests.rs

@@ -382,6 +382,7 @@ pub async fn proposer_boost_re_org_weight_misprediction() {
 /// - `num_empty_votes`: percentage of comm of attestations for the parent block
 /// - `num_head_votes`: number of attestations for the head block
 /// - `should_re_org`: whether the proposer should build on the parent rather than the head
+#[allow(clippy::large_stack_frames)]
 pub async fn proposer_boost_re_org_test(
     ReOrgTest {
         head_slot,