SingleAttestation implementation (#6488)

* First pass * Add restrictions to RuntimeVariableList api * Use empty_uninitialized and fix warnings * Fix some todos * Merge branch 'unstable' into max-blobs-preset * Fix take impl on RuntimeFixedList * cleanup * Fix test compilations * Fix some more tests * Fix test from unstable * Merge branch 'unstable' into max-blobs-preset * SingleAttestation * Add post attestation v2 endpoint logic to attestation service * Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_attestation * Implement "Bugfix and more withdrawal tests" * Implement "Add missed exit checks to consolidation processing" * Implement "Update initial earliest_exit_epoch calculation" * Implement "Limit consolidating balance by validator.effective_balance" * Implement "Use 16-bit random value in validator filter" * Implement "Do not change creds type on consolidation" * some tests and fixed attestqtion calc * Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_attestation * Rename PendingPartialWithdraw index field to validator_index * Skip slots to get test to pass and add TODO * Implement "Synchronously check all transactions to have non-zero length" * Merge remote-tracking branch 'origin/unstable' into max-blobs-preset * Remove footgun function * Minor simplifications * Move from preset to config * Fix typo * Revert "Remove footgun function" This reverts commit de01f923c7. * Try fixing tests * Implement "bump minimal preset MAX_BLOB_COMMITMENTS_PER_BLOCK and KZG_COMMITMENT_INCLUSION_PROOF_DEPTH" * Thread through ChainSpec * Fix release tests * Move RuntimeFixedVector into module and rename * Add test * Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_attestation * Added more test coverage, simplified Attestation conversion, and other minor refactors * Removed unusued codepaths * Fix failing test * Implement "Remove post-altair `initialize_beacon_state_from_eth1` from specs" * Update preset YAML * Remove empty RuntimeVarList awefullness * Make max_blobs_per_block a config parameter (#6329) Squashed commit of the following: commit 04b3743ec1 Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 17:36:58 2025 +1100 Add test commit 440e854199 Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 17:24:50 2025 +1100 Move RuntimeFixedVector into module and rename commit f66e179a40 Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 17:17:17 2025 +1100 Fix release tests commit e4bfe71cd1 Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 17:05:30 2025 +1100 Thread through ChainSpec commit 063b79c16a Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 15:32:16 2025 +1100 Try fixing tests commit 88bedf09bc Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 15:04:37 2025 +1100 Revert "Remove footgun function" This reverts commit de01f923c7. commit 32483d385b Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 15:04:32 2025 +1100 Fix typo commit 2e86585b47 Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 15:04:15 2025 +1100 Move from preset to config commit 1095d60a40 Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 14:38:40 2025 +1100 Minor simplifications commit de01f923c7 Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 14:06:57 2025 +1100 Remove footgun function commit 0c2c8c4224 Merge: 21ecb58ff f51a292f7 Author: Michael Sproul <michael@sigmaprime.io> Date: Mon Jan 6 14:02:50 2025 +1100 Merge remote-tracking branch 'origin/unstable' into max-blobs-preset commit f51a292f77 Author: Daniel Knopik <107140945+dknopik@users.noreply.github.com> Date: Fri Jan 3 20:27:21 2025 +0100 fully lint only explicitly to avoid unnecessary rebuilds (#6753) * fully lint only explicitly to avoid unnecessary rebuilds commit 7e0cddef32 Author: Akihito Nakano <sora.akatsuki@gmail.com> Date: Tue Dec 24 10:38:56 2024 +0900 Make sure we have fanout peers when publish (#6738) * Ensure that `fanout_peers` is always non-empty if it's `Some` commit 21ecb58ff8 Merge: 2fcb2935e 9aefb5539 Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Mon Oct 21 14:46:00 2024 -0700 Merge branch 'unstable' into max-blobs-preset commit 2fcb2935ec Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Fri Sep 6 18:28:31 2024 -0700 Fix test from unstable commit 12c6ef118a Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Wed Sep 4 16:16:36 2024 -0700 Fix some more tests commit d37733b846 Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Wed Sep 4 12:47:36 2024 -0700 Fix test compilations commit 52bb581e07 Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Tue Sep 3 18:38:19 2024 -0700 cleanup commit e71020e3e6 Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Tue Sep 3 17:16:10 2024 -0700 Fix take impl on RuntimeFixedList commit 13f9bba647 Merge: 60100fc6b 4e675cf5d Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Tue Sep 3 16:08:59 2024 -0700 Merge branch 'unstable' into max-blobs-preset commit 60100fc6be Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Fri Aug 30 16:04:11 2024 -0700 Fix some todos commit a9cb329a22 Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Fri Aug 30 15:54:00 2024 -0700 Use empty_uninitialized and fix warnings commit 4dc6e6515e Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Fri Aug 30 15:53:18 2024 -0700 Add restrictions to RuntimeVariableList api commit 25feedfde3 Author: Pawan Dhananjay <pawandhananjay@gmail.com> Date: Thu Aug 29 16:11:19 2024 -0700 First pass * Fix tests * Implement max_blobs_per_block_electra * Fix config issues * Simplify BlobSidecarListFromRoot * Disable PeerDAS tests * Cleanup single attestation imports * Fix some single attestation network plumbing * Merge remote-tracking branch 'origin/unstable' into max-blobs-preset * Bump quota to account for new target (6) * Remove clone * Fix issue from review * Try to remove ugliness * Merge branch 'unstable' into max-blobs-preset * Merge remote-tracking branch 'origin/unstable' into electra-alpha10 * Merge commit '04b3743ec1e0b650269dd8e58b540c02430d1c0d' into electra-alpha10 * Merge remote-tracking branch 'pawan/max-blobs-preset' into electra-alpha10 * Update tests to v1.5.0-beta.0 * Merge remote-tracking branch 'origin/electra-alpha10' into single_attestation * Fix some tests * Cargo fmt * lint * fmt * Resolve merge conflicts * Merge branch 'electra-alpha10' of https://github.com/sigp/lighthouse into single_attestation * lint * Linting * fmt * Merge branch 'electra-alpha10' of https://github.com/sigp/lighthouse into single_attestation * Fmt * Fix test and add TODO * Gracefully handle slashed proposers in fork choice tests * Merge remote-tracking branch 'origin/unstable' into electra-alpha10 * Keep latest changes from max_blobs_per_block PR in codec.rs * Revert a few more regressions and add a comment * Merge branch 'electra-alpha10' of https://github.com/sigp/lighthouse into single_attestation * Disable more DAS tests * Improve validator monitor test a little * Make test more robust * Fix sync test that didn't understand blobs * Fill out cropped comment * Merge remote-tracking branch 'origin/electra-alpha10' into single_attestation * Merge remote-tracking branch 'origin/unstable' into single_attestation * Merge remote-tracking branch 'origin/unstable' into single_attestation * Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_attestation * publish_attestations should accept Either<Attestation,SingleAttestation> * log an error when failing to convert to SingleAttestation * Use Cow to avoid clone * Avoid reconverting to SingleAttestation * Tweak VC error message * update comments * update comments * pass in single attestation as ref to subnetid calculation method * Improved API, new error variants and other minor tweaks * Fix single_attestation event topic boilerplate * fix sse event failure * Add single_attestation event topic test coverage
2026-03-09 03:31:45 +00:00 · 2025-01-17 01:27:08 +07:00
parent 669932aa67
commit 06329ec2d1
22 changed files with 831 additions and 104 deletions
--- a/beacon_node/http_api/src/lib.rs
+++ b/beacon_node/http_api/src/lib.rs
@@ -44,6 +44,7 @@ pub use block_id::BlockId;
 use builder_states::get_next_withdrawals;
 use bytes::Bytes;
 use directory::DEFAULT_ROOT_DIR;
+use either::Either;
 use eth2::types::{
    self as api_types, BroadcastValidation, EndpointVersion, ForkChoice, ForkChoiceNode,
    LightClientUpdatesQuery, PublishBlockRequest, ValidatorBalancesRequestBody, ValidatorId,
@@ -86,8 +87,8 @@ use types::{
    AttesterSlashing, BeaconStateError, CommitteeCache, ConfigAndPreset, Epoch, EthSpec, ForkName,
    ForkVersionedResponse, Hash256, ProposerPreparationData, ProposerSlashing, RelativeEpoch,
    SignedAggregateAndProof, SignedBlindedBeaconBlock, SignedBlsToExecutionChange,
-    SignedContributionAndProof, SignedValidatorRegistrationData, SignedVoluntaryExit, Slot,
-    SyncCommitteeMessage, SyncContributionData,
+    SignedContributionAndProof, SignedValidatorRegistrationData, SignedVoluntaryExit,
+    SingleAttestation, Slot, SyncCommitteeMessage, SyncContributionData,
 };
 use validator::pubkey_to_validator_index;
 use version::{
@@ -1832,8 +1833,47 @@ pub fn serve<T: BeaconChainTypes>(
        .and(task_spawner_filter.clone())
        .and(chain_filter.clone());

+    let beacon_pool_path_v2 = eth_v2
+        .and(warp::path("beacon"))
+        .and(warp::path("pool"))
+        .and(task_spawner_filter.clone())
+        .and(chain_filter.clone());
+
    // POST beacon/pool/attestations
-    let post_beacon_pool_attestations = beacon_pool_path_any
+    let post_beacon_pool_attestations = beacon_pool_path
+        .clone()
+        .and(warp::path("attestations"))
+        .and(warp::path::end())
+        .and(warp_utils::json::json())
+        .and(network_tx_filter.clone())
+        .and(reprocess_send_filter.clone())
+        .and(log_filter.clone())
+        .then(
+            // V1 and V2 are identical except V2 has a consensus version header in the request.
+            // We only require this header for SSZ deserialization, which isn't supported for
+            // this endpoint presently.
+            |task_spawner: TaskSpawner<T::EthSpec>,
+             chain: Arc<BeaconChain<T>>,
+             attestations: Vec<Attestation<T::EthSpec>>,
+             network_tx: UnboundedSender<NetworkMessage<T::EthSpec>>,
+             reprocess_tx: Option<Sender<ReprocessQueueMessage>>,
+             log: Logger| async move {
+                let attestations = attestations.into_iter().map(Either::Left).collect();
+                let result = crate::publish_attestations::publish_attestations(
+                    task_spawner,
+                    chain,
+                    attestations,
+                    network_tx,
+                    reprocess_tx,
+                    log,
+                )
+                .await
+                .map(|()| warp::reply::json(&()));
+                convert_rejection(result).await
+            },
+        );
+
+    let post_beacon_pool_attestations_v2 = beacon_pool_path_v2
        .clone()
        .and(warp::path("attestations"))
        .and(warp::path::end())
@@ -1842,16 +1882,13 @@ pub fn serve<T: BeaconChainTypes>(
        .and(reprocess_send_filter)
        .and(log_filter.clone())
        .then(
-            // V1 and V2 are identical except V2 has a consensus version header in the request.
-            // We only require this header for SSZ deserialization, which isn't supported for
-            // this endpoint presently.
-            |_endpoint_version: EndpointVersion,
-             task_spawner: TaskSpawner<T::EthSpec>,
+            |task_spawner: TaskSpawner<T::EthSpec>,
             chain: Arc<BeaconChain<T>>,
-             attestations: Vec<Attestation<T::EthSpec>>,
+             attestations: Vec<SingleAttestation>,
             network_tx: UnboundedSender<NetworkMessage<T::EthSpec>>,
             reprocess_tx: Option<Sender<ReprocessQueueMessage>>,
             log: Logger| async move {
+                let attestations = attestations.into_iter().map(Either::Right).collect();
                let result = crate::publish_attestations::publish_attestations(
                    task_spawner,
                    chain,
@@ -4509,6 +4546,9 @@ pub fn serve<T: BeaconChainTypes>(
                                api_types::EventTopic::Attestation => {
                                    event_handler.subscribe_attestation()
                                }
+                                api_types::EventTopic::SingleAttestation => {
+                                    event_handler.subscribe_single_attestation()
+                                }
                                api_types::EventTopic::VoluntaryExit => {
                                    event_handler.subscribe_exit()
                                }
@@ -4736,6 +4776,7 @@ pub fn serve<T: BeaconChainTypes>(
                    .uor(post_beacon_blocks_v2)
                    .uor(post_beacon_blinded_blocks_v2)
                    .uor(post_beacon_pool_attestations)
+                    .uor(post_beacon_pool_attestations_v2)
                    .uor(post_beacon_pool_attester_slashings)
                    .uor(post_beacon_pool_proposer_slashings)
                    .uor(post_beacon_pool_voluntary_exits)
--- a/beacon_node/http_api/src/publish_attestations.rs
+++ b/beacon_node/http_api/src/publish_attestations.rs
@@ -40,17 +40,19 @@ use beacon_chain::{
    BeaconChainTypes,
 };
 use beacon_processor::work_reprocessing_queue::{QueuedUnaggregate, ReprocessQueueMessage};
+use either::Either;
 use eth2::types::Failure;
 use lighthouse_network::PubsubMessage;
 use network::NetworkMessage;
 use slog::{debug, error, warn, Logger};
+use std::borrow::Cow;
 use std::sync::Arc;
 use std::time::Duration;
 use tokio::sync::{
    mpsc::{Sender, UnboundedSender},
    oneshot,
 };
-use types::Attestation;
+use types::{Attestation, EthSpec, SingleAttestation};

 // Error variants are only used in `Debug` and considered `dead_code` by the compiler.
 #[derive(Debug)]
@@ -62,6 +64,7 @@ enum Error {
    ReprocessDisabled,
    ReprocessFull,
    ReprocessTimeout,
+    FailedConversion(#[allow(dead_code)] BeaconChainError),
 }

 enum PublishAttestationResult {
@@ -73,24 +76,39 @@ enum PublishAttestationResult {

 fn verify_and_publish_attestation<T: BeaconChainTypes>(
    chain: &Arc<BeaconChain<T>>,
-    attestation: &Attestation<T::EthSpec>,
+    either_attestation: &Either<Attestation<T::EthSpec>, SingleAttestation>,
    seen_timestamp: Duration,
    network_tx: &UnboundedSender<NetworkMessage<T::EthSpec>>,
    log: &Logger,
 ) -> Result<(), Error> {
-    let attestation = chain
-        .verify_unaggregated_attestation_for_gossip(attestation, None)
+    let attestation = convert_to_attestation(chain, either_attestation)?;
+    let verified_attestation = chain
+        .verify_unaggregated_attestation_for_gossip(&attestation, None)
        .map_err(Error::Validation)?;

-    // Publish.
-    network_tx
-        .send(NetworkMessage::Publish {
-            messages: vec![PubsubMessage::Attestation(Box::new((
-                attestation.subnet_id(),
-                attestation.attestation().clone_as_attestation(),
-            )))],
-        })
-        .map_err(|_| Error::Publication)?;
+    match either_attestation {
+        Either::Left(attestation) => {
+            // Publish.
+            network_tx
+                .send(NetworkMessage::Publish {
+                    messages: vec![PubsubMessage::Attestation(Box::new((
+                        verified_attestation.subnet_id(),
+                        attestation.clone(),
+                    )))],
+                })
+                .map_err(|_| Error::Publication)?;
+        }
+        Either::Right(single_attestation) => {
+            network_tx
+                .send(NetworkMessage::Publish {
+                    messages: vec![PubsubMessage::SingleAttestation(Box::new((
+                        verified_attestation.subnet_id(),
+                        single_attestation.clone(),
+                    )))],
+                })
+                .map_err(|_| Error::Publication)?;
+        }
+    }

    // Notify the validator monitor.
    chain
@@ -98,12 +116,12 @@ fn verify_and_publish_attestation<T: BeaconChainTypes>(
        .read()
        .register_api_unaggregated_attestation(
            seen_timestamp,
-            attestation.indexed_attestation(),
+            verified_attestation.indexed_attestation(),
            &chain.slot_clock,
        );

-    let fc_result = chain.apply_attestation_to_fork_choice(&attestation);
-    let naive_aggregation_result = chain.add_to_naive_aggregation_pool(&attestation);
+    let fc_result = chain.apply_attestation_to_fork_choice(&verified_attestation);
+    let naive_aggregation_result = chain.add_to_naive_aggregation_pool(&verified_attestation);

    if let Err(e) = &fc_result {
        warn!(
@@ -129,10 +147,48 @@ fn verify_and_publish_attestation<T: BeaconChainTypes>(
    }
 }

+fn convert_to_attestation<'a, T: BeaconChainTypes>(
+    chain: &Arc<BeaconChain<T>>,
+    attestation: &'a Either<Attestation<T::EthSpec>, SingleAttestation>,
+) -> Result<Cow<'a, Attestation<T::EthSpec>>, Error> {
+    let a = match attestation {
+        Either::Left(a) => Cow::Borrowed(a),
+        Either::Right(single_attestation) => chain
+            .with_committee_cache(
+                single_attestation.data.target.root,
+                single_attestation
+                    .data
+                    .slot
+                    .epoch(T::EthSpec::slots_per_epoch()),
+                |committee_cache, _| {
+                    let Some(committee) = committee_cache.get_beacon_committee(
+                        single_attestation.data.slot,
+                        single_attestation.committee_index as u64,
+                    ) else {
+                        return Err(BeaconChainError::AttestationError(
+                            types::AttestationError::NoCommitteeForSlotAndIndex {
+                                slot: single_attestation.data.slot,
+                                index: single_attestation.committee_index as u64,
+                            },
+                        ));
+                    };
+
+                    let attestation =
+                        single_attestation.to_attestation::<T::EthSpec>(committee.committee)?;
+
+                    Ok(Cow::Owned(attestation))
+                },
+            )
+            .map_err(Error::FailedConversion)?,
+    };
+
+    Ok(a)
+}
+
 pub async fn publish_attestations<T: BeaconChainTypes>(
    task_spawner: TaskSpawner<T::EthSpec>,
    chain: Arc<BeaconChain<T>>,
-    attestations: Vec<Attestation<T::EthSpec>>,
+    attestations: Vec<Either<Attestation<T::EthSpec>, SingleAttestation>>,
    network_tx: UnboundedSender<NetworkMessage<T::EthSpec>>,
    reprocess_send: Option<Sender<ReprocessQueueMessage>>,
    log: Logger,
@@ -141,7 +197,10 @@ pub async fn publish_attestations<T: BeaconChainTypes>(
    // move the `attestations` vec into the blocking task, so this small overhead is unavoidable.
    let attestation_metadata = attestations
        .iter()
-        .map(|att| (att.data().slot, att.committee_index()))
+        .map(|att| match att {
+            Either::Left(att) => (att.data().slot, att.committee_index()),
+            Either::Right(att) => (att.data.slot, Some(att.committee_index as u64)),
+        })
        .collect::<Vec<_>>();

    // Gossip validate and publish attestations that can be immediately processed.