#8314 left a few ugly potentially panicking location behind - all of them believed to be unreachable, but this PR fixes them regardless for good hygiene.
Update to `ethereum_ssz 0.10.4` for two new helpers: `not_inplace` and `clone_zeroed`.
Remove remaining `expect` and `todo!` in favour of these helpers and one new fallible (but practically infallible) method.
Co-Authored-By: Daniel Knopik <daniel@dknopik.de>
In Gloas, beacon blocks are imported into fork choice immediately - the payload envelope and data columns arrive
separately. KZG commitments moved from the column sidecar into the execution payload bid, so the existing
`DataAvailabilityChecker` (which assumes block and data are coupled) can't be used for Gloas.
* Introduced `PendingPayloadCache` to keep track of payload and data columns per block root.
* Added gossip column verification
* Added support for Gloas data column reconstruction
* Payload envelope verification simplified: removed `MaybeAvailableEnvelope`, `ExecutedEnvelope`, `EnvelopeImportData`
Not yet implemented (tracked with TODOs):
- Proper lookup sync for Gloas columns arriving before blocks
- Partial column merging for Gloas
- Moving `load_gloas_payload_bid` disk reads off the async runtime
- Backfill/range sync for Gloas
Based on @eserilev's PR and work in progress. See also #9202 for verification.
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Co-Authored-By: Daniel Knopik <daniel@dknopik.de>
Co-Authored-By: Daniel Knopik <107140945+dknopik@users.noreply.github.com>
Co-Authored-By: dapplion <35266934+dapplion@users.noreply.github.com>
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
Using the `ignore-ws-check` doesn't actually let you start up a node thats outside the weak subjectivity period
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Test helpers `add_attested_block_at_slot` and `add_attested_blocks_at_slot` accepted `state_root` argument which was computed before applying the block.
Co-Authored-By: hopinheimer <knmanas6@gmail.com>
The previous check triggered ParentEnvelopeUnknown for any block whose
parent didn't have its envelope received, even when the block's bid
didn't actually reference that parent's payload (e.g. building on
grandparent's execution state).
This caused a permanent stall when an envelope was rejected (e.g.
WithdrawalsRootMismatch from a buggy proposer): the parent's
payload_received stayed false, and all subsequent child blocks would
trigger infinite lookup retries for the broken envelope.
Fix: only require the parent's envelope if the block's bid
parent_block_hash matches the parent's execution_payload_block_hash,
meaning this block directly depends on the parent's payload.
When gloas envelopes are imported optimistically (EL returns SYNCING),
the payload status is Pending. Previously, Pending used
execution_payload_parent_hash for the head_hash in forkchoiceUpdated,
which tells geth to stay at the parent — never advancing.
Fix: use execution_payload_block_hash (the bid's committed hash) for
Pending status, same as Full. This tells geth to sync to the new head,
which is the purpose of optimistic sync. Geth will validate it and
transition from SYNCING to VALID on subsequent newPayload calls.
Also re-enables backfill sync for gloas with a dedicated
import_historical_gloas_block_batch that properly handles RangeSyncBlock::Gloas
variants (storing envelopes alongside blocks).
Remove the OptimisticSyncNotSupported check in into_executed_payload_envelope.
When the EL returns SYNCING (optimistic) for a payload envelope's newPayload
call, the envelope import should proceed rather than being rejected. This
commonly occurs after range sync imports blocks that the EL hasn't yet
validated - subsequent envelopes arriving via gossip would be rejected because
the EL is still catching up.
The downstream import path already handles optimistic status correctly:
- fork choice marks the payload as received regardless of EL status
- the payload_verification_status is propagated to SSE events/metrics
- if the payload is later found invalid, the normal invalidation mechanism
handles it (same as for blocks)
1. Remove on_valid_payload_envelope_received call before process_block
in chain segment import. The block isn't in fork choice yet, so it
always fails with NodeUnknown. import_envelope_from_range_sync
handles this correctly after process_block.
2. Disable backfill sync when GLOaS is scheduled. Backfill calls
into_available_block which panics on GLOaS RangeSyncBlock variant.
Backfill for GLOaS is not yet implemented.
Two fixes for GLOaS range sync:
1. DuplicateFullyImported: persist envelope for blocks that are already
in fork choice (e.g. post-checkpoint-sync blocks between finalized
and head).
2. load_parent: if parent envelope isn't in store, check if parent is
already in fork choice. If it is, the parent was already imported
and validated — proceed without requiring the envelope in store.
This handles the case where PayloadEnvelopesByRange doesn't return
envelopes for all blocks (fewer envelopes than blocks).
We have a legacy `TestRandom` trait which generates random types for testing and fuzzing.
This function overlaps with `arbitrary` which is used very commonly in the ecosystem.
Remove `TestRandom` and generate random type instances using `Arbitrary`.
Co-Authored-By: Mac L <mjladson@pm.me>
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Allow for the vc to submit its proposer preferences to the network
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
- pending_column.rs: split try_to_sidecar into is_complete-checked to_sidecar
with typed PendingColumnError so 'incomplete column' is no longer conflated
with VariableList size-bound failures (jimmygchen, dapplion).
- pending_components.rs: get_cached_data_columns filters by is_complete first,
then logs an error if a complete column fails to assemble (dknopik's sanity
check on filter_map silent drops).
- data_column_verification.rs: add the missing column.slot == bid.slot
consistency check in validate_data_column_sidecar_for_gossip_gloas, using
the previously-defined-but-unused BlockSlotMismatch error variant
(jimmygchen).
Brings the FORK_NAME=gloas beacon_chain test suite from 31 failures to green:
- v1 KZG batch verifier couldn't verify Gloas columns. Added
verify_columns_against_block helper that picks commitments per fork
(Fulu: inline on column; Gloas: signed_execution_payload_bid).
- BeaconChainHarness::process_envelope didn't persist columns. Now mirrors
what production does in import_available_execution_payload_envelope.
- get_or_reconstruct_blobs returned an error for Gloas. Now short-circuits to
Ok(None); WSS test copies columns from source to dest directly.
- update_data_column_signed_header (block_verification tests) only handled
Fulu shape. Added a Gloas branch that re-keys to canonical_root.
- BlockError::EnvelopeBlockRootUnknown changed to tuple variant.
- Removed duplicate process_payload_envelope_availability.
Brings the FORK_NAME=gloas beacon_chain test suite from 31 failures to green:
- v1 KZG batch verifier couldn't verify Gloas columns. Added
verify_columns_against_block helper that picks commitments per fork
(Fulu: inline on column; Gloas: signed_execution_payload_bid).
- BeaconChainHarness::process_envelope didn't persist columns. Now mirrors
what production does in import_available_execution_payload_envelope.
- get_or_reconstruct_blobs returned an error for Gloas. Now short-circuits to
Ok(None); WSS test copies columns from source to dest directly.
- update_data_column_signed_header (block_verification tests) only handled
Fulu shape. Added a Gloas branch that re-keys to canonical_root.
- BlockError::EnvelopeBlockRootUnknown changed to tuple variant.
- Removed duplicate process_payload_envelope_availability.
- Replace the `PendingPayloadBid` projection (slot + blob_kzg_commitments)
with `Arc<SignedExecutionPayloadBid<E>>`. Cloning becomes a cheap Arc
bump and the bid carries enough context for future bid<->envelope
cross-checks. Add a `signed_payload_bid_from_block` helper.
- `PendingColumn` switches from a pre-sized `Vec<Option<_>>` to a sparse
`HashMap<usize, _>`; the `new_with_capacity(num_blobs)` constructor is
gone since callers no longer need to know the blob count up front.
- `PendingComponents::merge_data_columns` takes a slice instead of an
owning iterator (it only borrows + clones cells).
- Store `block_root` in `PendingComponents` so `make_available` and
`get_cached_data_columns` no longer require it as an argument (the
arg was misnamed `block_hash` in `make_available`).
- Rename `PendingComponents::empty` -> `new`; it is the only constructor.
