Brings the FORK_NAME=gloas beacon_chain test suite from 31 failures to green:
- v1 KZG batch verifier couldn't verify Gloas columns. Added
verify_columns_against_block helper that picks commitments per fork
(Fulu: inline on column; Gloas: signed_execution_payload_bid).
- BeaconChainHarness::process_envelope didn't persist columns. Now mirrors
what production does in import_available_execution_payload_envelope.
- get_or_reconstruct_blobs returned an error for Gloas. Now short-circuits to
Ok(None); WSS test copies columns from source to dest directly.
- update_data_column_signed_header (block_verification tests) only handled
Fulu shape. Added a Gloas branch that re-keys to canonical_root.
- BlockError::EnvelopeBlockRootUnknown changed to tuple variant.
- Removed duplicate process_payload_envelope_availability.
- Replace the `PendingPayloadBid` projection (slot + blob_kzg_commitments)
with `Arc<SignedExecutionPayloadBid<E>>`. Cloning becomes a cheap Arc
bump and the bid carries enough context for future bid<->envelope
cross-checks. Add a `signed_payload_bid_from_block` helper.
- `PendingColumn` switches from a pre-sized `Vec<Option<_>>` to a sparse
`HashMap<usize, _>`; the `new_with_capacity(num_blobs)` constructor is
gone since callers no longer need to know the blob count up front.
- `PendingComponents::merge_data_columns` takes a slice instead of an
owning iterator (it only borrows + clones cells).
- Store `block_root` in `PendingComponents` so `make_available` and
`get_cached_data_columns` no longer require it as an argument (the
arg was misnamed `block_hash` in `make_available`).
- Rename `PendingComponents::empty` -> `new`; it is the only constructor.
Closes:
- https://github.com/sigp/lighthouse/issues/8689
- Calculate the proposer index on the canonical chain (from canonical head) at `slot` and plumb it through to fork choice so it can be used to determine whether or not to apply the proposer boost. We use the proposer cache to handle state advances and avoid duplicate work.
- Update our FC tests to use `block.message().proposer_index()` (always pass), we are not attempting to test this feature in those tests. The EF tests use the correct canonical proposer idnex via `on_block`, except for invalid blocks which just auto-pass this check (these blocks get rejected by other checks in `on_block` anyway).
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
For gloas `attestation.data.index` should be set to 1 if we are attesting to a block whose slot is not the attestation duty slot and slot payload_status is `FULL`
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Co-Authored-By: dapplion <35266934+dapplion@users.noreply.github.com>
Just a little naming cleanup (no semantic changes) to remove mentions of pending and full states that were still lurking.
This hopefully helps Claude forget about the concept (it defaults to naming variables `pending_state`s without this change).
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Fix a vulnerability in the slasher whereby it would OOM upon processing an invalid attestation with an artificially high `validator_index`. This fix has already been made available to affected users on the `slasher-fix` branch.
- Prevent attestations from being passed to the slasher prior to signature verification. This was unnecessary, as they would later be passed on successful validation as well.
- Add a defensive cap on the maximum validator index processable by the slasher. The cap is high enough that it shouldn't be reached for several years, and will quickly result in warning logs if forgotten.
- Add a regression test that confirms that the issue is fixed.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Closes:
- https://github.com/sigp/lighthouse/issues/9002
- Commit `config.yaml` for minimal and mainnet to `consensus/types/configs`. For now we omit any auto-downloading logic, to avoid the hassles of dealing with Github rate limits etc on CI. Unfortunately these files are NOT bundled inside the spec tests.
- Fix the values of `min_builder_withdrawability_delay` for minimal and mainnet. These discrepancies aren't caught by the current spec tests, because the spec tests are missing data: https://github.com/ethereum/consensus-specs/pull/5005. Will be fixed in the next release/when we update to nightly.
- Fix the blob schedule for `minimal`, which should be empty, NOT inherited from mainnet.
- Keep `SECONDS_PER_SLOT` for now because the Kurtosis tests fail upon their complete removal. We will be able to completely remove `SECONDS_PER_SLOT` soon.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
With LH v8.1.3 supporting Fulu-on-Gnosis, we no longer need these DB migrations. All Lighthouse nodes running in prod will soon be updated to LH v8.0.0+ and schema v28+.
This PR helps with Gloas fork choice changes, by allowing us to avoid updating old schema migrations when adding V29 for Gloas:
- https://github.com/sigp/lighthouse/pull/9025
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Closes:
- https://github.com/sigp/lighthouse/issues/8958
- Update the `HotColdStore` to handle storage of cold states.
- Update `BeaconSnapshot` to hold the execution envelope. This is required to make `chain_dump`-related checks sane, and will be generally useful (see: https://github.com/sigp/lighthouse/issues/8956).
- Bug fix in the `BlockReplayer` for the case where the starting state is already `Full` (we should not try to apply another payload). This happens on the cold DB path because we try to replay from the closest cached state (which is often full).
- Update `test_gloas_hot_state_hierarchy` to cover the cold DB migration.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Closes:
- https://github.com/sigp/lighthouse/issues/8869
- Update `BlockReplayer` to support replay of execution payload envelopes.
- Update `HotColdDB` to load payload envelopes and feed them to the `BlockReplayer` for both hot + cold states. However the cold DB code is not fully working yet (see: https://github.com/sigp/lighthouse/issues/8958).
- Add `StatePayloadStatus` to allow callers to specify whether they want a state with a payload applied, or not.
- Fix the state cache to key by `StatePayloadStatus`.
- Lots of fixes to block production and block processing regarding state management.
- Initial test harness support for producing+processing Gloas blocks+envelopes
- A few new tests to cover Gloas DB operations
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
Following the release of Rust v1.94.0 there are new Clippy lints which do not pass and are blocking CI (which pulls in the latest version of Rust)
This is pretty much the minimum just to get CI running again. Most of the errors involve error types being too large. For now I've added allows but later it might be worth doing a refactor to `Box` or otherwise remove the problematic error types.
Co-Authored-By: Mac L <mjladson@pm.me>
I found myself having to update this code for Gloas, and figured we may as well delete it seeing as it doesn't work.
See:
- https://github.com/sigp/lighthouse/issues/4198
Delete all `fork_revert` logic and the accompanying test.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Fix the failure of the beacon-chain tests for phase0/altair, which now only runs nightly.
Just skip the payload invalidation tests, they don't make any sense prior to Bellatrix anyway.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
I accidentally broke `unstable` while merging some missed commits from `release-v8.0`. The merge was clean but semantically broken, and I didn't notice because I pushed without running CI 😬
- Fix the regression test added for #8528, for compatibility with the recent `RpcBlock` changes. I'm passing `is_available = false` which seems correct for this test.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Adds support for payload envelopes in the db. This is the minimum we'll need to store and fetch payloads.
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
