#8926
Add a step to CI which runs `cargo check` across all combinations of features for certain crates using `cargo-hack`
Co-Authored-By: Mac L <mjladson@pm.me>
Emit `execution_payload_gossip` on successful gossip verification of an execution payload. This is done as last step inside the verification function.
Co-Authored-By: Daniel Knopik <daniel@dknopik.de>
Lighthouse uses `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` for blob **and column retention** instead of `MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS` if Fulu activated less than `MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS` epochs ago - also if Fulu activated at genesis. This causes unexpected behaviour, as there are no blob sidecars to be stored or requested in such networks.
~~Add a special case to avoid that logic in post-Fulu genesis networks (`fulu_fork_epoch == 0`)~~
If the blob retention period would start in the fulu fork epoch, use the `min_epochs_for_data_column_sidecars_requests`, as there are no blobs to retain in Fulu.
Co-Authored-By: Daniel Knopik <daniel@dknopik.de>
Implement boilerplate for new SSE events as specified in
- https://github.com/ethereum/beacon-APIs/pull/588
While that one is not merged yet, I believe the SSE events might be utilized in Dora already.
Implement the boilerplate, i.e. subscription tracking and publish queues. A PR to implement to fully implement already implementable events will follow.
Co-Authored-By: Daniel Knopik <daniel@dknopik.de>
- Pin Tempo image to `grafana/tempo:2.10.3` — `grafana/tempo:latest` now resolves to an unreleased 3.0 build that removed the `compactor` config field, causing startup failure
- Replace deprecated `prometheus_grafana` additional service with separate `prometheus` + `grafana` services
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
Closes:
- https://github.com/sigp/lighthouse/issues/9002
- Commit `config.yaml` for minimal and mainnet to `consensus/types/configs`. For now we omit any auto-downloading logic, to avoid the hassles of dealing with Github rate limits etc on CI. Unfortunately these files are NOT bundled inside the spec tests.
- Fix the values of `min_builder_withdrawability_delay` for minimal and mainnet. These discrepancies aren't caught by the current spec tests, because the spec tests are missing data: https://github.com/ethereum/consensus-specs/pull/5005. Will be fixed in the next release/when we update to nightly.
- Fix the blob schedule for `minimal`, which should be empty, NOT inherited from mainnet.
- Keep `SECONDS_PER_SLOT` for now because the Kurtosis tests fail upon their complete removal. We will be able to completely remove `SECONDS_PER_SLOT` soon.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
`serde_yaml` is now deprecated. The API-compatible `yaml_serde` should be used instead.
Replace `serde_yaml` with `yaml_serde`. This is purely mechanical as the API is 1-to-1.
Co-Authored-By: Mac L <mjladson@pm.me>
With LH v8.1.3 supporting Fulu-on-Gnosis, we no longer need these DB migrations. All Lighthouse nodes running in prod will soon be updated to LH v8.0.0+ and schema v28+.
This PR helps with Gloas fork choice changes, by allowing us to avoid updating old schema migrations when adding V29 for Gloas:
- https://github.com/sigp/lighthouse/pull/9025
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Serves envelope by range and by root requests. Added PayloadEnvelopeStreamer so that we dont need to alter upstream code when we introduce blinded payload envelopes.
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Co-Authored-By: dapplion <35266934+dapplion@users.noreply.github.com>
Closes:
- https://github.com/sigp/lighthouse/issues/9003
Milhouse `List`s use a map in front of the binary tree to cache updates. Ever since we adopted Milhouse, we've been using `VecMap`, which is essentially `Vec<Option<T>>`. Turns out, when you've got 2M indices and only 2 non-`None` entries (changes), this is inefficient.
Milhouse is generic in the choice of map (`U: UpdateMap`) and has always supported `BTreeMap`, so this PR switches us over to `BTreeMap`. In previous benchmarks (years ago) it had been slower than `VecMap`, but now it is vastly superior.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Our release workflow is pretty inefficient and slow. This PR aims to consolidate and cut down on duplicate tasks.
1) We now run the whole build process both on pushing to the `stable` branch and pushing a version tag.
A quick win is to not fire off separate builds.
~~2) The Docker release workflow could re-use the binaries being built instead of doing its own cross-compilation. ~~
we won't take this on _right now_
Co-Authored-By: antondlr <anton@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Closes:
- https://github.com/sigp/lighthouse/issues/8958
- Update the `HotColdStore` to handle storage of cold states.
- Update `BeaconSnapshot` to hold the execution envelope. This is required to make `chain_dump`-related checks sane, and will be generally useful (see: https://github.com/sigp/lighthouse/issues/8956).
- Bug fix in the `BlockReplayer` for the case where the starting state is already `Full` (we should not try to apply another payload). This happens on the cold DB path because we try to replay from the closest cached state (which is often full).
- Update `test_gloas_hot_state_hierarchy` to cover the cold DB migration.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Some of our custom `lighthouse/analysis` endpoints will require maintenance for the Gloas hard fork. We have decided instead to remove those endpoints. We don't utilize them internally and they have pretty limited utility and so we feel they are not worth maintaining.
Remove `lighthouse/analysis/attestation_performance` and `lighthouse/analysis/block_packing_efficiency` endpoints.
Co-Authored-By: Mac L <mjladson@pm.me>
Add a queue that allows us to reprocess an envelope when it arrives over gossip references a unknown block root. When the block is finally imported, we immediately reprocess the queued envelope.
Note that we don't trigger a block lookup sync. Incoming attestations for this block root will already trigger a lookup for us. I think thats good enough
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Update spec code for compliance with spec v1.7.0-alpha.3: https://github.com/ethereum/consensus-specs/releases/tag/v1.7.0-alpha.3
The actual consensus changes are minimal. There are few more changes that are only relevant to fork choice or P2P validation that we will pick up in future PRs.
The change "Ignore beacon block if parent payload unknown" is currently covered in a hacky way by `load_parent` and can be improved once we have fork choice.
The change "Add parent_block_root to bid filtering key" is relevant to bid gossip validation, which we don't have at all in unstable yet.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
I believe one of our rewards endpoints is slightly out of spec. We do not return the `finalized` status for `post_beacon_rewards_attestations`.
Additionally, the `eth2` client doesn't expect the correct wrapper types for some other endpoints.
- Update `post_beacon_rewards_attestations` server implementation to match spec.
- Update all three client functions in `eth2` to the correct wrapper type.
- Add missing tests for `http_api` to detect any regressions.
Co-Authored-By: Mac L <mjladson@pm.me>
Fixes intermittent simulator test failures with error: `Head not synced for node 2. Found 127; Should be 128`
Modify the delayed node in `basic_sim` to join earlier, giving it sufficient time to discover peers and form a proper gossip mesh before the sync verification check.
**Change:** Delayed node now joins at `END_EPOCH - 3` (epoch 13) instead of `END_EPOCH - 1` (epoch 15).
Co-Authored-By: Mark Mackey <mark@sigmaprime.io>
Co-Authored-By: ethDreamer <37123614+ethDreamer@users.noreply.github.com>
Changes four `ValidatorStore` batch signing methods to return `impl Stream` instead of `Future`. Services consume the stream and publish each batch as it arrives. No behavioral change for lh since `LighthouseValidatorStore` wraps everything in `stream::once`
Also replaces anonymous tuples in method signatures with named structs
Co-Authored-By: shane-moore <skm1790@gmail.com>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: Mac L <mjladson@pm.me>
Closes:
- https://github.com/sigp/lighthouse/issues/8869
- Update `BlockReplayer` to support replay of execution payload envelopes.
- Update `HotColdDB` to load payload envelopes and feed them to the `BlockReplayer` for both hot + cold states. However the cold DB code is not fully working yet (see: https://github.com/sigp/lighthouse/issues/8958).
- Add `StatePayloadStatus` to allow callers to specify whether they want a state with a payload applied, or not.
- Fix the state cache to key by `StatePayloadStatus`.
- Lots of fixes to block production and block processing regarding state management.
- Initial test harness support for producing+processing Gloas blocks+envelopes
- A few new tests to cover Gloas DB operations
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
#7330
Removes `c-kzg` from our `kzg` crate and rely fully on the `rust_eth_kzg` crate.
This removes the old `Blob` type entirely and instead handles `rust_eth_kzg::KzgBlobRef`s directly which allows us to avoid some extra stack allocations . Similarly, we make `Bytes32` and `Bytes48` type aliases rather than structs as this fits better with the new `rust_eth_kzg` API.
Co-Authored-By: Mac L <mjladson@pm.me>
