Fixes intermittent simulator test failures with error: `Head not synced for node 2. Found 127; Should be 128`
Modify the delayed node in `basic_sim` to join earlier, giving it sufficient time to discover peers and form a proper gossip mesh before the sync verification check.
**Change:** Delayed node now joins at `END_EPOCH - 3` (epoch 13) instead of `END_EPOCH - 1` (epoch 15).
Co-Authored-By: Mark Mackey <mark@sigmaprime.io>
Co-Authored-By: ethDreamer <37123614+ethDreamer@users.noreply.github.com>
Changes four `ValidatorStore` batch signing methods to return `impl Stream` instead of `Future`. Services consume the stream and publish each batch as it arrives. No behavioral change for lh since `LighthouseValidatorStore` wraps everything in `stream::once`
Also replaces anonymous tuples in method signatures with named structs
Co-Authored-By: shane-moore <skm1790@gmail.com>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: Mac L <mjladson@pm.me>
Closes:
- https://github.com/sigp/lighthouse/issues/8869
- Update `BlockReplayer` to support replay of execution payload envelopes.
- Update `HotColdDB` to load payload envelopes and feed them to the `BlockReplayer` for both hot + cold states. However the cold DB code is not fully working yet (see: https://github.com/sigp/lighthouse/issues/8958).
- Add `StatePayloadStatus` to allow callers to specify whether they want a state with a payload applied, or not.
- Fix the state cache to key by `StatePayloadStatus`.
- Lots of fixes to block production and block processing regarding state management.
- Initial test harness support for producing+processing Gloas blocks+envelopes
- A few new tests to cover Gloas DB operations
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
#7330
Removes `c-kzg` from our `kzg` crate and rely fully on the `rust_eth_kzg` crate.
This removes the old `Blob` type entirely and instead handles `rust_eth_kzg::KzgBlobRef`s directly which allows us to avoid some extra stack allocations . Similarly, we make `Bytes32` and `Bytes48` type aliases rather than structs as this fits better with the new `rust_eth_kzg` API.
Co-Authored-By: Mac L <mjladson@pm.me>
Fix the issue with the `proposer_duties` endpoint using the wrong dependent root post-Fulu by implementing the new v2 endpoint:
- https://github.com/ethereum/beacon-APIs/pull/563
We need to add this in time for Gloas, and then we can we can deprecate and remove v1.
- Add a new API handler for the v2 endpoint
- Add client code in the `eth2` crate
- Update existing tests and add some new ones to confirm the different behaviour of v1 and v2
There's a bit of test duplication with v1, but this will be resolved once v1 and its tests are deleted.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: chonghe <44791194+chong-he@users.noreply.github.com>
Following the release of Rust v1.94.0 there are new Clippy lints which do not pass and are blocking CI (which pulls in the latest version of Rust)
This is pretty much the minimum just to get CI running again. Most of the errors involve error types being too large. For now I've added allows but later it might be worth doing a refactor to `Box` or otherwise remove the problematic error types.
Co-Authored-By: Mac L <mjladson@pm.me>
Mark pointed out that these APIs will require updates for Gloas, so I figured we may as well get rid of them. As far as I know, blockprint was the only use case and it is now defunct. The consensus block value is included in getBlock API responses, so there's no reason for VCs to use the `POST` API, and there is now a standard API for the rewards of canonical blocks. The SSE event was non-standard, and likely only used by blockprint as well.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
We have duplicated features which enable `arbitrary` throughout the codebase. These are `arbitrary` and `arbitrary-fuzz`. I think historically these were supposed to be distinct however in practice these function identically and so we can unify them into a single feature to avoid confusion.
Co-Authored-By: Mac L <mjladson@pm.me>
Following the release of Rust v1.94.0 there are new Clippy lints which do not pass and are blocking CI (which pulls in the latest version of Rust)
This is pretty much the minimum just to get CI running again. Most of the errors involve error types being too large. For now I've added allows but later it might be worth doing a refactor to `Box` or otherwise remove the problematic error types.
Co-Authored-By: Mac L <mjladson@pm.me>
Create a `testing` feature which we can use to gate off `test_utils.rs` and its associated dependencies from the rest of the crate.
Co-Authored-By: Mac L <mjladson@pm.me>
We received a bug report of a node restarting custody backfill unnecessarily after upgrading to Lighthouse v8.1.1. What happened is:
- User started LH v8.0.1 many months ago, CGC updated 0 -> N but the CGC was not eagerly persisted.
- LH experienced an unclean shutdown (not sure of what type).
- Upon restarting (still running v8.0.1), the custody context read from disk contains CGC=0: `DEBUG Loaded persisted custody context custody_context: CustodyContext { validator_custody_count: 0, ...`).
- CGC updates again to N, retriggering custody backfill: `DEBUG Validator count at head updated old_count: 0, new_count: N`.
- Custody backfill does a bunch of downloading for no gain: `DEBUG Imported historical data columns epoch: Epoch(428433), total_imported: 0`
- While custody backfill is running user updated to v8.1.1, and we see logs for the CGC=N being peristed upon clean shutdown, and then correctly read on startup with v8.1.1.
- Custody backfill keeps running and downloading due to the CGC change still being considered in progress.
- Call `persist_custody_context` inside the `register_validators` handler so that it is written to disk eagerly whenever it changes. The performance impact of this should be minimal as the amount of data is very small and this call can only happen at most ~128 times (once for each change) in the entire life of a beacon node.
- Call `persist_custody_context` inside `BeaconChainBuilder::build` so that changes caused by CLI flags are persisted (otherwise starting a node with `--semi-supernode` and no validators, then shutting it down uncleanly would cause use to forget the CGC).
These changes greatly reduce the timespan during which an unclean shutdown can create inconsistency. In the worst case, we only lose backfill progress that runs concurrently with the `register_validators` handler (should be extremely minimal, nigh impossible).
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
I found myself having to update this code for Gloas, and figured we may as well delete it seeing as it doesn't work.
See:
- https://github.com/sigp/lighthouse/issues/4198
Delete all `fork_revert` logic and the accompanying test.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Which issue # does this PR address?
None
All of these are performing a check, and adding a batch, or creating a new lookup, or a new query, etc..
Hence all of these limits would be off by one.
Example:
```rust
// BACKFILL_BATCH_BUFFER_SIZE = 5
if self.batches.iter().filter(...).count() >= BACKFILL_BATCH_BUFFER_SIZE {
return None; // ← REJECT
}
// ... later adds batch via Entry::Vacant(entry).insert(...)
```
Without the `>` being changed to a `>=` , we would allow 6. The same idea applies to all changes proposed.
Co-Authored-By: Antoine James <antoine@ethereum.org>
Co-Authored-By: Jimmy Chen <jimmy@sigmaprime.io>
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
#8547
Bump the version of `uuid` in our Cargo.toml to version `1` which removes `uuid 0.8` and unifies it across the workspace to version `1.19.0`.
Co-Authored-By: Mac L <mjladson@pm.me>
N/A
Another find by @gitToki. Sort the preferred_ids in descending order as originally intended from the comment in the function.
Co-Authored-By: Pawan Dhananjay <pawandhananjay@gmail.com>
Fix the failure of the beacon-chain tests for phase0/altair, which now only runs nightly.
Just skip the payload invalidation tests, they don't make any sense prior to Bellatrix anyway.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Fix a bug in v8.1.0 whereby the VC times out continuously with:
> Feb 18 02:03:48.030 WARN Head service failed retrying starting next slot error: "Head monitoring stream error, node: 0, error: SseClient(Transport(reqwest::Error { kind: Decode, source: reqwest::Error { kind: Body, source: TimedOut } }))"
- Remove the existing timeout for the events API by using `Duration::MAX`. This is necessary as the client is configured with a default timeout. This is the only way to override/remove it.
- DO NOT add a `read_timeout` (yet), as this would need to be configured on a per-client basis. We do not want to create a new Client for every call as the early commits on this branch were doing, as this would bypass the TLS cert config, and is also wasteful.
Co-Authored-By: hopinheimer <knmanas6@gmail.com>
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
