Some of our custom `lighthouse/analysis` endpoints will require maintenance for the Gloas hard fork. We have decided instead to remove those endpoints. We don't utilize them internally and they have pretty limited utility and so we feel they are not worth maintaining.
Remove `lighthouse/analysis/attestation_performance` and `lighthouse/analysis/block_packing_efficiency` endpoints.
Co-Authored-By: Mac L <mjladson@pm.me>
I believe one of our rewards endpoints is slightly out of spec. We do not return the `finalized` status for `post_beacon_rewards_attestations`.
Additionally, the `eth2` client doesn't expect the correct wrapper types for some other endpoints.
- Update `post_beacon_rewards_attestations` server implementation to match spec.
- Update all three client functions in `eth2` to the correct wrapper type.
- Add missing tests for `http_api` to detect any regressions.
Co-Authored-By: Mac L <mjladson@pm.me>
Closes:
- https://github.com/sigp/lighthouse/issues/8869
- Update `BlockReplayer` to support replay of execution payload envelopes.
- Update `HotColdDB` to load payload envelopes and feed them to the `BlockReplayer` for both hot + cold states. However the cold DB code is not fully working yet (see: https://github.com/sigp/lighthouse/issues/8958).
- Add `StatePayloadStatus` to allow callers to specify whether they want a state with a payload applied, or not.
- Fix the state cache to key by `StatePayloadStatus`.
- Lots of fixes to block production and block processing regarding state management.
- Initial test harness support for producing+processing Gloas blocks+envelopes
- A few new tests to cover Gloas DB operations
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
Fix the issue with the `proposer_duties` endpoint using the wrong dependent root post-Fulu by implementing the new v2 endpoint:
- https://github.com/ethereum/beacon-APIs/pull/563
We need to add this in time for Gloas, and then we can we can deprecate and remove v1.
- Add a new API handler for the v2 endpoint
- Add client code in the `eth2` crate
- Update existing tests and add some new ones to confirm the different behaviour of v1 and v2
There's a bit of test duplication with v1, but this will be resolved once v1 and its tests are deleted.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Michael Sproul <michaelsproul@users.noreply.github.com>
Co-Authored-By: chonghe <44791194+chong-he@users.noreply.github.com>
Mark pointed out that these APIs will require updates for Gloas, so I figured we may as well get rid of them. As far as I know, blockprint was the only use case and it is now defunct. The consensus block value is included in getBlock API responses, so there's no reason for VCs to use the `POST` API, and there is now a standard API for the rewards of canonical blocks. The SSE event was non-standard, and likely only used by blockprint as well.
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Following the release of Rust v1.94.0 there are new Clippy lints which do not pass and are blocking CI (which pulls in the latest version of Rust)
This is pretty much the minimum just to get CI running again. Most of the errors involve error types being too large. For now I've added allows but later it might be worth doing a refactor to `Box` or otherwise remove the problematic error types.
Co-Authored-By: Mac L <mjladson@pm.me>
We received a bug report of a node restarting custody backfill unnecessarily after upgrading to Lighthouse v8.1.1. What happened is:
- User started LH v8.0.1 many months ago, CGC updated 0 -> N but the CGC was not eagerly persisted.
- LH experienced an unclean shutdown (not sure of what type).
- Upon restarting (still running v8.0.1), the custody context read from disk contains CGC=0: `DEBUG Loaded persisted custody context custody_context: CustodyContext { validator_custody_count: 0, ...`).
- CGC updates again to N, retriggering custody backfill: `DEBUG Validator count at head updated old_count: 0, new_count: N`.
- Custody backfill does a bunch of downloading for no gain: `DEBUG Imported historical data columns epoch: Epoch(428433), total_imported: 0`
- While custody backfill is running user updated to v8.1.1, and we see logs for the CGC=N being peristed upon clean shutdown, and then correctly read on startup with v8.1.1.
- Custody backfill keeps running and downloading due to the CGC change still being considered in progress.
- Call `persist_custody_context` inside the `register_validators` handler so that it is written to disk eagerly whenever it changes. The performance impact of this should be minimal as the amount of data is very small and this call can only happen at most ~128 times (once for each change) in the entire life of a beacon node.
- Call `persist_custody_context` inside `BeaconChainBuilder::build` so that changes caused by CLI flags are persisted (otherwise starting a node with `--semi-supernode` and no validators, then shutting it down uncleanly would cause use to forget the CGC).
These changes greatly reduce the timespan during which an unclean shutdown can create inconsistency. In the worst case, we only lose backfill progress that runs concurrently with the `register_validators` handler (should be extremely minimal, nigh impossible).
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
The flow for local block building is
1. Create execution payload and bid
2. Construct beacon block
3. Sign beacon block and publish
4. Sign execution payload and publish
This PR adds the beacon block v4 flow , GET payload envelope and POST payload envelope (local block building only). The spec for these endpoints can be found here: https://github.com/ethereum/beacon-APIs/pull/552 and is subject to change.
We needed a way to store the unsigned execution payload envelope associated to the execution payload bid that was included in the block. I introduced a new cache that stores these unsigned execution payload envelopes. the GET payload envelope queries this cache directly so that a proposer, after publishing a block, can fetch the payload envelope + sign and publish it.
I kept payload signing and publishing within the validators block service to keep things simple for now. The idea was to build out a block production MVP for devnet 0, try not to affect any non gloas code paths and build things out in such a way that it will be easy to deprecate pre-gloas code paths later on (for example block production v2 and v3).
We will eventually need to track which beacon node was queried for the block so that we can later query it for the payload. But thats not needed for the devnet.
Co-Authored-By: Eitan Seri- Levi <eserilev@gmail.com>
Co-Authored-By: Michael Sproul <michael@sigmaprime.io>
Co-Authored-By: Jimmy Chen <jchen.tc@gmail.com>
Co-Authored-By: Eitan Seri-Levi <eserilev@ucsc.edu>
This reverts some of the changes from #8524 by adding back the typed network endpoints with an optional `network` feature. Without the `network` feature, these endpoints (and associated dependencies) will not be built.
This means the `enr`, `multiaddr` and `libp2p-identity` dependencies have returned but are now optional
Co-Authored-By: Mac L <mjladson@pm.me>
