* add light client optimistic and finality update rpc
* Arc the updates in the response
* add conditional advertisement for both LightClientOptimisticUpdate and LightClientFinalityUpdate
* alter display for inboundrequest light client optimistic and finality updates
* remove LightClientOptimistic/FinalityReuest struct and some minor fixes
* rebase
* failing rpc_test for LightClientBootstrap and beginning of MockLib2pLightClient
* minor change
* added MockRPCHandler by importing everything except OutboundRequest. Need to implement the ConnectionHandler trait now should be copy pastable
* almost there but ran into issue where needed to implement BaseOutboundRequest.
* failing but running with a light client service of sorts
* small test change
* changed Protocol::LightClientBootstrap response limit
* deleted some stuff from ConnectionHandler Implementation for the mock light client if you need to make something with multiple requests work maybe check here
* deleted purging expired inbound/outbound streams code
* deleted drive inbound streams that need to be processed
* removed unused imports
* made things private again
* deleted inject_fully_negotiated_inbound
* made more things private again
* more
* turned the logger off in the test
* added failing test for new rpc
* add rate limit for new rpcs
* change InboundUpgrade function to use new rpcs. fmt. add test for LightClientFinalityUpdate
* rebasing fix
* add LightClientUpdate to handle_rpc functions
* added context bytes
* fmt
* use correct unsed_tcp4_port function
* fix for recent config changes and adding context_bytes for the light client protocols
* fix clippy complaint
* Merge branch 'unstable' into lc-reqresp
# Conflicts:
# beacon_node/beacon_processor/src/lib.rs
# beacon_node/lighthouse_network/src/peer_manager/mod.rs
# beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
# beacon_node/lighthouse_network/src/rpc/config.rs
# beacon_node/lighthouse_network/src/rpc/methods.rs
# beacon_node/lighthouse_network/src/rpc/mod.rs
# beacon_node/lighthouse_network/src/rpc/outbound.rs
# beacon_node/lighthouse_network/src/rpc/protocol.rs
# beacon_node/lighthouse_network/src/rpc/rate_limiter.rs
# beacon_node/lighthouse_network/src/rpc/self_limiter.rs
# beacon_node/lighthouse_network/src/service/api_types.rs
# beacon_node/lighthouse_network/tests/common/mod.rs
# beacon_node/lighthouse_network/tests/rpc_tests.rs
# beacon_node/network/src/network_beacon_processor/rpc_methods.rs
# beacon_node/network/src/router.rs
* Error handling updates and various cleanups.
* Moar minor clean ups.
* Do not ban peer for rate limiting light client requests
* Merge branch 'unstable' into lc-reqresp. Also removed the mock light client tests to make it compile (See #4940).
# Conflicts:
# beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
# beacon_node/lighthouse_network/src/rpc/methods.rs
# beacon_node/lighthouse_network/src/rpc/mod.rs
# beacon_node/lighthouse_network/src/rpc/protocol.rs
# beacon_node/lighthouse_network/src/service/api_types.rs
# beacon_node/lighthouse_network/tests/common/mod.rs
# beacon_node/network/src/network_beacon_processor/rpc_methods.rs
# beacon_node/network/src/router.rs
# consensus/types/src/light_client_bootstrap.rs
# consensus/types/src/light_client_finality_update.rs
# consensus/types/src/light_client_optimistic_update.rs
* Remove unnecessary changes
* Add missing light client queue handling.
* Merge branch 'unstable' into lc-reqresp
* Merge branch 'unstable' into lc-reqresp
# Conflicts:
# beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
# beacon_node/lighthouse_network/src/service/api_types.rs
# consensus/types/src/light_client_finality_update.rs
# consensus/types/src/light_client_optimistic_update.rs
* Add context bytes for light client RPC responses.
* Add RPC limits for light client object.
* Fix lint
* Fix incorrect light client max size computation.
* Merge branch 'unstable' into lc-reqresp
# Conflicts:
# beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
# beacon_node/lighthouse_network/src/rpc/protocol.rs
# beacon_node/lighthouse_network/src/service/api_types.rs
* Remove unwanted local changes.
* Merge branch 'unstable' into lc-reqresp
* Replace `unimplemented` electra code path with deneb values.
## Issue Addressed
#4118
## Proposed Changes
This PR introduces a "progressive balances" cache on the `BeaconState`, which keeps track of the accumulated target attestation balance for the current & previous epochs. The cached values are utilised by fork choice to calculate unrealized justification and finalization (instead of converting epoch participation arrays to balances for each block we receive).
This optimization will be rolled out gradually to allow for more testing. A new `--progressive-balances disabled|checked|strict|fast` flag is introduced to support this:
- `checked`: enabled with checks against participation cache, and falls back to the existing epoch processing calculation if there is a total target attester balance mismatch. There is no performance gain from this as the participation cache still needs to be computed. **This is the default mode for now.**
- `strict`: enabled with checks against participation cache, returns error if there is a mismatch. **Used for testing only**.
- `fast`: enabled with no comparative checks and without computing the participation cache. This mode gives us the performance gains from the optimization. This is still experimental and not currently recommended for production usage, but will become the default mode in a future release.
- `disabled`: disable the usage of progressive cache, and use the existing method for FFG progression calculation. This mode may be useful if we find a bug and want to stop the frequent error logs.
### Tasks
- [x] Initial cache implementation in `BeaconState`
- [x] Perform checks in fork choice to compare the progressive balances cache against results from `ParticipationCache`
- [x] Add CLI flag, and disable the optimization by default
- [x] Testing on Goerli & Benchmarking
- [x] Move caching logic from state processing to the `ProgressiveBalancesCache` (see [this comment](https://github.com/sigp/lighthouse/pull/4362#discussion_r1230877001))
- [x] Add attesting balance metrics
Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
* Fix arbitrary check kintsugi
* Add merge chain spec fields, and a function to determine which constant to use based on the state variant
* increment spec test version
* Remove `Transaction` enum wrapper
* Remove Transaction new-type
* Remove gas validations
* Add `--terminal-block-hash-epoch-override` flag
* Increment spec tests version to 1.1.5
* Remove extraneous gossip verification https://github.com/ethereum/consensus-specs/pull/2687
* - Remove unused Error variants
- Require both "terminal-block-hash-epoch-override" and "terminal-block-hash-override" when either flag is used
* - Remove a couple more unused Error variants
Co-authored-by: Paul Hauner <paul@paulhauner.com>
## Proposed Changes
Implement the consensus changes necessary for the upcoming Altair hard fork.
## Additional Info
This is quite a heavy refactor, with pivotal types like the `BeaconState` and `BeaconBlock` changing from structs to enums. This ripples through the whole codebase with field accesses changing to methods, e.g. `state.slot` => `state.slot()`.
Co-authored-by: realbigsean <seananderson33@gmail.com>
