Commit Graph

622 Commits

Author SHA1 Message Date
Eitan Seri-Levi
df6e1c9add Fix BitVectors TestRandom implementation (#5854)
* fix bitvector test random impl
2024-05-30 14:09:22 +00:00
Akihito Nakano
6f05863007 Remove unused dependencies (#5812)
* Remove unused dependencies from lighthouse_network

* Remove unused dependencies from gossipsub

* Remove unused dependencies from network

* Remove unused dependencies from consensus/types

* Remove unused dependencies from account_manager

* Remove unused dependencies from boot_node

* Remove unused dependencies from database_manager

* Remove unused dependencies from validator_manager

* Remove unused dependencies from testing/ef_tests

* Remove unused dependencies from testing/simulator

* Remove unused dependencies from testing/execution_engine_integration

* Remove unused dependencies from common/system_health

* Remove unused dependencies from common/eth2

* Remove unused dependencies from common/lighthouse_metrics

* Remove unused dependencies from common/eth2_network_config

* Remove unused dependencies from common/logging

* Remove unused dependencies from watch

* Remove unused dependencies from slasher

* Remove unused dependencies from lighthouse

* Remove unused dependencies from lcli

* Remove unused dependencies from beacon_node

* Remove unused dependencies from beacon_node/builder_client

* Remove unused dependencies from beacon_node/beacon_processor

* Remove unused dependencies from beacon_node/execution_layer

* Remove unused dependencies from beacon_node/eth1

* Remove unused dependencies from beacon_node/client

* Remove unused dependencies from beacon_node/store

* Fix missing deps which is used only for testing
2024-05-27 07:59:07 +00:00
Eitan Seri-Levi
6636167503 Log block import source (#5738)
* the default target peers is 100

* add some comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into track-block-import-source

* add block import source

* revert

* update logging text

* fix tests

* lint

* use % instaed of to_string
2024-05-15 09:17:06 +00:00
Pawan Dhananjay
1af3f0f9d8 Make modified helpers in electra fork aware (#5698)
* Make modified helpers in electra fork aware

* Make more functions fork aware

* formatting fixes only.
2024-05-03 09:54:01 +00:00
Michael Sproul
ee974db0ba Add metric for current epoch total balance (#5688)
* Add metric for current epoch total balance
2024-05-02 04:50:30 +00:00
Pawan Dhananjay
8b24880df7 Add more electra helpers (#5653)
* Add new helpers

* Fix some stuff

* Fix compilation errors

* lint

* Address review
2024-04-27 01:09:29 +00:00
realbigsean
000a4fdf4d Electra other containers (#5652)
* add new fields to execution payload and header

* beacon state changes

* partial beacon state

* safe arith in upgrade to electra

* initialize balances cache in interop genesis state

* Revert "initialize balances cache in interop genesis state"

This reverts commit c60b522865.

* always initialize balances cache if necessary in electra upgrade

* build cache earlier

* fix block test

* per fork NUM_FIELDS_POW2

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-other-containers

* fix lints

* get fields based on post state, as is spec'd

* fix type and move cache build
2024-04-26 20:17:23 +00:00
Lion - dapplion
a1141ea1ef Deterministic block generation for tests (#5654)
* Deterministic block generation for tests
2024-04-26 13:25:45 +00:00
Mac L
13f94ef0f3 Rename Merge to Bellatrix (#5601)
* Rename Merge to Bellatrix

* Remove tree-hash-cache which got readded from the rebase
2024-04-25 20:19:41 +00:00
realbigsean
320345695d Add electra presets to beacon API (#5630)
* add presets to API

* add extra fields to config spec in beacon API

* remove unused

* add mainnet presets for gnosis and fix minimal preset default values
2024-04-25 18:33:03 +00:00
ethDreamer
4a48d7b546 Encode Execution Engine Client Version In Graffiti (#5290)
* Add `engine_clientVersionV1` structs

* Implement `engine_clientVersionV1`

* Update to latest spec changes

* Implement GraffitiCalculator Service

* Added Unit Tests for GraffitiCalculator

* Address Mac's Comments

* Remove need to use clap in beacon chain

* Merge remote-tracking branch 'upstream/unstable' into el_client_version_graffiti

* Merge branch 'unstable' into el_client_version_graffiti

# Conflicts:
#	beacon_node/beacon_chain/Cargo.toml
2024-04-24 06:02:48 +00:00
realbigsean
c4a2bcb9c7 Yaml rust2 (#5635)
* use yaml-rust2
2024-04-24 06:02:10 +00:00
Michael Sproul
61962898e2 In-memory tree states (#5533)
* Consensus changes

* EF tests

* lcli

* common and watch

* account manager

* cargo

* fork choice

* promise cache

* beacon chain

* interop genesis

* http api

* lighthouse

* op pool

* beacon chain misc

* parallel state cache

* store

* fix issues in store

* IT COMPILES

* Remove some unnecessary module qualification

* Revert Arced pubkey optimization (#5536)

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Fix caching, rebasing and some tests

* Remove unused deps

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Small cleanups

* Revert shuffling cache/promise cache changes

* Fix state advance bugs

* Fix shuffling tests

* Remove some resolved FIXMEs

* Remove StateProcessingStrategy

* Optimise withdrawals calculation

* Don't reorg if state cache is missed

* Remove inconsistent state func

* Fix beta compiler

* Rebase early, rebase often

* Fix state caching behaviour

* Update to milhouse release

* Fix on-disk consensus context format

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Squashed commit of the following:

commit 3a16649023
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Thu Apr 18 14:26:09 2024 +1000

    Fix on-disk consensus context format

* Keep indexed attestations, thanks Sean

* Merge branch 'on-disk-consensus-context' into tree-states-memory

* Merge branch 'unstable' into tree-states-memory

* Address half of Sean's review

* More simplifications from Sean's review

* Cache state after get_advanced_hot_state
2024-04-24 01:22:36 +00:00
ethDreamer
05fbbdd840 Electra: Add Preset, Constants, & Config (#5606)
* Electra: Add Presets, Constants, & Config
2024-04-23 14:46:49 +00:00
ethDreamer
82b131d37f Electra: Add New Containers (#5607)
* Electra: Add New Containers
2024-04-23 00:21:21 +00:00
João Oliveira
9b5895ca89 Fix cargo audit RUSTSEC-2024-0336 (#5612)
* replace unmaintained rust_yaml with serde_yml

* update warp
2024-04-22 13:01:06 +00:00
Michael Sproul
5a9e973f04 Fix on-disk consensus context format (#5598)
* Fix on-disk consensus context format

* Keep indexed attestations, thanks Sean
2024-04-19 08:18:30 +00:00
ethDreamer
cda926ce1b Rename Functions to More Closely Match Spec (#5591)
* Rename Functions to More Closely Match Spec
2024-04-16 19:03:08 +00:00
Lion - dapplion
ced653873e Impl Ord on ForkName for ChainSpec usage (#5531)
* Impl Ord on ForkName for ChainSpec usage

* add fork ord consistent test
2024-04-10 09:02:23 +00:00
realbigsean
d527d124dd Beta compiler fixes (#5543)
* remove duplicate imports in gossip tests. get rid of duplicate trait bound locations

* more double import fixes

* cargo fmt
2024-04-10 07:47:05 +00:00
GeemoCandama
32be063f0f Support LightClientFinalityUpdate and LightClientOptimisticUpdate rpcs (#3849)
* add light client optimistic and finality update rpc

* Arc the updates in the response

* add conditional advertisement for both LightClientOptimisticUpdate and LightClientFinalityUpdate

* alter display for inboundrequest light client optimistic and finality updates

* remove LightClientOptimistic/FinalityReuest struct and some minor fixes

* rebase

* failing rpc_test for LightClientBootstrap and beginning of MockLib2pLightClient

* minor change

* added MockRPCHandler by importing everything except OutboundRequest. Need to implement the ConnectionHandler trait now should be copy pastable

* almost there but ran into issue where needed to implement BaseOutboundRequest.

* failing but running with a light client service of sorts

* small test change

* changed Protocol::LightClientBootstrap response limit

* deleted some stuff from ConnectionHandler Implementation for the mock light client if you need to make something with multiple requests work maybe check here

* deleted purging expired inbound/outbound streams code

* deleted drive inbound streams that need to be processed

* removed unused imports

* made things private again

* deleted inject_fully_negotiated_inbound

* made more things private again

* more

* turned the logger off in the test

* added failing test for new rpc

* add rate limit for new rpcs

* change InboundUpgrade function to use new rpcs. fmt. add test for LightClientFinalityUpdate

* rebasing fix

* add LightClientUpdate to handle_rpc functions

* added context bytes

* fmt

* use correct unsed_tcp4_port function

* fix for recent config changes and adding context_bytes for the light client protocols

* fix clippy complaint

* Merge branch 'unstable' into lc-reqresp

# Conflicts:
#	beacon_node/beacon_processor/src/lib.rs
#	beacon_node/lighthouse_network/src/peer_manager/mod.rs
#	beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
#	beacon_node/lighthouse_network/src/rpc/config.rs
#	beacon_node/lighthouse_network/src/rpc/methods.rs
#	beacon_node/lighthouse_network/src/rpc/mod.rs
#	beacon_node/lighthouse_network/src/rpc/outbound.rs
#	beacon_node/lighthouse_network/src/rpc/protocol.rs
#	beacon_node/lighthouse_network/src/rpc/rate_limiter.rs
#	beacon_node/lighthouse_network/src/rpc/self_limiter.rs
#	beacon_node/lighthouse_network/src/service/api_types.rs
#	beacon_node/lighthouse_network/tests/common/mod.rs
#	beacon_node/lighthouse_network/tests/rpc_tests.rs
#	beacon_node/network/src/network_beacon_processor/rpc_methods.rs
#	beacon_node/network/src/router.rs

* Error handling updates and various cleanups.

* Moar minor clean ups.

* Do not ban peer for rate limiting light client requests

* Merge branch 'unstable' into lc-reqresp. Also removed the mock light client tests to make it compile (See #4940).

# Conflicts:
#	beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
#	beacon_node/lighthouse_network/src/rpc/methods.rs
#	beacon_node/lighthouse_network/src/rpc/mod.rs
#	beacon_node/lighthouse_network/src/rpc/protocol.rs
#	beacon_node/lighthouse_network/src/service/api_types.rs
#	beacon_node/lighthouse_network/tests/common/mod.rs
#	beacon_node/network/src/network_beacon_processor/rpc_methods.rs
#	beacon_node/network/src/router.rs
#	consensus/types/src/light_client_bootstrap.rs
#	consensus/types/src/light_client_finality_update.rs
#	consensus/types/src/light_client_optimistic_update.rs

* Remove unnecessary changes

* Add missing light client queue handling.

* Merge branch 'unstable' into lc-reqresp

* Merge branch 'unstable' into lc-reqresp

# Conflicts:
#	beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
#	beacon_node/lighthouse_network/src/service/api_types.rs
#	consensus/types/src/light_client_finality_update.rs
#	consensus/types/src/light_client_optimistic_update.rs

* Add context bytes for light client RPC responses.

* Add RPC limits for light client object.

* Fix lint

* Fix incorrect light client max size computation.

* Merge branch 'unstable' into lc-reqresp

# Conflicts:
#	beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
#	beacon_node/lighthouse_network/src/rpc/protocol.rs
#	beacon_node/lighthouse_network/src/service/api_types.rs

* Remove unwanted local changes.

* Merge branch 'unstable' into lc-reqresp

* Replace `unimplemented` electra code path with deneb values.
2024-04-09 07:23:39 +00:00
Michael Sproul
06eb181b42 Delete ParticipationCache (#5525)
* Delete `ParticipationCache`
2024-04-05 03:15:42 +00:00
Eitan Seri-Levi
ee69e14db9 Add is_parent_strong proposer re-org check (#5417)
* initial fork choice additions

* add helper fns

* add is_parent_strong

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into add_is_parent_strong_check

* disabling proposer reorg should set parent_threshold to u64 max

* add new flag, is_parent_strong check in override fcu params

* cherry-pick changes

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into add_is_parent_strong_check

* cleanup

* fmt

* Minor review tweaks
2024-04-04 19:38:06 +00:00
Michael Sproul
feb531f85b Single-pass epoch processing and optimised block processing (#5279)
* Single-pass epoch processing (#4483, #4573)

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Delete unused epoch processing code (#5170)

* Delete unused epoch processing code

* Compare total deltas

* Remove unnecessary apply_pending

* cargo fmt

* Remove newline

* Use epoch cache in block packing (#5223)

* Remove progressive balances mode (#5224)

* inline inactivity_penalty_quotient_for_state

* drop previous_epoch_total_active_balance

* fc lint

* spec compliant process_sync_aggregate (#15)

* spec compliant process_sync_aggregate

* Update consensus/state_processing/src/per_block_processing/altair/sync_committee.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Delete the participation cache (#16)

* update help

* Fix op_pool tests

* Fix fork choice tests

* Merge remote-tracking branch 'sigp/unstable' into epoch-single-pass

* Simplify exit cache (#5280)

* Fix clippy on exit cache

* Clean up single-pass a bit (#5282)

* Address Mark's review of single-pass (#5386)

* Merge remote-tracking branch 'origin/unstable' into epoch-single-pass

* Address Sean's review comments (#5414)

* Address most of Sean's review comments

* Simplify total balance cache building

* Clean up unused junk

* Merge remote-tracking branch 'origin/unstable' into epoch-single-pass

* More self-review

* Merge remote-tracking branch 'origin/unstable' into epoch-single-pass

* Merge branch 'unstable' into epoch-single-pass

* Fix imports for beta compiler

* Fix tests, probably
2024-04-04 13:14:36 +00:00
Mac L
969d12dc6f Use E for EthSpec globally (#5264)
* Use `E` for `EthSpec` globally

* Fix tests

* Merge branch 'unstable' into e-ethspec

* Merge branch 'unstable' into e-ethspec

# Conflicts:
#	beacon_node/execution_layer/src/engine_api.rs
#	beacon_node/execution_layer/src/engine_api/http.rs
#	beacon_node/execution_layer/src/engine_api/json_structures.rs
#	beacon_node/execution_layer/src/test_utils/handle_rpc.rs
#	beacon_node/store/src/partial_beacon_state.rs
#	consensus/types/src/beacon_block.rs
#	consensus/types/src/beacon_block_body.rs
#	consensus/types/src/beacon_state.rs
#	consensus/types/src/config_and_preset.rs
#	consensus/types/src/execution_payload.rs
#	consensus/types/src/execution_payload_header.rs
#	consensus/types/src/light_client_optimistic_update.rs
#	consensus/types/src/payload.rs
#	lcli/src/parse_ssz.rs
2024-04-02 15:12:25 +00:00
Mac L
f8fdb71f50 Add Electra fork boilerplate (#5122)
* Add Electra fork boilerplate

* Remove electra from spec tests

* Fix tests

* Remove sneaky log file

* Fix more tests

* Fix even more tests and add suggestions

* Remove unrelated lcli addition

* Update more tests

* Merge branch 'unstable' into electra

* Add comment for test-suite lcli override

* Merge branch 'unstable' into electra

* Cleanup

* Merge branch 'unstable' into electra

* Apply suggestions

* Merge branch 'unstable' into electra

* Merge sigp/unstable into electra

* Merge branch 'unstable' into electra
2024-04-02 12:35:02 +00:00
Eitan Seri-Levi
e4d4e439cb Add Capella & Deneb light client support (#4946)
* rebase and add comment

* conditional test

* test

* optimistic chould be working now

* finality should be working now

* try again

* try again

* clippy fix

* add lc bootstrap beacon api

* add lc optimistic/finality update to events

* fmt

* That error isn't occuring on my computer but I think this should fix it

* Merge branch 'unstable' into light_client_beacon_api_1

# Conflicts:
#	beacon_node/beacon_chain/src/events.rs
#	beacon_node/http_api/src/lib.rs
#	beacon_node/http_api/src/test_utils.rs
#	beacon_node/http_api/tests/main.rs
#	beacon_node/lighthouse_network/src/rpc/codec/ssz_snappy.rs
#	beacon_node/lighthouse_network/src/rpc/methods.rs
#	beacon_node/lighthouse_network/src/service/api_types.rs
#	beacon_node/network/src/beacon_processor/worker/rpc_methods.rs
#	beacon_node/tests/test.rs
#	common/eth2/src/types.rs
#	lighthouse/src/main.rs

* Add missing test file

* Update light client types to comply with Altair light client spec.

* Fix test compilation

* Merge branch 'unstable' into light_client_beacon_api_1

* Support deserializing light client structures for the Bellatrix fork

* Move `get_light_client_bootstrap` logic to `BeaconChain`. `LightClientBootstrap` API to return `ForkVersionedResponse`.

* Misc fixes.
- log cleanup
- move http_api config mutation to `config::get_config` for consistency
- fix light client API responses

* Add light client bootstrap API test and fix existing ones.

* Merge branch 'unstable' into light_client_beacon_api_1

* Fix test for `light-client-server` http api config.

* Appease clippy

* Add Altair light client SSZ tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into light_client_beacon_api_1

* updates to light client header

* light client header from signed beacon block

* using options

* implement helper functions

* placeholder conversion from vec hash256 to exec branch

* add deneb

* using fixed vector

* remove unwraps

* by epoch

* compute merkle proof

* merkle proof

* update comments

* resolve merge conflicts

* linting

* Merge branch 'unstable' into light-client-ssz-tests

# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	consensus/types/src/light_client_bootstrap.rs
#	consensus/types/src/light_client_header.rs

* superstruct attempt

* superstruct changes

* lint

* altair

* update

* update

* changes to light_client_optimistic_ and finality

* merge unstable

* refactor

* resolved merge conflicts

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* block_to_light_client_header fork aware

* fmt

* comment fix

* comment fix

* include merge fork, update deserialize_by_fork, refactor

* fmt

* pass by ref to prevent clone

* rename merkle proof fn

* add FIXME

* LightClientHeader TestRandom

* fix comments

* fork version deserialize

* merge unstable

* move fn arguments, fork name calc

* use task executor

* remove unneeded fns

* remove dead code

* add manual ssz decoding/encoding and add ssz_tests_by_fork macro

* merge deneb types with tests

* merge ssz tests, revert code deletion, cleanup

* move chainspec

* update ssz tests

* fmt

* light client ssz tests

* change to superstruct

* changes from feedback

* linting

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* test fix

* cleanup

* Remove unused `derive`.

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into capella_deneb_light_client_types

* beta compiler fix

* merge
2024-03-25 11:01:56 +00:00
dknopik
0a6e4a11d7 Verify whether validators really are unknown during sync committee duty API request (#5174)
* Verify whether validators really are unknown during sync committee duty API request

* Merge branch 'unstable' into fix-4717

* Merge branch 'unstable' into fix-4717

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into fix-4717
2024-03-23 00:01:11 +00:00
Michael Sproul
65a6118c53 Fix gossip verification of duplicate attester slashings (#5385)
* Fix gossip verification of duplicate attester slashings
2024-03-20 20:47:38 +00:00
Eitan Seri-Levi
01ec42e75a Fix Rust beta compiler errors 1.78.0-beta.1 (#5439)
* remove redundant imports

* fix test

* contains key

* fmt

* Merge branch 'unstable' into fix-beta-compiler
2024-03-20 05:17:02 +00:00
Pawan Dhananjay
84a902a589 Reduce load on validator subscription channels (#5311)
* Fix tests

* Merge branch 'unstable' into unclog-channels

* Avoid reallocations

* Reduce subscription load on beacon node
2024-03-07 12:32:27 +00:00
chonghe
258eeb5f09 Delete milagro library (#5298)
* fix lib.rs and tests.rs

* update decode.rs

* auto-delete in Cargo.lock

* delete milagro in cargo.toml

* remove milagro from makefile

* remove milagro from the name

* delete milagro in comment

* delete milagro in cargo.toml

* delete in /testing/ef_tests/cargo.toml

* delete milagro in the logical OR

* delete milagro in /lighthouse/src/main.rs

* delete milagro in /crypto/bls/tests/tests.rs

* delete milagro in comment

* delete milagro in /testing//ef_test/src//cases/bls_eth_aggregate_pubkeys.rs

* delete milagro

* delete more in lib.rs

* delete more in lib.rs

* delete more in lib.rs

* delete milagro in /crypto/bls/src/lib.rs

* delete milagro in crypto/bls/src/mod.rs

* delete milagro.rs
2024-03-06 23:17:42 +00:00
Michael Sproul
c9702cb0a1 Download checkpoint blobs during checkpoint sync (#5252)
* MVP implementation (untested)

* update store checkpoint sync test

* update cli help

* Merge pull request #5253 from realbigsean/checkpoint-blobs-sean

Checkpoint blobs sean

* Warn only if blobs are missing from server

* Merge remote-tracking branch 'origin/unstable' into checkpoint-blobs

* Verify checkpoint blobs

* Move blob verification earlier
2024-02-19 02:22:23 +00:00
Paul Hauner
e22c9eed8f Add Deneb fork epoch for Gnosis (#5242)
* Add Deneb fork epoch for Gnosis

* Add deneb constants

* Update common/eth2_network_config/built_in_network_configs/gnosis/config.yaml

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* Adjust `min_epochs_for_block_requests`

* Change `MAX_PER_EPOCH_ACTIVATION_CHURN_LIMIT`

* Fix chain spec `max_per_epoch_activation_churn_limit`

* Fix chain spec values again
2024-02-19 02:22:19 +00:00
Michael Sproul
0e819fa785 Schedule Deneb on mainnet (#5233)
* Schedule Deneb on mainnet

* Fix trusted setup roundtrip test

* Fix BN CLI tests for insecure genesis sync
2024-02-15 12:23:51 +00:00
realbigsean
4172d9f75c Update to consensus spec v1.4.0-beta.6 (#5094)
* get latest ef tests passing

* fix tests

* Fix invalid payload recovery tests

* Merge branch 'unstable' into update-to-spec-v1.4.0-beta.6

* Revert "fix tests"

This reverts commit 0c875b02e0.

* Fix fork choice def. tests

* Update beacon_node/beacon_chain/tests/payload_invalidation.rs
2024-02-08 18:08:21 +00:00
Jimmy Chen
8fa11aa792 Fix incorrect value set for blobs_by_root_request rpc limit. (#5181)
* Fix incorrect value set for `blobs_by_root_request` rpc limit.
2024-02-05 18:37:28 +00:00
Jimmy Chen
39e9f7dc6b Fix Rust beta compiler errors (1.77) (#5180)
* Lint fixes

* More fixes for beta compiler.

* Format fixes

* Move `#[allow(dead_code)]` to field level.

* Remove old comment.

* Update beacon_node/execution_layer/src/test_utils/mod.rs

Co-authored-by: João Oliveira <hello@jxs.pt>

* remove duplicate line
2024-02-05 17:54:11 +00:00
Divma
8353ec9785 Update deterministic subnets upgrade to allow prefix computation (#4959)
* add new spec field to spec

* add spec changes

* fix bug and update expected subnets with validation from pyspec

* fix values and make test prettier (?)

* fix style

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
2024-01-31 09:30:26 +00:00
Lion - dapplion
b035638f9b Compute recent lightclient updates (#4969)
* Compute recent lightclient updates

* Review PR

* Merge remote-tracking branch 'upstream/unstable' into lc-prod-recent-updates

* Review PR

* consistent naming

* add metrics

* revert dropping reprocessing queue

* Update light client optimistic update re-processing logic. (#7)

* Add light client server simulator tests. Co-authored by @dapplion.

* Merge branch 'unstable' into fork/dapplion/lc-prod-recent-updates

* Fix lint

* Enable light client server in simulator test.

* Fix test for light client optimistic updates and finality updates.
2024-01-31 05:25:51 +00:00
Michael Sproul
a403138ed0 Reduce size of futures in HTTP API to prevent stack overflows (#5104)
* Box::pin a few big futures

* Arc the blocks early in publication

* Fix more tests
2024-01-23 15:32:07 +11:00
Michael Sproul
185646acb2 Fix PublishBlockRequest SSZ decoding (#5078)
* Fix PublishBlockRequest SSZ decoding

* Send header for block v1 ssz client

* Delete unused function
2024-01-19 12:56:30 +11:00
Mac L
a0b407c15d Add Deneb readiness logging (#5074) 2024-01-18 15:21:38 -05:00
realbigsean
b47e3f252e Runtime rpc request sizes (#4841)
* add runtime variable list type

* add configs to ChainSpec

* git rid of max request blocks type

* fix tests and lints

* remove todos

* git rid of old const usage

* fix decode impl

* add new config to `Config` api struct

* add docs fix compilt

* move methods for per-fork-spec to chainspec

* get values off chain spec

* fix compile

* remove min by root size

* add tests for runtime var list

---------

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2024-01-09 10:23:47 +11:00
Michael Sproul
af11e78ae1 Clean up blockv3 metadata and client (#5015)
* Improve block production v3 client

* Delete wayward line

* Overhaul JSON endpoint as well

* Rename timeout param

* Update tests

* I broke everything

* Ah this is an insane fix

* Remove unnecessary optionals

* Doc fix
2023-12-22 09:39:17 -05:00
Jimmy Chen
b0c374c1ca Update dependencies to get rid of the yanked deps (#4994)
* Initial attempt to upgrade hashbrown to latest to get rid of the crate warnings.

* Replace `.expect()` usage with use of `const`.

* Update ahash 0.7 as well

* Remove unsafe code.

* Update `lru` to 0.12 and fix release test errors.

* Set non-blocking socket

* Bump testcontainers to 0.15.

* Fix lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2023-12-15 18:31:59 +11:00
Pawan Dhananjay
31044402ee Sidecar inclusion proof (#4900)
* Refactor BlobSidecar to new type

* Fix some compile errors

* Gossip verification compiles

* Fix http api types take 1

* Fix another round of compile errors

* Beacon node crate compiles

* EF tests compile

* Remove all blob signing from VC

* fmt

* Tests compile

* Fix some tests

* Fix more http tests

* get compiling

* Fix gossip conditions and tests

* Add basic proof generation and verification

* remove unnecessary ssz decode

* add back build_sidecar

* remove default at fork for blobs

* fix beacon chain tests

* get relase tests compiling

* fix lints

* fix existing spec tests

* add new ef tests

* fix gossip duplicate rule

* lints

* add back sidecar signature check in gossip

* add finalized descendant check to blob sidecar gossip

* fix error conversion

* fix release tests

* sidecar inclusion self review cleanup

* Add proof verification and computation metrics

* Remove accidentally committed file

* Unify some block and blob errors; add slashing conditions for sidecars

* Address review comment

* Clean up re-org tests (#4957)

* Address more review comments

* Add Comments & Eliminate Unnecessary Clones

* update names

* Update beacon_node/beacon_chain/src/metrics.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Update beacon_node/network/src/network_beacon_processor/tests.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* pr feedback

* fix test compile

* Sidecar Inclusion proof small refactor and updates (#4967)

* Update some comments, variables and small cosmetic fixes.

* Couple blobs and proofs into a tuple in `PayloadAndBlobs` for simplicity and safety.

* Update function comment.

* Update testing/ef_tests/src/cases/merkle_proof_validity.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Rename the block and blob wrapper types used in the beacon API interfaces.

* make sure gossip invalid blobs are passed to the slasher (#4970)

* Add blob headers to slasher before adding to DA checker

* Replace Vec with HashSet in BlockQueue

* fmt

* Rename gindex -> index

* Simplify gossip condition

---------

Co-authored-by: realbigsean <seananderson33@gmail.com>
Co-authored-by: realbigsean <sean@sigmaprime.io>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: Mark Mackey <mark@sigmaprime.io>
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-12-05 11:19:59 -05:00
GeemoCandama
8a599ec7dc API for LightClientBootstrap, LightClientFinalityUpdate, LightClientOptimisticUpdate and light client events (#3954)
* rebase and add comment

* conditional test

* test

* optimistic chould be working now

* finality should be working now

* try again

* try again

* clippy fix

* add lc bootstrap beacon api

* add lc optimistic/finality update to events

* fmt

* That error isn't occuring on my computer but I think this should fix it

* Add missing test file

* Update light client types to comply with Altair light client spec.

* Fix test compilation

* Support deserializing light client structures for the Bellatrix fork

* Move `get_light_client_bootstrap` logic to `BeaconChain`. `LightClientBootstrap` API to return `ForkVersionedResponse`.

* Misc fixes.
- log cleanup
- move http_api config mutation to `config::get_config` for consistency
- fix light client API responses

* Add light client bootstrap API test and fix existing ones.

* Fix test for `light-client-server` http api config.

* Appease clippy

* Efficiency improvement when retrieving beacon state.

---------

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2023-11-28 17:14:29 +11:00
Jimmy Chen
6b63d18420 Fix Rust beta compiler warnings (rustc 1.75.0-beta.1 (782883f60 2023-11-12)) (#4932) 2023-11-18 03:55:11 +11:00
Joel Rousseau
ac8811afac Add missed blocks to monitored validators (#4731)
* add missed_block metric

* init missed_block in constructor

* declare beaconproposercache in ValidatorMonitor

* refacto proposer_shuffling_decision_root to use epoch instead of current.epoch

* imple new proposer_shuffling_decision_root in callers

* push missed_blocks

* prune missed_blocks

* only add to hashmap if it's a monitored validator

* remove current_epoch dup + typos

* extract in func

* add prom metrics

* checkpoint is not only epoch but slot as well

* add safeguard if we start a new chain at slot 0

* clean

* remove unnecessary negative value for a slot

* typo in comment

* remove unused current_epoch

* share beacon_proposer_cache between validator_monitor and beacon_chain

* pass Hash256::zero()

* debug objects

* fix loop: lag is at the head

* sed s/get_slot/get_epoch

* fewer calls to cache.get_epoch

* fix typos

* remove cache first call

* export TYPICAL_SLOTS_PER_EPOCH and use it in validator_monitor

* switch to gauge & loop over missed_blocks hashset

* fix subnet_service tests

* remove unused var

* clean + fix nits

* add beacon_proposer_cache + validator_monitor in builder

* fix store_tests

* fix builder tests

* add tests

* add validator monitor set of tests

* clean tests

* nits

* optimise imports

* lint

* typo

* added self.aggregatable

* duplicate proposer_shuffling_decision_root

* remove duplication in passing beacon_proposer_cache

* remove duplication in passing beacon_proposer_cache

* using indices

* fmt

* implement missed blocks total

* nits

* avoid heap allocation

* remove recursion limit

* fix lint

* Fix valdiator monitor builder pattern

Unify validator monitor config struct

* renaming metrics

* renaming metrics in validator monitor

* add log if there's a missing validator index

* consistent log

* fix loop

* better loop

* move gauge to counter

* fmt

* add error message

* lint

* fix prom metrics

* set gauge to 0 when non-finalized epochs

* better wording

* remove hash256::zero in favour of block_root

* fix gauge total label

* fix last missed block validator

* Add `MissedBlock` struct

* Fix comment

* Refactor non-finalized block loop

* Fix off-by-one

* Avoid string allocation

* Fix compile error

* Remove non-finalized blocks metric

* fix func clojure

* remove unused variable

* remove unused DEFAULT_INDIVIDUAL_TRACKING_THRESHOLD

* remove unused DEFAULT_INDIVIDUAL_TRACKING_THRESHOLD in builder

* add validator index depending on the fork name

* typos

---------

Co-authored-by: Paul Hauner <paul@paulhauner.com>
2023-11-09 15:05:14 +11:00