Commit Graph

6489 Commits

Author SHA1 Message Date
hopinheimer
da290e8e2e Added required --force-bls-withdrawal-credentials description to --disable-deposits usage (#6436)
* cli description

* complied docs changes

* reverted changes and script amended

* fix

* reverting unwanted changes

* making linter happy

* requested changes

* Merge branch 'unstable' into cli-fix

* Merge branch 'unstable' into cli-fix
2024-10-10 11:32:41 +00:00
Lion - dapplion
244a460e70 Bound min size of dynamic processor queues (#6466)
* Bound min size of dynamic processor queues

* Use max

* Add test
2024-10-10 02:34:41 +00:00
Eitan Seri-Levi
352a9cf054 Add lockbud task to CI (#6470)
* Add lockbud task to CI

* Allow unknown lint

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into lockbud

* remove potential deadlock

* ignore tokio util crate

* Update image
2024-10-09 23:11:24 +00:00
Lion - dapplion
71c5388461 Transition block lookup sync to range sync (#6122)
* Transition block lookup sync to range sync

* Log unexpected state

* Merge remote-tracking branch 'sigp/unstable' into lookup-to-range

* Add docs

* Merge remote-tracking branch 'sigp/unstable' into lookup-to-range
2024-10-08 21:18:41 +00:00
Pawan Dhananjay
48dd3f385c Filter out BlsToExecutionChange messages for 0x02 validators (#6464)
* Filter out 0x02 validators from `get_bls_to_execution_changes`

* Prune bls to execution changes that have a 0x02 credential

* lint
2024-10-07 16:35:38 +00:00
Michael Sproul
1bd8f31545 Clean up temporary state flags while running (#6422)
* Clean up temporary state flags while running

* Add regression test

* Simplify
2024-10-07 09:41:52 +00:00
Akihito Nakano
8cf686f5c1 Add test for ActiveSamplingRequest (#6307)
* Add test for ActiveSamplingRequest

* Fix the column_indexes field from the requested ones to the responded ones

* Fix clippy errors

* Move tests to tests.rs

* Fix unused import

* Fix clippy error

* Merge branch 'unstable' into fork/add-test-for-active-sampling-request

# Conflicts:
#	beacon_node/network/Cargo.toml
#	beacon_node/network/src/sync/sampling.rs

* Merge branch 'unstable' into fork/add-test-for-active-sampling-request
2024-10-04 03:00:32 +00:00
Jimmy Chen
f3a5e256da Implement Subnet Sampling for PeerDAS (#6410)
* Add `SAMPLES_PER_SLOT` config.

* Rename `sampling` module to `peer_sampling`

* Implement subnet sampling.

* Update lookup test.

* Merge branch 'unstable' into subnet-sampling

* Merge branch 'unstable' into subnet-sampling

# Conflicts:
#	beacon_node/beacon_chain/src/data_availability_checker.rs
#	beacon_node/http_api/src/publish_blocks.rs
#	beacon_node/lighthouse_network/src/types/globals.rs
#	beacon_node/network/src/sync/manager.rs

* Merge branch 'unstable' into subnet-sampling
2024-10-04 00:27:30 +00:00
Akihito Nakano
a4a673b780 Output network-test logs into files in CI (#6355)
* Add ci_logger

* Update artifact name

* Add env var

* Add fork_name

* Fix clippy error

* Add comments
2024-10-03 13:53:36 +00:00
Mac L
f870b66f49 Rework Validator Client fallback mechanism (#4393)
* Rework Validator Client fallback mechanism

* Add CI workflow for fallback simulator

* Tie-break with sync distance for non-synced nodes

* Fix simulator

* Cleanup unused code

* More improvements

* Add IsOptimistic enum for readability

* Use configurable sync distance tiers

* Fix tests

* Combine status and health and improve logging

* Fix nodes not being marked as available

* Fix simulator

* Fix tests again

* Increase fallback simulator tolerance

* Add http api endpoint

* Fix todos and tests

* Update simulator

* Merge branch 'unstable' into vc-fallback

* Add suggestions

* Add id to ui endpoint

* Remove unnecessary clones

* Formatting

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Fix flag tests

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Fix conflicts

* Merge branch 'unstable' into vc-fallback

* Remove unnecessary pubs

* Simplify `compute_distance_tier` and reduce notifier awaits

* Use the more descriptive `user_index` instead of `id`

* Combine sync distance tolerance flags into one

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* wip

* Use new simulator from unstable

* Fix cli text

* Remove leftover files

* Remove old commented code

* Merge branch 'unstable' into vc-fallback

* Update cli text

* Silence candidate errors when pre-genesis

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Retry on failure

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Remove disable_run_on_all

* Remove unused error variant

* Fix out of date comment

* Merge branch 'unstable' into vc-fallback

* Remove unnecessary as_u64

* Remove more out of date comments

* Use tokio RwLock and remove parking_lot

* Merge branch 'unstable' into vc-fallback

* Formatting

* Ensure nodes are still added to total when not available

* Allow VC to detect when BN comes online

* Fix ui endpoint

* Don't have block_service as an Option

* Merge branch 'unstable' into vc-fallback

* Clean up lifetimes and futures

* Revert "Don't have block_service as an Option"

This reverts commit b5445a09e9.

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Improve rwlock sanitation using clones

* Merge branch 'unstable' into vc-fallback

* Drop read lock immediately by cloning the vec.
2024-10-03 05:57:12 +00:00
Jimmy Chen
17849b58ec Fix invalid data column sidecars getting accepted (#6454)
* Fix invalid data column sidecars getting accepted.

* Update code to match spec function.
2024-10-03 03:06:05 +00:00
Lion - dapplion
428310c881 Fit sampling log statements to fmt width (#6433)
* Fit sampling log statements to fmt width
2024-10-03 03:06:02 +00:00
Eitan Seri-Levi
f6d46fd6e9 Fix rolling file appender bug (#6266)
* ensure file path passed to the rolling file appender only contains directories

* handle edge case

* handle edge case

* fix based on feedback

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into rolling-file-apender-bug

* fmt

* linting

* ensure only bn inits tracing logfile

* ensure only bn inits tracing logfile

* Merge branch 'unstable' into rolling-file-apender-bug

* Get the metadata of `tracing_log_path` instead of `p`, which is a part of the path

* Merge pull request #11 from ackintosh/rolling-file-apender-bug-ackintosh

Get the metadata of `tracing_log_path` instead of `p`

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into rolling-file-apender-bug

* fmt
2024-10-03 02:00:56 +00:00
Eitan Seri-Levi
82faf975b3 Add {fork_name}_enabled functions (#5951)
* add fork_name_enabled fn to Forkname impl

* refactor codebase to use new fork_enabled fn

* fmt

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into fork-ord-impl

* small code cleanup

* resolve merge conflicts

* fix beacon chain test

* merge conflicts

* fix ef test issue

* resolve merge conflicts
2024-10-03 02:00:52 +00:00
Michael Sproul
dd08ebb2b0 Prevent Kurtosis container pollution (#6441)
* Prevent Kurtosis container pollution
2024-10-01 02:59:03 +00:00
João Oliveira
4a62b2418c estimate the total inbound bandwidth of IDONTWANT messages in bytes (#6438)
* estimate the total inbound bandwidth of IDONTWANT messages
2024-10-01 02:13:51 +00:00
João Oliveira
82098e1ef7 add a unique integer id to Rpc requests (#6444)
* add id to rpc requests

* rename rpc request and response types for more accurate meaning

* remove unrequired build_request function

* remove unirequired Request wrapper types and unify Outbound and Inbound Request

* add RequestId to NetworkMessage::SendResponse

,NetworkMessage::SendErrorResponse to be passed to Rpc::send_response
2024-10-01 01:36:17 +00:00
João Oliveira
5d1ff7c6f8 fix Rpc Ping sequence number (#6408)
* fix Rpc Ping sequence number

* bubble up Outbound Err's and Responses even if the peer disconnected

* send pings via Rpc from main network

* add comment to connected check

* Merge branch 'unstable' into fix-ping-seq-number
2024-09-26 14:52:03 +00:00
João Oliveira
50d8375d46 Remove Score Ord, PartialOrd, Eq and PartialEq impls (#6420)
* drop score Ord, PartialOrd, Eq and PartialEq impls

and impl total_cmp instead

* Revert "Fix test failure on Rust v1.81 (#6407)"

This reverts commit 8a085fc828.

* reverse in the compare function

* lint mdfiles
2024-09-25 13:45:35 +00:00
Michael Sproul
2792705331 Lenient duplicate checks on HTTP API for block publication (#5574)
* start splitting gossip verification

* WIP

* Gossip verify separate (#7)

* save

* save

* make ProvenancedBlock concrete

* delete into gossip verified block contents

* get rid of IntoBlobSidecar trait

* remove IntoGossipVerified trait

* get tests compiling

* don't check sidecar slashability in publish

* remove second publish closure

* drop blob bool. also prefer using message index over index of position in list

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Fix low-hanging tests

* Fix tests and clean up

* Clean up imports

* more cleanup

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Further refine behaviour and add tests

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Remove empty line

* Fix test (block is not fully imported just gossip verified)

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Update for unstable & use empty blob list

* Update comment

* Add test for duplicate block case

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Clarify unreachable case

* Fix another publish_block case

* Remove unreachable case in filter chain segment

* Revert unrelated blob optimisation

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Fix merge conflicts

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Fix some compilation issues. Impl is fucked though

* Support peerDAS

* Fix tests

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Fix conflict

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Address review comments

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate
2024-09-24 04:52:44 +00:00
Michael Sproul
1447eeb40b Improve single-node testnet support and Arc NetworkConfig/ChainSpec (#6396)
* Arc ChainSpec and NetworkConfig

* Fix release tests

* Fix lint

* Merge remote-tracking branch 'origin/unstable' into single-node-testnet
2024-09-24 00:16:18 +00:00
Lion - dapplion
d84df5799c Attribute invalid column proof error to correct peer (#6377)
* Attribute invalid column proof error to correct peer

* Update beacon_node/beacon_chain/src/data_availability_checker.rs

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>

* fix conflicts
2024-09-23 18:49:26 +00:00
Lion - dapplion
012e7e7bfa Allow custody by root requests to have no peers (#6417)
* Allow custody by root requests to have no peers
2024-09-23 18:49:23 +00:00
Eitan Seri-Levi
b619f1ab5c Make BeaconChain::kzg field mandatory (#6267)
* make kzg field required

* update todo

* always load trusted setup WIP

* fmt

* use new rust_eth_kzg version

* merge conlficts

* add kzg fn with trusted setup disabled

* as_slice

* add kzg with no precomp

* ignore udep for kzg

* refactor kzg init

* fix peerdas kzg schedule

* fix

* udeps

* uuuudeps

* merge conflict resolved

* merge conflict

* merge conflicts

* resolve TODO

* update

* move kzg to a test util fn

* remove trusted setup default impl

* lint fmt

* fix failing test

* lint

* fix test

* Merge branch 'unstable' into beacon-chain-kzg-field-required
2024-09-23 04:54:32 +00:00
Age Manning
18c97a7d64 Update blog reference to new combined blog (#6414)
* Update blog reference to new combined blog

* Update README.md
2024-09-23 00:17:09 +00:00
Age Manning
a97d77c147 Correct ENR decoding on extension trait (#6402)
* Correct enr extension encodings

* Clippy my ol friend

* Correct all encoding and comparisons

* Found some more encodings

* Fix remaining tests
2024-09-20 12:14:57 +00:00
Jimmy Chen
46e0d66e2d Fix deadlock on block cache. (#6412)
* Fix deadlock on block cache.
2024-09-19 04:58:43 +00:00
Eitan Seri-Levi
8a085fc828 Fix test failure on Rust v1.81 (#6407)
* generate rand f64 instead of arbitrary to prevent NaN vals

* reintroduce quickcheck arbitrary but prevet NaN
2024-09-19 03:12:25 +00:00
Daniel Knopik
8b085dd167 Fix phase0 block reward in rewards API (#5101)
* Added Block Rewards

* added new type

* added enum

* Fix phase0 block reward in rewards API (#4929)

* Merge 'guav00a/proposer-rewards-api'

* Merge unstable

* Revamp phase0 reward API tests

- Add test_rewards_base_slashings (testing #5101)
- Improve fix to not include proposer reward in attestation reward API calculation (#4856)
- Adjust test approach for phase0 tests: Pad with empty epochs to include all rewards in calculation
- Simplify and unify code across all reward tests

* Merge branch 'unstable' into fix-4929

* Merge branch 'unstable' into fix-4929

* Merge remote-tracking branch 'origin/unstable' into fix-4929

* Fix merge fallout

* Remove junk revived in merge

* Address review

- check for attestations with lower inclusion delay
- check for double attestations in block
- add test

* Merge branch 'unstable' into fix-4929

* Merge branch 'unstable' into fix-4929
2024-09-17 06:45:02 +00:00
João Oliveira
2f6ad34795 Improve rpc logic (#6400)
* update rpc imports to be explicit

* avoid exposing HandlerEvent outside RPC

it's unnecessary.

* handle Pongs at RPC handler level
2024-09-17 06:12:21 +00:00
Lion - dapplion
e0ccadbae2 Move sync active requests to own modules (#6272)
* Move sync active requests to own modules

* Merge branch 'unstable' into sync-requests-modules
2024-09-12 06:29:34 +00:00
Akihito Nakano
351dd6cb80 Set custody subnets on tests (#6382)
* Set custody subnets on tests

* Enable sampling_with_retries test

* Add another supernode to ensure the node can retry
2024-09-12 06:29:26 +00:00
Age Manning
e5a40fb73b Upgrade to discv5 0.7.0 (#6385)
* Upgrade to discv5 v0.7.0
2024-09-12 00:26:20 +00:00
Eitan Seri-Levi
a94b12b4d5 Persist light client bootstrap (#5915)
* persist light client updates

* update beacon chain to serve light client updates

* resolve todos

* cache best update

* extend cache parts

* is better light client update

* resolve merge conflict

* initial api changes

* add lc update db column

* fmt

* added tests

* add sim

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* fix some weird issues with the simulator

* tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* test changes

* merge conflict

* testing

* started work on ef tests and some code clean up

* update tests

* linting

* noop pre altair, were still failing on electra though

* allow for zeroed light client header

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* merge unstable

* remove unwraps

* remove unwraps

* fetch bootstrap without always querying for state

* storing bootstrap parts in db

* mroe code cleanup

* test

* prune sync committee branches from dropped chains

* Update light_client_update.rs

* merge unstable

* move functionality to helper methods

* refactor is best update fn

* refactor is best update fn

* improve organization of light client server cache logic

* fork diget calc, and only spawn as many blcoks as we need for the lc update test

* resovle merge conflict

* add electra bootstrap logic, add logic to cache current sync committee

* add latest sync committe branch cache

* fetch lc update from the cache if it exists

* fmt

* Fix beacon_chain tests

* Add debug code to update ranking_order ef test

* Fix compare code

* merge conflicts

* merge conflict

* add better error messaging

* resolve merge conflicts

* remove lc update from basicsim

* rename sync comittte variable and fix persist condition

* refactor get_light_client_update logic

* add better comments, return helpful error messages over http and rpc

* pruning canonical non checkpoint slots

* fix test

* rerun test

* update pruning logic, add tests

* fix tests

* fix imports

* fmt

* refactor db code

* Refactor db method

* Refactor db method

* add additional comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-bootstrap

* fix merge

* linting

* merge conflict

* prevent overflow

* enable lc server for http api tests

* fix tests

* remove prints

* remove warning

* revert change
2024-09-10 00:27:49 +00:00
Lion - dapplion
51091a40fa Register processor queue length as histogram (#6012)
* Register processor queue length as histogram

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into processor-queue-histogram
2024-09-09 12:18:30 +00:00
Jimmy Chen
815567a91a Add more granularity to data column proof computation metric (#6371)
* Add more granularity to data column proof computation metric to capture more variations between 0.25 and 1 second.
2024-09-09 11:02:16 +00:00
João Oliveira
873748d2c3 simplify rpc codec logic (#6304)
* simplify rpc codec logic

* Merge branch 'unstable' of github.com:sigp/lighthouse into simplify-rpc-codec

* Merge branch 'unstable' of github.com:sigp/lighthouse into simplify-rpc-codec

* Merge branch 'unstable' of github.com:sigp/lighthouse into simply-rpc-codec

* Merge branch 'unstable' into simplify-rpc-codec

* Merge branch 'unstable' into simplify-rpc-codec
2024-09-06 09:42:34 +00:00
Michael Sproul
d6861380a2 Delete legacy payload reconstruction (#6213)
* Delete legacy payload reconstruction

* Delete unneeded failing test

* Merge remote-tracking branch 'origin/unstable' into remove-more-ethers

* Merge remote-tracking branch 'origin/unstable' into remove-more-ethers

* Cleanups
2024-09-06 08:10:55 +00:00
Jimmy Chen
c0b4f01cf3 Improve get_custody_columns validation, caching and error handling (#6308)
* Improve `get_custody_columns` validation, caching and error handling.

* Merge branch 'unstable' into get-custody-columns-error-handing

* Fix failing test and add more test.

* Fix failing test and add more test.

* Merge branch 'unstable' into get-custody-columns-error-handing

# Conflicts:
#	beacon_node/lighthouse_network/src/discovery/subnet_predicate.rs
#	beacon_node/lighthouse_network/src/peer_manager/peerdb.rs
#	beacon_node/lighthouse_network/src/peer_manager/peerdb/peer_info.rs
#	beacon_node/lighthouse_network/src/types/globals.rs
#	beacon_node/network/src/service.rs
#	consensus/types/src/data_column_subnet_id.rs

* Add unit test to make sure the default specs won't panic on the `compute_custody_requirement_subnets` function.

* Add condition when calling `compute_custody_subnets_from_metadata` and update logs.

* Validate `csc` when returning from enr. Remove `csc` computation on connection since we get them on metadata anyway.

* Add `peers_per_custody_subnet_count` to track peer csc and supernodes.

* Disconnect peers with invalid metadata and find other peers instead.

* Fix sampling tests.

* Merge branch 'unstable' into get-custody-columns-error-handing

* Merge branch 'unstable' into get-custody-columns-error-handing
2024-09-06 07:39:16 +00:00
Håvard Anda Estensen
0c5e25b62a Use tikv-jemallocator instead of jemallocator (#6354)
* Use tikv-jemallocator instead of jemallocator

* Merge branch 'unstable' into tikv-jemallocator

* Bump tikv-jemallocator and tikv-jemalloc-ctl
2024-09-06 03:21:21 +00:00
Michael Sproul
c824142a6d Fix lints for Rust 1.81 (#6363)
* Fix lints for Rust 1.81
2024-09-06 01:45:34 +00:00
Age Manning
df19b6220a Remove beta tag from gossipsub 1.2 (#6344)
* Remove the beta tag from gossipsub v1.2

* fix clippy

* Merge branch 'unstable' into remove-beta-tag
2024-09-05 16:19:07 +00:00
Lion - dapplion
369807becc Check known parent on rpc blob process (#5893)
* Check known parent on rpc blob process

* fix test

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into blob-unknown-parent
2024-09-05 15:24:21 +00:00
Michael Sproul
0e94fe1aa8 Clarify validator monitor block log (#6342)
* Clarify validator monitor block log

* Merge branch 'unstable' into clarify-block-log
2024-09-05 05:39:05 +00:00
Pawan Dhananjay
0fb4a2046c Metadata request ordering (#6336)
* Send metadata request ordering

* Merge branch 'unstable' into metadata-order
2024-09-05 05:39:03 +00:00
Jimmy Chen
b50ce60662 Add blob count label to DATA_COLUMN_SIDECAR_COMPUTATION metric (#6340)
* Add blob count label to `DATA_COLUMN_SIDECAR_COMPUTATION` metric, and move metrics into the compute function, recording only successful computation.

* Move `discard_timer_on_break` usage to caller site.

* Merge branch 'unstable' into compute-data-column-metric

* Merge branch 'unstable' into compute-data-column-metric
2024-09-05 05:39:00 +00:00
Michael Sproul
26c19d65a3 Enable large_stack_frames lint (#6343)
* Enable `large_stack_frames` lint
2024-09-05 05:01:16 +00:00
Michael Sproul
672dcbd868 Ignore Rust 1.82 warnings about void patterns (#6357)
* Ignore Rust 1.82 warnings about void patterns
2024-09-05 02:41:19 +00:00
ethDreamer
4e675cf5da Light Client Bug Fix (#6299)
* Light Client Bug Fix
2024-09-03 13:47:49 +00:00
chonghe
84f0011889 Update manual checkpoint sync content in Lighthouse book (#6338)
* Update manual checkpiont sync

* Update faq

* Minor revision

* Minor revision
2024-09-03 13:47:46 +00:00