Commit Graph

3017 Commits

Author SHA1 Message Date
Lion - dapplion
319b4a2467 Skip creating child lookup if parent is never created (#5803)
* Skip creating child lookup if parent is never created
2024-05-17 10:58:27 +00:00
realbigsean
1d6160549d use electra feature in notifier completeness check (#5786)
* use electra feature in notifier completeness check

* update capella and denab readiness chacks to use fork name rather than random fields
2024-05-15 10:37:59 +00:00
Eitan Seri-Levi
6636167503 Log block import source (#5738)
* the default target peers is 100

* add some comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into track-block-import-source

* add block import source

* revert

* update logging text

* fix tests

* lint

* use % instaed of to_string
2024-05-15 09:17:06 +00:00
Lion - dapplion
6f45ad4534 Log stuck lookups (#5778)
* Log stuck lookups every interval

* Implement debug manually

* Add comment

* Do not print peers twice

* Add SYNC_LOOKUPS_STUCK metric

* Skip logging request root

* use derivative

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into log-stuck-lookups

* add req id to debug

* Merge remote-tracking branch 'sigp/unstable' into log-stuck-lookups

* Fix conflict with unstable
2024-05-14 17:34:26 +00:00
Lion - dapplion
683d9df63b Don't request block components until having block (#5774)
* Don't request block components until having block

* Update tests

* Resolve todo comment

* Merge branch 'unstable' into request-blocks-first
2024-05-14 14:50:38 +00:00
Lion - dapplion
ce66ab374e Enforce sync lookup receives a single result (#5777)
* Enforce sync lookup receives a single result
2024-05-14 10:12:48 +00:00
Lion - dapplion
f37ffe4b8d Do not request current child lookup peers (#5724)
* Do not request current child lookup peers

* Update tests
2024-05-13 15:13:32 +00:00
Lion - dapplion
93e0649abc Notify lookup sync of gossip processing results (#5722)
* Notify lookup sync of gossip processing results

* Add tests

* Add GossipBlockProcessResult event

* Re-add dropped comments

* Update beacon_node/network/src/network_beacon_processor/sync_methods.rs

* update test_lookup_disconnection_peer_left
2024-05-13 11:41:29 +00:00
Lion - dapplion
5135a77e38 Move sync lookup trait function to its caller (#5704)
* Move sync lookup trait function to its caller

* lint
2024-05-06 17:55:12 +00:00
Lion - dapplion
b87c36ac0e Report RPC Errors to the application on peer disconnections (#5680)
* Report RPC Errors to the application on peer disconnections

Co-authored-by: Age Manning <Age@AgeManning.com>

* Expect RPCError::Disconnect to fail ongoing requests

* Drop lookups after peer disconnect and not awaiting events

* Allow RPCError disconnect through network service

* Update beacon_node/lighthouse_network/src/service/mod.rs

Co-authored-by: Age Manning <Age@AgeManning.com>

* Merge branch 'unstable' into rpc-error-on-disconnect
2024-05-06 17:18:47 +00:00
Lion - dapplion
436d54e4bf Consistent logging of full root in lookup debug logs (#5700)
* Consistent logging of full root in lookup debug logs

* Tag sync log with service

* More logs

* Log when new peers are added

* Don't shortcircuit add_peer
2024-05-06 15:41:38 +00:00
Michael Sproul
d3d429ff5c Improve logging and metrics for block publication (#5699)
* Improve logging and metrics for block publication

* Add better buckets

* Bump SQL connection timeout for tests.
2024-05-03 08:24:49 +00:00
Lion - dapplion
01e4e3527e Check da_checker before doing a block lookup request (#5681)
* Check da_checker before doing a block lookup request

* Ensure consistent handling of lookup result

* use req resp pre import cache rather than da checker
2024-05-01 14:45:36 +00:00
Age Manning
59753f5fed Improve ENR updates (#5483)
* Improve ENR updates

* forever fmt

* Appease my old friend clippy

* Merge network unstable
2024-05-01 08:40:19 +00:00
realbigsean
b9655b658e Ensure block only range requests don't fail on download (#5675)
* ensure pruned blobs don't fail on download

* Typo
2024-05-01 06:10:15 +00:00
realbigsean
beaa586d98 delete spammy log (#5672)
*  delete spammy log
2024-04-30 21:31:58 +00:00
Lion - dapplion
ce66582c16 Merge parent and current sync lookups (#5655)
* Drop lookup type trait for a simple arg

* Drop reconstructed for processing

* Send parent blocks one by one

* Merge current and parent lookups

* Merge current and parent lookups clean up todos

* Merge current and parent lookups tests

* Merge remote-tracking branch 'origin/unstable' into sync-merged-lookup

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into sync-merged-lookup

* fix compile after merge

* #5655 pr review (#26)

* fix compile after merge

* remove todos, fix typos etc

* fix compile

* stable rng

* delete TODO and unfilled out test

* make download result a struct

* enums instead of bools as params

* fix comment

* Various fixes

* Track ignored child components

* Track dropped lookup reason as metric

* fix test

* add comment describing behavior of avail check error

*  update ordering
2024-04-30 20:12:15 +00:00
Jimmy Chen
63fad7e330 Remove snapshot cache related code (#5661)
* Remove snapshot cache and other references.

* Fix default state cache size in docs

* Remove cache miss comment entirely

* Add state cache CLI tests
2024-04-30 16:08:13 +00:00
Eitan Seri-Levi
d3bf9a8956 Proposer and attester slashing sse events (#5327)
* default vc to block v3 endpoint and deprecate block-v3 flag

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into unstable

* add proposer and attester event variants

* add TOOOs

* add tests, event triggers

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into proposer-and-attester-slashing-sse-events

* revert

* revert

* remove double event tracking

* Merge branch 'unstable' into proposer-and-attester-slashing-sse-events

* remove todo, fix test

* resolve merge conflicts

* Merge branch 'proposer-and-attester-slashing-sse-events' of https://github.com/eserilev/lighthouse into proposer-and-attester-slashing-sse-events

* leftover debugging

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into proposer-and-attester-slashing-sse-events

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into proposer-and-attester-slashing-sse-events
2024-04-30 08:56:45 +00:00
realbigsean
c33edc82eb Beta compiler fix (#5659)
* fix beta compiler compilation

* remove unused import

* Revert "remove unused import"

This reverts commit 0bef36b05b.

* Revert "fix beta compiler compilation"

This reverts commit 23152cf4cc.

* rename ununsed fields

* allow dead code on some error variants

* remove unused blob download queue

* add back debug to backfill error

* more allow dead code on errors
2024-04-29 20:45:54 +00:00
Lion - dapplion
40d412629e Ignore gossip blob already imported (#5656)
* Ignore gossip blob already imported
2024-04-29 14:30:13 +00:00
realbigsean
000a4fdf4d Electra other containers (#5652)
* add new fields to execution payload and header

* beacon state changes

* partial beacon state

* safe arith in upgrade to electra

* initialize balances cache in interop genesis state

* Revert "initialize balances cache in interop genesis state"

This reverts commit c60b522865.

* always initialize balances cache if necessary in electra upgrade

* build cache earlier

* fix block test

* per fork NUM_FIELDS_POW2

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into electra-other-containers

* fix lints

* get fields based on post state, as is spec'd

* fix type and move cache build
2024-04-26 20:17:23 +00:00
Mac L
13f94ef0f3 Rename Merge to Bellatrix (#5601)
* Rename Merge to Bellatrix

* Remove tree-hash-cache which got readded from the rebase
2024-04-25 20:19:41 +00:00
ethDreamer
4a48d7b546 Encode Execution Engine Client Version In Graffiti (#5290)
* Add `engine_clientVersionV1` structs

* Implement `engine_clientVersionV1`

* Update to latest spec changes

* Implement GraffitiCalculator Service

* Added Unit Tests for GraffitiCalculator

* Address Mac's Comments

* Remove need to use clap in beacon chain

* Merge remote-tracking branch 'upstream/unstable' into el_client_version_graffiti

* Merge branch 'unstable' into el_client_version_graffiti

# Conflicts:
#	beacon_node/beacon_chain/Cargo.toml
2024-04-24 06:02:48 +00:00
Lion - dapplion
c38b05d640 Drop lookup type trait for a simple arg (#5620)
* Drop lookup type trait for a simple arg
2024-04-24 06:02:45 +00:00
Michael Sproul
61962898e2 In-memory tree states (#5533)
* Consensus changes

* EF tests

* lcli

* common and watch

* account manager

* cargo

* fork choice

* promise cache

* beacon chain

* interop genesis

* http api

* lighthouse

* op pool

* beacon chain misc

* parallel state cache

* store

* fix issues in store

* IT COMPILES

* Remove some unnecessary module qualification

* Revert Arced pubkey optimization (#5536)

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Fix caching, rebasing and some tests

* Remove unused deps

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Small cleanups

* Revert shuffling cache/promise cache changes

* Fix state advance bugs

* Fix shuffling tests

* Remove some resolved FIXMEs

* Remove StateProcessingStrategy

* Optimise withdrawals calculation

* Don't reorg if state cache is missed

* Remove inconsistent state func

* Fix beta compiler

* Rebase early, rebase often

* Fix state caching behaviour

* Update to milhouse release

* Fix on-disk consensus context format

* Merge remote-tracking branch 'origin/unstable' into tree-states-memory

* Squashed commit of the following:

commit 3a16649023
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Thu Apr 18 14:26:09 2024 +1000

    Fix on-disk consensus context format

* Keep indexed attestations, thanks Sean

* Merge branch 'on-disk-consensus-context' into tree-states-memory

* Merge branch 'unstable' into tree-states-memory

* Address half of Sean's review

* More simplifications from Sean's review

* Cache state after get_advanced_hot_state
2024-04-24 01:22:36 +00:00
Michael Sproul
72a33604b3 Add timing for block availability (#5510)
* Add timing for block availability

* Attestation metrics analysis

* Prettier printing

* Add some metrics and timings to track late blocks

* Update to latest unstable

* fmt

* Merge latest unstable

* Small tweaks

* Try pushing blob timing down into verification

* Simplify for clippy
2024-04-23 13:13:34 +00:00
Jimmy Chen
532206e008 Fix stuck backfill when scheduled work queue is at capacity (#5575)
* Fix stuck backfill and add regression test.

* Remove unnecessary `yield_now`

* Merge branch 'unstable' into fix-stuck-backfill

* Revert previous change and add extra comment.

* Merge branch 'unstable' into fix-stuck-backfill

* Update tests to use configured event schedule instead of hard coded values.

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into fix-stuck-backfill
2024-04-22 16:06:46 +00:00
Lion - dapplion
f7aca97a55 Handle sync lookup request streams in network context (#5583)
* by-root-stream-terminator

* Fix tests

* Resolve merge conflicts

* Log report reason

* Some lints and bugfixes (#23)

* fix lints

* bug fixes

* Fix tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into handle-sync-lookup-requests

* Pr 5583 review (#24)

* add bad state warn log

* add rust docs to new fields in `SyncNetworkContext`

* remove timestamp todo

* add back lookup verify error

* remove TODOs
2024-04-22 16:06:39 +00:00
Mac L
67f8405921 Update Simulator tests (#5520)
* Rewrite Simulator

* Add fallback simulator

* Try Sean's test fix

* More fixes

* Cleanup

* Merge branch 'unstable' into update-simulator

* Update cli.rs

* Add sync sim to basic sim

* Formatting

* Add fixes and new block production check

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into update-simulator

* fix compile
2024-04-22 15:08:36 +00:00
Michael Sproul
5a9e973f04 Fix on-disk consensus context format (#5598)
* Fix on-disk consensus context format

* Keep indexed attestations, thanks Sean
2024-04-19 08:18:30 +00:00
AMIT SINGH
62e4abfbff Fix execution layer redundancy (#5588)
* remove execution layer url redundancy

* fix typo

* fix tests

* fix formatting
2024-04-18 07:14:56 +00:00
ethDreamer
cda926ce1b Rename Functions to More Closely Match Spec (#5591)
* Rename Functions to More Closely Match Spec
2024-04-16 19:03:08 +00:00
ethDreamer
f68989815c Restore Log on Error & Spawn Blocking in Streamer (#5585)
* Restore Logging in Error Cases

* Use Spawn Blocking for Loading Blocks in Streamer

* Merge remote-tracking branch 'upstream/unstable' into request_logging_spawn_blocking

* Address Sean's Comments

* save a clone
2024-04-16 14:56:00 +00:00
Lion - dapplion
e5b8d1237d Use Action in single_block_component_processed (#5564)
* Use Action in single_block_component_processed

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_block_component_processed-action

* fix compile after merge

* add continue action for when we are awaiting other parts of missing components
2024-04-16 12:59:53 +00:00
Lion - dapplion
f5e0404fb8 Ensure proper ReqResp blocks_by_* response stream termination (#5582)
* Ensure proper ReqResp blocks_by_* response stream termination

* retrigger CI
2024-04-15 21:20:50 +00:00
Lion - dapplion
fee2ee9c08 Make SingleLookupRequestState fields private (#5563)
* Make SingleLookupRequestState fields private

* Fix tests
2024-04-15 13:47:08 +00:00
Lion - dapplion
b6a1c863a2 Use spawn_async in ByRoot handling workers (#5557)
* Use spawn_async in ByRoot handling workers

* box large variants
2024-04-12 19:30:04 +00:00
Lion - dapplion
116a55e8a5 Ensure proper ReqResp response stream termination (#5556)
* Ensure proper ReqResp response stream termination

* Update beacon_node/network/src/network_beacon_processor/rpc_methods.rs

* Update beacon_node/network/src/network_beacon_processor/rpc_methods.rs

* cargo fmt
2024-04-12 16:15:04 +00:00
Lion - dapplion
6fb0b2ed78 Sync lookup dedup range and blobs (#5561)
* Handle sync range blocks as blocks and blobs

* Merge range sync and backfill sync handling

* Update tests

* Add no_blobs_into_responses test

* Address @realbigsean comments

* Merge remote-tracking branch 'origin/unstable' into sync-lookup-dedup-range-and-blobs
2024-04-12 15:39:11 +00:00
Lion - dapplion
5fdd3b39bb Allow 1 count block request to return 0 blocks (#5554)
* Allow 1 count block request to return 0 blocks

* Address @pawanjay176 review
2024-04-12 14:22:20 +00:00
Eitan Seri-Levi
6bac5ce12b Deprecate http-spec-fork and http-allow-sync-stalled (#5500)
* deprecate flags

* fmt

* remove backslash

* remove hidden flags from the book

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into deprecate-http-spec-fork-and-http-allow-sync-stalled

* add warn, re-add tests

* Apply suggestions from code review

* Merge remote-tracking branch 'origin/unstable' into deprecate-http-spec-fork-and-http-allow-sync-stalled

* Fix imports
2024-04-12 04:21:00 +00:00
Jimmy Chen
d30ba976a1 Use the mesh_n value from NetworkLoad for PeerScoreSettings (#5013)
* Build `gs_config` and use the `mesh_n` config for `PeerScoreSettings`

* Remove unused imports

* Merge branch 'unstable' into peer-score-settings

# Conflicts:
#	beacon_node/lighthouse_network/src/config.rs
#	beacon_node/lighthouse_network/src/service/gossipsub_scoring_parameters.rs
#	beacon_node/lighthouse_network/src/service/mod.rs
2024-04-11 20:14:15 +00:00
Jimmy Chen
7e49f82726 Only load Kzg in tests if necessary and only load it once (#5555)
* Only load KZG once if necessary  in tests.
2024-04-11 20:14:11 +00:00
Lion - dapplion
34dbb32610 Nest lookup type into request id SingleBlock and SingleBlob (#5562)
* Nest lookup type into block lookup RequestId
2024-04-11 16:32:06 +00:00
realbigsean
3d4e6e263e Remove availability view trait (#5544)
* Move processing cache out of DA

* Merge branch 'sigp/unstable' into non-da-processing-cach

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into non-da-processing-cache

* remove unused file, remove outdated TODO, add is_deneb check to missing blob id calculations

* remove availability view trait

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove-availability-view-trait

* fix lints
2024-04-11 15:33:54 +00:00
Lion - dapplion
54fbdda4d2 Add logging in UnknownBlockHashFromAttestation handling (#5546)
* Use should_search_for_block in unknown block root event

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into should_search_for_block
2024-04-10 20:52:22 +00:00
Lion - dapplion
30dc260472 Move processing cache out of DA (#5420)
* Move processing cache out of DA

* Merge branch 'sigp/unstable' into non-da-processing-cach

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into non-da-processing-cache

* remove unused file, remove outdated TODO, add is_deneb check to missing blob id calculations

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into non-da-processing-cache

* fix lints
2024-04-10 17:19:40 +00:00
Lion - dapplion
b1f9751a69 Event-based block lookup tests (#5534)
* WIP

* Initial working version of new sync tests.

* Remove sync traits and fix lints.

* Reduce internal method visibility and make test method instead. Remove extra beacon chain harness instance created in tests.

* Improve `SyncTester` api.

* Fix lint.

* Test example

* Lookup tests using rig

* Tests should interface with events only

* lint

* Skip deneb test pre-deneb

* Add more assertions

* Remove logging changes

* Address @jimmygchen comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into bn-p2p-tests

* remove unused assertions

* fix lint
2024-04-10 12:05:18 +00:00
pahor167
72af6fb83a Skip calculation of boosted_relay_value when builder_boost_factor pro… (#5352)
* Skip calculation of boosted_relay_value when builder_boost_factor provided

* PR comments
2024-04-10 12:05:13 +00:00