Commit Graph

1322 Commits

Author SHA1 Message Date
Michael Sproul
a249dbfc1a Merge remote-tracking branch 'origin/unstable' into tree-states-update 2024-03-11 15:29:03 +11:00
Michael Sproul
f93844e63b Optimise concurrent block production (#5368)
* Optimise concurrent block production
2024-03-08 05:15:28 +00:00
João Oliveira
65c4ff0775 remove exit-future (#5183)
* remove exit-future usage,

as it is non maintained, and replace with async-channel which is already in the repo.

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove-exit-future

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove-exit-future
2024-02-27 22:12:44 +00:00
Lion - dapplion
928915c718 as_store_bytes not fallible (#5285) 2024-02-27 12:23:05 +11:00
Lion - dapplion
5dfc5c1f88 Remove milhouse export aliases (#5286)
Closes #5141
2024-02-27 12:19:22 +11:00
Pawan Dhananjay
3ab9d3a84e Add a cli option for the snapshot cache size (#5270)
* Add a cli option for snapshot cache size

* Remove junk

* Make snapshot_cache module public

* lint

* Update docs
2024-02-26 05:19:39 +00:00
dapplion
a5d3408c59 Merge remote-tracking branch 'sigp/epoch-single-pass' into tree-states 2024-02-23 11:49:06 +08:00
Michael Sproul
f9c9c40a67 Fix tree-states tests (#5277)
* Fix beta compiler lints

* Fix fork choice tests

* Fix op_pool tests

---------

Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
2024-02-23 09:30:31 +11:00
Michael Sproul
26117a558f Merge remote-tracking branch 'origin/unstable' into tree-states 2024-02-22 15:43:04 +11:00
Michael Sproul
c9702cb0a1 Download checkpoint blobs during checkpoint sync (#5252)
* MVP implementation (untested)

* update store checkpoint sync test

* update cli help

* Merge pull request #5253 from realbigsean/checkpoint-blobs-sean

Checkpoint blobs sean

* Warn only if blobs are missing from server

* Merge remote-tracking branch 'origin/unstable' into checkpoint-blobs

* Verify checkpoint blobs

* Move blob verification earlier
2024-02-19 02:22:23 +00:00
realbigsean
f21472991d check the da cache and the attester cache in responding to RPC requests (#5138)
* check the da cache and the attester cache in responding to RPC requests

* use the processing cache instead

* update comment

* add da cache metrics

* rename early attester cache method

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into check-da-cache-in-rpc-response

* make rustup update run on the runners

* Revert "make rustup update run on the runners"

This reverts commit d097e9bfa8.
2024-02-19 02:22:15 +00:00
ethDreamer
a264afd19f Verify Versioned Hashes During Optimistic Sync (#4832)
* Convert NewPayloadRequest to use Reference

* Refactor for Clarity

* Verify Versioned Hashes

* Added Tests for Version Hash Verification

* Added Moar Tests

* Fix Problems Caused By  Merge

* Update to use Alloy Instead of Reth Crates (#14)

* Update beacon_node/execution_layer/src/engine_api/new_payload_request.rs

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* Faster Versioned Hash Extraction

* Update to rust 1.75 & Pin alloy-consensus
2024-02-18 12:40:45 +00:00
Michael Sproul
b1f7f0aaf9 Remove progressive balances mode (#5224) 2024-02-14 11:55:27 +11:00
Michael Sproul
9eb1685506 Use epoch cache in block packing (#5223) 2024-02-14 11:55:11 +11:00
realbigsean
4172d9f75c Update to consensus spec v1.4.0-beta.6 (#5094)
* get latest ef tests passing

* fix tests

* Fix invalid payload recovery tests

* Merge branch 'unstable' into update-to-spec-v1.4.0-beta.6

* Revert "fix tests"

This reverts commit 0c875b02e0.

* Fix fork choice def. tests

* Update beacon_node/beacon_chain/tests/payload_invalidation.rs
2024-02-08 18:08:21 +00:00
Jimmy Chen
39e9f7dc6b Fix Rust beta compiler errors (1.77) (#5180)
* Lint fixes

* More fixes for beta compiler.

* Format fixes

* Move `#[allow(dead_code)]` to field level.

* Remove old comment.

* Update beacon_node/execution_layer/src/test_utils/mod.rs

Co-authored-by: João Oliveira <hello@jxs.pt>

* remove duplicate line
2024-02-05 17:54:11 +00:00
Michael Sproul
0b6416c444 Re-disable block verification tests in debug (#5155)
* Re-disable block verification tests in debug
2024-02-01 08:35:10 +00:00
Age Manning
7582da7855 Test backfill (#5109)
* Test backfill

* Revert cargo.toml

* Update beacon_node/beacon_chain/src/builder.rs

Co-authored-by: João Oliveira <hello@jxs.pt>

* Remove redundant code
2024-01-31 18:11:45 +00:00
Lion - dapplion
b035638f9b Compute recent lightclient updates (#4969)
* Compute recent lightclient updates

* Review PR

* Merge remote-tracking branch 'upstream/unstable' into lc-prod-recent-updates

* Review PR

* consistent naming

* add metrics

* revert dropping reprocessing queue

* Update light client optimistic update re-processing logic. (#7)

* Add light client server simulator tests. Co-authored by @dapplion.

* Merge branch 'unstable' into fork/dapplion/lc-prod-recent-updates

* Fix lint

* Enable light client server in simulator test.

* Fix test for light client optimistic updates and finality updates.
2024-01-31 05:25:51 +00:00
Michael Sproul
68a9a2ee20 Merge remote-tracking branch 'origin/unstable' into tree-states 2024-01-30 17:13:57 +11:00
Michael Sproul
8e68926162 fsync during backfill to prevent DB corruption (#5144) 2024-01-30 17:08:43 +11:00
Michael Sproul
7862c71bb5 Fix tree-states sub-epoch diffs (#5097) 2024-01-30 15:56:48 +11:00
Michael Sproul
6f3af67362 Fix off-by-one in backfill sig verification (#5120)
* Fix off-by-one in backfill sig verification

* Add self-referential PR link
2024-01-30 00:33:01 +00:00
realbigsean
a4fcf60bcc Increase attestation cache sizes (#5135)
* increase observed attesters and aggregates cache sizes

* fix comment
2024-01-30 00:32:57 +00:00
Sergey Kisel
64efdaf39a #5102 Fix load_state_for_block_production metric mapping (#5103)
* #5102 Fix load_state_for_block_production metric mapping
2024-01-30 00:32:52 +00:00
Michael Sproul
262e5f22bc Merge remote-tracking branch 'origin/unstable' into tree-states 2024-01-25 15:10:19 +11:00
realbigsean
1cebf41452 Backfill blob storage fix (#5119)
* store blobs in the correct db in backfill

* add database migration

* add migration file

* remove log info suggesting deneb isn't schedule

* add batching in blob migration
2024-01-24 09:35:02 +11:00
Michael Sproul
a403138ed0 Reduce size of futures in HTTP API to prevent stack overflows (#5104)
* Box::pin a few big futures

* Arc the blocks early in publication

* Fix more tests
2024-01-23 15:32:07 +11:00
Lion - dapplion
585124fb2f Hold HeadTracker lock until persisting to disk (#5084)
* Fix head tracker drop order on un-ordered shutdown

* lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2024-01-22 15:14:11 +11:00
Mac L
a0b407c15d Add Deneb readiness logging (#5074) 2024-01-18 15:21:38 -05:00
Michael Sproul
6262be7219 Don't error on inactive indices in att. rewards 2024-01-14 09:41:42 +11:00
Michael Sproul
664a7784f8 Add cache for parallel HTTP requests (#4879) 2024-01-11 17:13:43 +11:00
Michael Sproul
8db17dac1d Merge remote-tracking branch 'origin/unstable' into tree-states 2024-01-11 13:15:06 +11:00
Paul Hauner
be79f74c6d Fix IncorrectAttestationSource error in attn. simulator (#5048)
* Don't produce attestations when syncing

* Handle `IncorrectAttestationSource`
2024-01-10 10:47:00 +11:00
Michael Sproul
7e948eec9d Fix block v3 reward encodings (#5049)
* Fix block v3 reward encodings

* Use crates.io version
2024-01-10 10:44:07 +11:00
realbigsean
b47e3f252e Runtime rpc request sizes (#4841)
* add runtime variable list type

* add configs to ChainSpec

* git rid of max request blocks type

* fix tests and lints

* remove todos

* git rid of old const usage

* fix decode impl

* add new config to `Config` api struct

* add docs fix compilt

* move methods for per-fork-spec to chainspec

* get values off chain spec

* fix compile

* remove min by root size

* add tests for runtime var list

---------

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2024-01-09 10:23:47 +11:00
realbigsean
f70c32ec70 create unified slashing cache (#5033)
* create unified slashing cache

* add observed slashable file

* fix broadcast validation tests

* revert block seen cache changes

* clean up slashable cache test

* check header signatures for RPC blobs

* don't throw error on RPC signature invalie
2024-01-08 10:30:57 -05:00
Eitan Seri-Levi
9c1505d082 Block v3 builder boost factor (#5035)
* builder boost factor

* default boost factor

* revert

* deprecate always_prefer_builder_payload, builder-profit-threshold, ignore_builder_override_suggestion_threshold and builder_comparison_factor flags

* revert

* set deprecated flags to no op, revert should_override_builder

* fix test, calc boosted relay value correctly, dont calculate if none

* Add deprecation warnings and restore CLI docs
2024-01-08 11:10:32 +11:00
Michael Sproul
af11e78ae1 Clean up blockv3 metadata and client (#5015)
* Improve block production v3 client

* Delete wayward line

* Overhaul JSON endpoint as well

* Rename timeout param

* Update tests

* I broke everything

* Ah this is an insane fix

* Remove unnecessary optionals

* Doc fix
2023-12-22 09:39:17 -05:00
realbigsean
c55608be10 suppress error on duplicate blobs (#4995) 2023-12-18 12:15:12 -05:00
Jimmy Chen
b0c374c1ca Update dependencies to get rid of the yanked deps (#4994)
* Initial attempt to upgrade hashbrown to latest to get rid of the crate warnings.

* Replace `.expect()` usage with use of `const`.

* Update ahash 0.7 as well

* Remove unsafe code.

* Update `lru` to 0.12 and fix release test errors.

* Set non-blocking socket

* Bump testcontainers to 0.15.

* Fix lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>
2023-12-15 18:31:59 +11:00
Michael Sproul
f1113540d8 Fix off-by-one bug in missed block detection (#5012)
* Regression test

* Fix the bug
2023-12-15 14:23:54 +11:00
Pawan Dhananjay
ae4a296089 Convert a FullPayload to a BlindedPayload correctly (#5005) 2023-12-14 08:44:14 -05:00
Joel Rousseau
189430a45c Add attestation simulator (#4880)
* basic scaffold

* remove unnecessary ?

* check if committee cache is init

* typed ValidatorMonitor with ethspecs + store attestations within

* nits

* process unaggregated attestation

* typo

* extract in func

* add tests

* better naming

* better naming 2

* less verbose

* use same naming as validator monitor

* use attestation_simulator

* add metrics

* remove cache

* refacto flag_indices process

* add lag

* remove copying state

* clean and lint

* extract metrics

* nits

* compare prom metrics in tests

* implement lag

* nits

* nits

* add attestation simulator service

* fmt

* return beacon_chain as arc

* nit: debug

* sed s/unaggregated/unagg.//

* fmt

* fmt

* nit: remove unused comments

* increase max unaggregated attestation hashmap to 64

* nit: sed s/clone/copied//

* improve perf: remove unecessary hashmap copy

* fix flag indices comp

* start service in client builder

* remove //

* cargo fmt

* lint

* cloned keys

* fmt

* use Slot value instead of pointer

* Update beacon_node/beacon_chain/src/attestation_simulator.rs

Co-authored-by: Paul Hauner <paul@paulhauner.com>

---------

Co-authored-by: Paul Hauner <paul@paulhauner.com>
2023-12-14 11:44:56 +11:00
Michael Sproul
c8dc082ba8 Merge remote-tracking branch 'origin/unstable' into tree-states 2023-12-14 09:59:43 +11:00
Pawan Dhananjay
a3a370302a Use the block header to compute the canonical_root (#5003) 2023-12-14 09:24:36 +11:00
Gua00va
a3fb27c99b add forK_choice_read_lock as parameter (#4978) 2023-12-13 16:06:00 +11:00
ethDreamer
78ffa378b4 Batch Verify RPC Blobs (#4934) 2023-12-08 16:48:03 -05:00
realbigsean
46184e5ce4 Remove delayed lookups (#4992)
* initial rip out

* fix unused imports

* delete tests and fix lint

* fix peers scoring for blobs
2023-12-08 15:42:55 -05:00
Jimmy Chen
67e0569d9b Fix corrupted DB on networks where the first slot is skipped (Holesky) (#4985)
* Fix zero block roots on skip slots.

* Remove temporary comment, println code and unused imports.

* Remove `println!` in test.
2023-12-07 15:12:06 +11:00