Commit Graph

1408 Commits

Author SHA1 Message Date
Pawan Dhananjay
05727290fb Make max_blobs_per_block a config parameter (#6329)
* First pass

* Add restrictions to RuntimeVariableList api

* Use empty_uninitialized and fix warnings

* Fix some todos

* Merge branch 'unstable' into max-blobs-preset

* Fix take impl on RuntimeFixedList

* cleanup

* Fix test compilations

* Fix some more tests

* Fix test from unstable

* Merge branch 'unstable' into max-blobs-preset

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

* Remove footgun function

* Minor simplifications

* Move from preset to config

* Fix typo

* Revert "Remove footgun function"

This reverts commit de01f923c7.

* Try fixing tests

* Thread through ChainSpec

* Fix release tests

* Move RuntimeFixedVector into module and rename

* Add test

* Remove empty RuntimeVarList awefullness

* Fix tests

* Simplify BlobSidecarListFromRoot

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

* Bump quota to account for new target (6)

* Remove clone

* Fix issue from review

* Try to remove ugliness

* Merge branch 'unstable' into max-blobs-preset

* Fix max value

* Fix doctest

* Fix formatting

* Fix max check

* Delete hardcoded max_blobs_per_block in RPC limits

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset
2025-01-10 06:34:58 +00:00
Mac L
ecdf2d891f Add Fulu boilerplate (#6695)
* Add Fulu boilerplate

* Add more boilerplate

* Change fulu_time to osaka_time

* Merge branch 'unstable' into fulu-boilerplate

* Fix tests

* Merge branch 'unstable' into fulu-boilerplate

* More test fixes

* Apply suggestions

* Remove `get_payload` boilerplate

* Add lightclient fulu types and fix beacon-chain-tests

* Disable Fulu in ef-tests

* Reduce boilerplate for future forks

* Small fixes

* One more fix

* Apply suggestions

* Merge branch 'unstable' into fulu-boilerplate

* Fix lints
2025-01-10 05:25:23 +00:00
Pawan Dhananjay
1f6850fae2 Rust 1.84 lints (#6781)
* Fix few lints

* Fix remaining lints

* Use fully qualified syntax
2025-01-10 01:13:29 +00:00
Jimmy Chen
80cfbea7fe Fix incorrect data column metric name (#6761)
* Fix incorrect data column metric name.
2025-01-08 03:12:37 +00:00
Jimmy Chen
7ec748a108 Implement getBlobSidecars support for PeerDAS (#6755)
* Implement getBlobSidecars endpoint for PeerDAS.

* Merge branch 'unstable' into peerdas-get-blob-sidecars

* Fix incorrect logging.

* Replace `and_then` usage.
2025-01-08 03:12:34 +00:00
Ekaterina Riazantseva
57141d8b4b Add 'beacon_' prefix to PeerDAS metrics names (#6537)
* Add 'beacon_' prefix to PeerDAS metrics names

* Merge remote-tracking branch 'origin/unstable' into peerdas-metrics

* Merge 'origin/unstable' into peerdas-metrics

* Merge remote-tracking branch 'origin/unstable/ into peerdas-metrics

* Add 'beacon_' prefix to 'kzg_data_column' metrics
2025-01-08 00:39:53 +00:00
Michael Sproul
07e82dabc0 Delete OTB verification service (#6631)
* Delete OTB verification service

* Merge branch 'unstable' into delete-otb
2024-12-19 05:46:06 +00:00
Mac L
b2b1faad4e Enforce alphabetically ordered cargo deps (#6678)
* Enforce alphabetically ordered cargo deps

* Fix test-suite

* Another CI fix

* Merge branch 'unstable' into cargo-sort

* Fix conflicts

* Merge remote-tracking branch 'origin/unstable' into cargo-sort
2024-12-19 05:46:03 +00:00
Pawan Dhananjay
2662dc7f8f Fix Sse client api (#6685)
* Use reqwest eventsource for get_events api

* await for Event::Open before returning stream

* fmt

* Merge branch 'unstable' into sse-client-fix

* Ignore lint
2024-12-18 00:05:58 +00:00
Daniel Knopik
02cb2d68ff Enable lints for tests only running optimized (#6664)
* enable linting optimized-only tests

* fix automatically fixable or obvious lints

* fix suspicious_open_options by removing manual options

* fix `await_holding_lock`s

* avoid failing lint due to now disabled `#[cfg(debug_assertions)]`

* reduce future sizes in tests

* fix accidently flipped assert logic

* restore holding lock for web3signer download

* Merge branch 'unstable' into lint-opt-tests
2024-12-17 00:40:35 +00:00
ethDreamer
86891e6d0f builder gas limit & some refactoring (#6583)
* Cache gas_limit

* Payload Parameters Refactor

* Enforce Proposer Gas Limit

* Fixed and Added New Tests

* Fix Beacon Chain Tests
2024-12-16 05:43:58 +00:00
Michael Sproul
a6de0d5e12 Merge remote-tracking branch 'origin/stable' into back-merge-6.0.1 2024-12-16 15:37:52 +11:00
Lion - dapplion
c92c07ff49 Track beacon processor import result metrics (#6541)
* Track beacon processor import result metrics

* Update metric name
2024-12-16 04:33:33 +00:00
Michael Sproul
f3b78889e5 Compact more when pruning states (#6667)
* Compact more when pruning states

* Merge branch 'release-v6.0.1' into compact-more
2024-12-14 08:43:00 +00:00
Michael Sproul
fc0e0ae613 Prevent reconstruction starting prematurely (#6669)
* Prevent reconstruction starting prematurely

* Simplify condition

* Merge remote-tracking branch 'origin/release-v6.0.1' into dont-start-reconstruction-early
2024-12-12 01:58:41 +00:00
Michael Sproul
b2590bcb37 Tweak reconstruction batch size (#6668)
* Tweak reconstruction batch size

* Merge branch 'release-v6.0.1' into reconstruction-batch-size
2024-12-11 22:51:46 +00:00
Michael Sproul
fa6c4c02a3 Fix Rust 1.83 Clippy lints (#6629)
* Fix Rust 1.83 Clippy lints

* Cargo fmt
2024-11-29 02:23:54 +00:00
Michael Sproul
b1e9f69460 Fix v22 schema upgrade (#6591)
* Fix v22 schema upgrade

* Ownership
2024-11-19 22:43:18 +00:00
Michael Sproul
9fdd53df56 Hierarchical state diffs (#5978)
* Start extracting freezer changes for tree-states

* Remove unused config args

* Add comments

* Remove unwraps

* Subjective more clear implementation

* Clean up hdiff

* Update xdelta3

* Tree states archive metrics (#6040)

* Add store cache size metrics

* Add compress timer metrics

* Add diff apply compute timer metrics

* Add diff buffer cache hit metrics

* Add hdiff buffer load times

* Add blocks replayed metric

* Move metrics to store

* Future proof some metrics

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Port and clean up forwards iterator changes

* Add and polish hierarchy-config flag

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Cleaner errors

* Fix beacon_chain test compilation

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Patch a few more freezer block roots

* Fix genesis block root bug

* Fix test failing due to pending updates

* Beacon chain tests passing

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix doc lint

* Implement DB schema upgrade for hierarchical state diffs (#6193)

* DB upgrade

* Add flag

* Delete RestorePointHash

* Update docs

* Update docs

* Implement hierarchical state diffs config migration (#6245)

* Implement hierarchical state diffs config migration

* Review PR

* Remove TODO

* Set CURRENT_SCHEMA_VERSION correctly

* Fix genesis state loading

* Re-delete some PartialBeaconState stuff

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix test compilation

* Update schema downgrade test

* Fix tests

* Fix null anchor migration

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix tree states upgrade migration (#6328)

* Towards crash safety

* Fix compilation

* Move cold summaries and state roots to new columns

* Rename StateRoots chunked field

* Update prune states

* Clean hdiff CLI flag and metrics

* Fix "staged reconstruction"

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix alloy issues

* Fix staged reconstruction logic

* Prevent weird slot drift

* Remove "allow" flag

* Update CLI help

* Remove FIXME about downgrade

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Remove some unnecessary error variants

* Fix new test

* Tree states archive - review comments and metrics (#6386)

* Review PR comments and metrics

* Comments

* Add anchor metrics

* drop prev comment

* Update metadata.rs

* Apply suggestions from code review

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/store/src/hot_cold_store.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Clarify comment and remove anchor_slot garbage

* Simplify database anchor (#6397)

* Simplify database anchor

* Update beacon_node/store/src/reconstruct.rs

* Add migration for anchor

* Fix and simplify light_client store tests

* Fix incompatible config test

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* More metrics

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* New historic state cache (#6475)

* New historic state cache

* Add more metrics

* State cache hit rate metrics

* Fix store metrics

* More logs and metrics

* Fix logger

* Ensure cached states have built caches :O

* Replay blocks in preference to diffing

* Two separate caches

* Distribute cache build time to next slot

* Re-plumb historic-state-cache flag

* Clean up metrics

* Update book

* Update beacon_node/store/src/hdiff.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update beacon_node/store/src/historic_state_cache.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update database docs

* Update diagram

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Update lockbud to work with bindgen/etc

* Correct pkg name for Debian

* Remove vestigial epochs_per_state_diff

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Markdown lint

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Address Jimmy's review comments

* Simplify ReplayFrom case

* Fix and document genesis_state_root

* Typo

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Merge branch 'unstable' into tree-states-archive

* Compute diff of validators list manually (#6556)

* Split hdiff computation

* Dedicated logic for historical roots and summaries

* Benchmark against real states

* Mutated source?

* Version the hdiff

* Add lighthouse DB config for hierarchy exponents

* Tidy up hierarchy exponents flag

* Apply suggestions from code review

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Address PR review

* Remove hardcoded paths in benchmarks

* Delete unused function in benches

* lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Test hdiff binary format stability (#6585)

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Add deprecation warning for SPRP

* Update xdelta to get rid of duplicate deps

* Document test
2024-11-18 01:51:44 +00:00
Eitan Seri-Levi
654fc6acdc Additional light client metrics (#6545)
* Fix db query and add some additional metrics

* fmt

* Update beacon_node/beacon_chain/src/metrics.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Update beacon_node/beacon_chain/src/metrics.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2024-11-15 07:09:54 +00:00
Jimmy Chen
5f053b0b6d Improving blob propagation post-PeerDAS with Decentralized Blob Building (#6268)
* Get blobs from EL.

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Avoid cloning blobs after fetching blobs.

* Address review comments and refactor code.

* Fix lint.

* Move blob computation metric to the right spot.

* Merge branch 'unstable' into das-fetch-blobs

* Merge branch 'unstable' into das-fetch-blobs

# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	beacon_node/beacon_chain/src/block_verification.rs
#	beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs

* Merge branch 'unstable' into das-fetch-blobs

# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs

* Gradual publication of data columns for supernodes.

* Recompute head after importing block with blobs from the EL.

* Fix lint

* Merge branch 'unstable' into das-fetch-blobs

* Use blocking task instead of async when computing cells.

* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs

* Merge remote-tracking branch 'origin/unstable' into das-fetch-blobs

* Fix semantic conflicts

* Downgrade error log.

* Merge branch 'unstable' into das-fetch-blobs

# Conflicts:
#	beacon_node/beacon_chain/src/data_availability_checker.rs
#	beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
#	beacon_node/execution_layer/src/engine_api.rs
#	beacon_node/execution_layer/src/engine_api/json_structures.rs
#	beacon_node/network/src/network_beacon_processor/gossip_methods.rs
#	beacon_node/network/src/network_beacon_processor/mod.rs
#	beacon_node/network/src/network_beacon_processor/sync_methods.rs

* Merge branch 'unstable' into das-fetch-blobs

* Publish block without waiting for blob and column proof computation.

* Address review comments and refactor.

* Merge branch 'unstable' into das-fetch-blobs

* Fix test and docs.

* Comment cleanups.

* Merge branch 'unstable' into das-fetch-blobs

* Address review comments and cleanup

* Address review comments and cleanup

* Refactor to de-duplicate gradual publication logic.

* Add more logging.

* Merge remote-tracking branch 'origin/unstable' into das-fetch-blobs

# Conflicts:
#	Cargo.lock

* Fix incorrect comparison on `num_fetched_blobs`.

* Implement gradual blob publication.

* Merge branch 'unstable' into das-fetch-blobs

* Inline `publish_fn`.

* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs

* Gossip verify blobs before publishing

* Avoid queries for 0 blobs and error for duplicates

* Gossip verified engine blob before processing them, and use observe cache to detect duplicates before publishing.

* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs

# Conflicts:
#	beacon_node/network/src/network_beacon_processor/mod.rs

* Merge branch 'unstable' into das-fetch-blobs

* Fix invalid commitment inclusion proofs in blob sidecars created from EL blobs.

* Only publish EL blobs triggered from gossip block, and not RPC block.

* Downgrade gossip blob log to `debug`.

* Merge branch 'unstable' into das-fetch-blobs

* Merge branch 'unstable' into das-fetch-blobs

* Grammar
2024-11-15 03:34:13 +00:00
Lion - dapplion
38388979db Strict match of errors in backfill sync (#6520)
* Strict match of errors in backfill sync

* Fix tests
2024-11-05 01:00:10 +00:00
Michael Sproul
fe889c619c Simplify light client tests (#6542)
* Simplify light client tests
2024-10-29 01:31:36 +00:00
Age Manning
e31ac508d4 Modularize tracing executor and metrics rename (#6424)
* Tracing executor and metrics rename

* Appease clippy

* Merge branch 'unstable' into modularise-task-executor
2024-10-28 09:41:45 +00:00
Eitan Seri-Levi
9d069a9588 Fix electra light client types (#6361)
* persist light client updates

* update beacon chain to serve light client updates

* resolve todos

* cache best update

* extend cache parts

* is better light client update

* resolve merge conflict

* initial api changes

* add lc update db column

* fmt

* added tests

* add sim

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* fix some weird issues with the simulator

* tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* test changes

* merge conflict

* testing

* started work on ef tests and some code clean up

* update tests

* linting

* noop pre altair, were still failing on electra though

* allow for zeroed light client header

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* merge unstable

* remove unwraps

* remove unwraps

* fetch bootstrap without always querying for state

* storing bootstrap parts in db

* mroe code cleanup

* test

* prune sync committee branches from dropped chains

* Update light_client_update.rs

* merge unstable

* move functionality to helper methods

* refactor is best update fn

* refactor is best update fn

* improve organization of light client server cache logic

* fork diget calc, and only spawn as many blcoks as we need for the lc update test

* resovle merge conflict

* add electra bootstrap logic, add logic to cache current sync committee

* add latest sync committe branch cache

* fetch lc update from the cache if it exists

* fmt

* Fix beacon_chain tests

* Add debug code to update ranking_order ef test

* Fix compare code

* merge conflicts

* merge conflict

* add better error messaging

* resolve merge conflicts

* remove lc update from basicsim

* rename sync comittte variable and fix persist condition

* refactor get_light_client_update logic

* add better comments, return helpful error messages over http and rpc

* pruning canonical non checkpoint slots

* fix test

* rerun test

* update pruning logic, add tests

* fix tests

* fix imports

* fmt

* refactor db code

* Refactor db method

* Refactor db method

* lc electra changes

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra

* add additional comments

* testing lc merkle changes

* lc electra

* update struct defs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra

* fix merge

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-bootstrap

* fix merge

* linting

* merge conflict

* prevent overflow

* enable lc server for http api tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra

* get tests working:

* remove related TODOs

* fix test lint

* Merge branch 'persist-light-client-bootstrap' of https://github.com/eserilev/lighthouse into light-client-electra

* fix tests

* fix conflicts

* remove prints

* Merge branch 'persist-light-client-bootstrap' of https://github.com/eserilev/lighthouse into light-client-electra

* remove warning

* resolve conflicts

* merge conflicts

* linting

* remove comments

* cleanup

* linting

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra

* pre/post electra light client cached data

* add proof type alias

* move is_empty_branch method out of impl

* add ssz tests for all forks

* refactor beacon state proof codepaths

* rename method

* fmt

* clean up proof logic

* refactor merkle proof api

* fmt

* Merge branch 'unstable' into light-client-electra

* Use superstruct mapping macros

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into light-client-electra

* rename proof to merkleproof

* fmt

* Resolve merge conflicts

* merge conflicts
2024-10-25 05:19:13 +00:00
Michael Sproul
a732a87846 Remove TTD flags and safe-slots-to-import-* (#6489)
* Delete SAFE_SLOTS_TO_IMPORT_OPTIMISTICALLY

* Update fork choice tests

* Remove TTD related flags

* Add deprecation warning

* Remove more dead code

* Delete EF on_merge_block tests

* Remove even more dead code

* Address Mac's review comments
2024-10-21 01:28:55 +00:00
Jimmy Chen
ee7fca3ebd Refactor data column reconstruction and avoid blocking processing (#6403)
* Move reconstruction logic out of `overflow_lru_cache` to simplify the code and avoids having to pass `DataColumnsToPublish` around and blocking other processing.

* Publish reconstructed cells before recomputing head. Remove duplicate functions.

* Merge branch 'unstable' into non-blocking-reconstruction

* Merge branch 'unstable' into non-blocking-reconstruction

# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	beacon_node/beacon_chain/src/data_availability_checker.rs
#	beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
#	beacon_node/network/src/network_beacon_processor/sync_methods.rs

* Spawn a blocking task for reconstruction.

* Merge branch 'unstable' into non-blocking-reconstruction

# Conflicts:
#	beacon_node/network/src/network_beacon_processor/mod.rs

* Fix fmt

* Merge branch 'unstable' into non-blocking-reconstruction

# Conflicts:
#	beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs

* Fix race condition by making check and mutation atomic as suggested by Lion. Also added error handling to reconstruction failure.

* Add reconstruction reason metric and more debug logging to da checker.

* Add comment and logging.

* Rename `NotRequired` to `NotStarted`.

* Remove extra character added.
2024-10-17 04:56:25 +00:00
Pawan Dhananjay
83d5c521d7 Electra updates for v1.5.0-alpha.6 (#6445)
* Update process_slashing

* Update test vectors version

* Delete Domain::Consolidation

* Rename to get_max_effective_balance

* Fix unused; lint

* Add the pre-electra slashing processing

* lint

* Change request json types

* Move requests from payload to beacon block body

* Refactor engine api

* Fix warnings

* Update engine api to latest

* engine api changed..again

* yet again

* Merge branch 'engine-requests' into electra-updates

* Fix tests

* Store reference instead of bytes in NewPayloadRequest

* Merge branch 'unstable' into electra-updates

* Update beacon_node/execution_layer/src/engine_api/json_structures.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/execution_layer/src/lib.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/execution_layer/src/test_utils/handle_rpc.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>
2024-10-15 17:38:43 +00:00
Lion - dapplion
71c5388461 Transition block lookup sync to range sync (#6122)
* Transition block lookup sync to range sync

* Log unexpected state

* Merge remote-tracking branch 'sigp/unstable' into lookup-to-range

* Add docs

* Merge remote-tracking branch 'sigp/unstable' into lookup-to-range
2024-10-08 21:18:41 +00:00
Michael Sproul
1bd8f31545 Clean up temporary state flags while running (#6422)
* Clean up temporary state flags while running

* Add regression test

* Simplify
2024-10-07 09:41:52 +00:00
Jimmy Chen
f3a5e256da Implement Subnet Sampling for PeerDAS (#6410)
* Add `SAMPLES_PER_SLOT` config.

* Rename `sampling` module to `peer_sampling`

* Implement subnet sampling.

* Update lookup test.

* Merge branch 'unstable' into subnet-sampling

* Merge branch 'unstable' into subnet-sampling

# Conflicts:
#	beacon_node/beacon_chain/src/data_availability_checker.rs
#	beacon_node/http_api/src/publish_blocks.rs
#	beacon_node/lighthouse_network/src/types/globals.rs
#	beacon_node/network/src/sync/manager.rs

* Merge branch 'unstable' into subnet-sampling
2024-10-04 00:27:30 +00:00
Akihito Nakano
a4a673b780 Output network-test logs into files in CI (#6355)
* Add ci_logger

* Update artifact name

* Add env var

* Add fork_name

* Fix clippy error

* Add comments
2024-10-03 13:53:36 +00:00
Jimmy Chen
17849b58ec Fix invalid data column sidecars getting accepted (#6454)
* Fix invalid data column sidecars getting accepted.

* Update code to match spec function.
2024-10-03 03:06:05 +00:00
Eitan Seri-Levi
82faf975b3 Add {fork_name}_enabled functions (#5951)
* add fork_name_enabled fn to Forkname impl

* refactor codebase to use new fork_enabled fn

* fmt

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into fork-ord-impl

* small code cleanup

* resolve merge conflicts

* fix beacon chain test

* merge conflicts

* fix ef test issue

* resolve merge conflicts
2024-10-03 02:00:52 +00:00
Michael Sproul
2792705331 Lenient duplicate checks on HTTP API for block publication (#5574)
* start splitting gossip verification

* WIP

* Gossip verify separate (#7)

* save

* save

* make ProvenancedBlock concrete

* delete into gossip verified block contents

* get rid of IntoBlobSidecar trait

* remove IntoGossipVerified trait

* get tests compiling

* don't check sidecar slashability in publish

* remove second publish closure

* drop blob bool. also prefer using message index over index of position in list

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Fix low-hanging tests

* Fix tests and clean up

* Clean up imports

* more cleanup

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Further refine behaviour and add tests

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Remove empty line

* Fix test (block is not fully imported just gossip verified)

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Update for unstable & use empty blob list

* Update comment

* Add test for duplicate block case

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Clarify unreachable case

* Fix another publish_block case

* Remove unreachable case in filter chain segment

* Revert unrelated blob optimisation

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Fix merge conflicts

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Fix some compilation issues. Impl is fucked though

* Support peerDAS

* Fix tests

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Fix conflict

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate

* Address review comments

* Merge remote-tracking branch 'origin/unstable' into gossip-verify-separate
2024-09-24 04:52:44 +00:00
Michael Sproul
1447eeb40b Improve single-node testnet support and Arc NetworkConfig/ChainSpec (#6396)
* Arc ChainSpec and NetworkConfig

* Fix release tests

* Fix lint

* Merge remote-tracking branch 'origin/unstable' into single-node-testnet
2024-09-24 00:16:18 +00:00
Lion - dapplion
d84df5799c Attribute invalid column proof error to correct peer (#6377)
* Attribute invalid column proof error to correct peer

* Update beacon_node/beacon_chain/src/data_availability_checker.rs

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>

* fix conflicts
2024-09-23 18:49:26 +00:00
Eitan Seri-Levi
b619f1ab5c Make BeaconChain::kzg field mandatory (#6267)
* make kzg field required

* update todo

* always load trusted setup WIP

* fmt

* use new rust_eth_kzg version

* merge conlficts

* add kzg fn with trusted setup disabled

* as_slice

* add kzg with no precomp

* ignore udep for kzg

* refactor kzg init

* fix peerdas kzg schedule

* fix

* udeps

* uuuudeps

* merge conflict resolved

* merge conflict

* merge conflicts

* resolve TODO

* update

* move kzg to a test util fn

* remove trusted setup default impl

* lint fmt

* fix failing test

* lint

* fix test

* Merge branch 'unstable' into beacon-chain-kzg-field-required
2024-09-23 04:54:32 +00:00
Jimmy Chen
46e0d66e2d Fix deadlock on block cache. (#6412)
* Fix deadlock on block cache.
2024-09-19 04:58:43 +00:00
Daniel Knopik
8b085dd167 Fix phase0 block reward in rewards API (#5101)
* Added Block Rewards

* added new type

* added enum

* Fix phase0 block reward in rewards API (#4929)

* Merge 'guav00a/proposer-rewards-api'

* Merge unstable

* Revamp phase0 reward API tests

- Add test_rewards_base_slashings (testing #5101)
- Improve fix to not include proposer reward in attestation reward API calculation (#4856)
- Adjust test approach for phase0 tests: Pad with empty epochs to include all rewards in calculation
- Simplify and unify code across all reward tests

* Merge branch 'unstable' into fix-4929

* Merge branch 'unstable' into fix-4929

* Merge remote-tracking branch 'origin/unstable' into fix-4929

* Fix merge fallout

* Remove junk revived in merge

* Address review

- check for attestations with lower inclusion delay
- check for double attestations in block
- add test

* Merge branch 'unstable' into fix-4929

* Merge branch 'unstable' into fix-4929
2024-09-17 06:45:02 +00:00
Eitan Seri-Levi
a94b12b4d5 Persist light client bootstrap (#5915)
* persist light client updates

* update beacon chain to serve light client updates

* resolve todos

* cache best update

* extend cache parts

* is better light client update

* resolve merge conflict

* initial api changes

* add lc update db column

* fmt

* added tests

* add sim

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* fix some weird issues with the simulator

* tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* test changes

* merge conflict

* testing

* started work on ef tests and some code clean up

* update tests

* linting

* noop pre altair, were still failing on electra though

* allow for zeroed light client header

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* merge unstable

* remove unwraps

* remove unwraps

* fetch bootstrap without always querying for state

* storing bootstrap parts in db

* mroe code cleanup

* test

* prune sync committee branches from dropped chains

* Update light_client_update.rs

* merge unstable

* move functionality to helper methods

* refactor is best update fn

* refactor is best update fn

* improve organization of light client server cache logic

* fork diget calc, and only spawn as many blcoks as we need for the lc update test

* resovle merge conflict

* add electra bootstrap logic, add logic to cache current sync committee

* add latest sync committe branch cache

* fetch lc update from the cache if it exists

* fmt

* Fix beacon_chain tests

* Add debug code to update ranking_order ef test

* Fix compare code

* merge conflicts

* merge conflict

* add better error messaging

* resolve merge conflicts

* remove lc update from basicsim

* rename sync comittte variable and fix persist condition

* refactor get_light_client_update logic

* add better comments, return helpful error messages over http and rpc

* pruning canonical non checkpoint slots

* fix test

* rerun test

* update pruning logic, add tests

* fix tests

* fix imports

* fmt

* refactor db code

* Refactor db method

* Refactor db method

* add additional comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-bootstrap

* fix merge

* linting

* merge conflict

* prevent overflow

* enable lc server for http api tests

* fix tests

* remove prints

* remove warning

* revert change
2024-09-10 00:27:49 +00:00
Jimmy Chen
815567a91a Add more granularity to data column proof computation metric (#6371)
* Add more granularity to data column proof computation metric to capture more variations between 0.25 and 1 second.
2024-09-09 11:02:16 +00:00
Michael Sproul
d6861380a2 Delete legacy payload reconstruction (#6213)
* Delete legacy payload reconstruction

* Delete unneeded failing test

* Merge remote-tracking branch 'origin/unstable' into remove-more-ethers

* Merge remote-tracking branch 'origin/unstable' into remove-more-ethers

* Cleanups
2024-09-06 08:10:55 +00:00
Michael Sproul
c824142a6d Fix lints for Rust 1.81 (#6363)
* Fix lints for Rust 1.81
2024-09-06 01:45:34 +00:00
Lion - dapplion
369807becc Check known parent on rpc blob process (#5893)
* Check known parent on rpc blob process

* fix test

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into blob-unknown-parent
2024-09-05 15:24:21 +00:00
Michael Sproul
0e94fe1aa8 Clarify validator monitor block log (#6342)
* Clarify validator monitor block log

* Merge branch 'unstable' into clarify-block-log
2024-09-05 05:39:05 +00:00
Jimmy Chen
b50ce60662 Add blob count label to DATA_COLUMN_SIDECAR_COMPUTATION metric (#6340)
* Add blob count label to `DATA_COLUMN_SIDECAR_COMPUTATION` metric, and move metrics into the compute function, recording only successful computation.

* Move `discard_timer_on_break` usage to caller site.

* Merge branch 'unstable' into compute-data-column-metric

* Merge branch 'unstable' into compute-data-column-metric
2024-09-05 05:39:00 +00:00
Michael Sproul
26c19d65a3 Enable large_stack_frames lint (#6343)
* Enable `large_stack_frames` lint
2024-09-05 05:01:16 +00:00
Lion - dapplion
ed7cd3bf47 Drop block data from BlockError and BlobError (#5735)
* Drop block data from BlockError and BlobError

* Debug release tests

* Fix release tests

* Revert unnecessary changes to lighthouse_metrics
2024-09-03 01:07:07 +00:00
Eitan Seri-Levi
99e53b88c3 Migrate from ethereum-types to alloy-primitives (#6078)
* Remove use of ethers_core::RlpStream

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* Remove old code

* Simplify keccak call

* Remove unused package

* Merge branch 'unstable' of https://github.com/ethDreamer/lighthouse into remove_use_of_ethers_core

* Merge branch 'unstable' into remove_use_of_ethers_core

* Run clippy

* Merge branch 'remove_use_of_ethers_core' of https://github.com/dospore/lighthouse into remove_use_of_ethers_core

* Check all cargo fmt

* migrate to alloy primitives init

* fix deps

* integrate alloy-primitives

* resolve dep issues

* more changes based on dep changes

* add TODOs

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* Revert lock

* Add BeaconBlocksByRange v3

* continue migration

* Revert "Add BeaconBlocksByRange v3"

This reverts commit e3ce7fc5ea.

* impl hash256 extended trait

* revert some uneeded diffs

* merge conflict resolved

* fix subnet id rshift calc

* rename to FixedBytesExtended

* debugging

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fix failed test

* fixing more tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into remove_use_of_ethers_core

* introduce a shim to convert between the two u256 types

* move alloy to wrokspace

* align alloy versions

* update

* update web3signer test certs

* refactor

* resolve failing tests

* linting

* fix graffiti string test

* fmt

* fix ef test

* resolve merge conflicts

* remove udep and revert cert

* cargo patch

* cyclic dep

* fix build error

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* resolve conflicts, update deps

* merge unstable

* fmt

* fix deps

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* resolve merge conflicts

* resolve conflicts, make necessary changes

* Remove patch

* fmt

* remove file

* merge conflicts

* sneaking in a smol change

* bump versions

* Merge remote-tracking branch 'origin/unstable' into migrate-to-alloy-primitives

* Updates for peerDAS

* Update ethereum_hashing to prevent dupe

* updated alloy-consensus, removed TODOs

* cargo update

* endianess fix

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fmt

* fix merge

* fix test

* fixed_bytes crate

* minor fixes

* convert u256 to i64

* panic free mixin to_low_u64_le

* from_str_radix

* computbe_subnet api and ensuring we use big-endian

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into migrate-to-alloy-primitives

* fix test

* Simplify subnet_id test

* Simplify some more tests

* Add tests to fixed_bytes crate

* Merge branch 'unstable' into migrate-to-alloy-primitives
2024-09-02 08:03:24 +00:00