Commit Graph

747 Commits

Author SHA1 Message Date
Daniel Knopik
12eb7d39f8 Squashed decouple-eth2 2025-03-13 14:13:10 +01:00
Daniel Knopik
3316681ea5 Squashed changes from modularize-validator-store 2025-03-13 13:07:39 +01:00
Daniel Knopik
ac53bee0fd Merge remote-tracking branch 'origin/release-v7.0.0' into into-anchor 2025-03-13 12:54:35 +01:00
Daniel Knopik
f61f0b654c Squashed reset to unstable 2025-03-13 12:51:19 +01:00
Eitan Seri-Levi
3a555f571f Address cargo audit failure RUSTSEC-2024-0437 (#7114)
Resolves #7091


  The `prometheus` crate pulls in `protobuf 2.x` which fails cargo audit. We actually dont use any `protobuf` related features in LH. By disabling default features for `prometheus`, we no longer pull in the `protobuf` crate
2025-03-13 03:17:33 +00:00
Eitan Seri-Levi
0f5e680149 Address cargo audit failure RUSTSEC-2025-0009 (#7086) 2025-03-10 23:58:58 +00:00
Michael Sproul
1888be554c Release v7.0.0-beta.0 (#6962)
New release for Electra on Holesky and Sepolia.

Includes PRs:

- https://github.com/sigp/lighthouse/pull/6808
- https://github.com/sigp/lighthouse/pull/6914
- https://github.com/sigp/lighthouse/pull/6949
- https://github.com/sigp/lighthouse/pull/6950
- https://github.com/sigp/lighthouse/pull/6958
2025-02-13 03:06:20 +00:00
Jimmy Chen
0728140086 Address cargo audit failure RUSTSEC-2025-0006 (#6972)
PR to fix cargo audit failure and bump `hickory-proto` version:
https://github.com/sigp/lighthouse/actions/runs/13252660830/job/36993664960
2025-02-11 04:17:02 +00:00
Daniel Knopik
0ee83716aa Squashed changed from modularized-validator-store 2025-02-10 08:49:41 +01:00
Daniel Knopik
b83ffb2131 Merge branch 'unstable' into into-anchor 2025-02-10 08:41:20 +01:00
Age Manning
62a0f25f97 IPv6 By Default (#6808) 2025-02-10 01:58:11 +00:00
chonghe
3d06bc26d1 Add test to beacon node fallback feature (#6568) 2025-02-04 06:43:37 +00:00
kevaundray
df131b2a6c chore: update peerDAS KZG library to 0.5.3 (#6906)
This optimizes the time it takes to load the context, so that tests do not time out
2025-02-04 00:47:11 +00:00
Eitan Seri-Levi
7e4b27c922 Migrate validator client to clap derive (#6300)
Partially #5900


  Migrate the validator client cli to clap derive
2025-02-03 20:08:31 +00:00
Eitan Seri-Levi
1e2b547b35 Add builder SSZ flow (#6859) 2025-02-03 06:07:42 +00:00
Michael Sproul
6b40b98537 Cargo update for openssl vuln (#6901)
Run `cargo update` to address [RUSTSEC-2025-0004](https://rustsec.org/advisories/RUSTSEC-2025-0004), a vulnerability in `openssl`. I don't think we are affected, but this PR is required for us to pass `cargo audit` and unblock CI.
2025-02-03 04:27:07 +00:00
João Oliveira
ddb845d503 update libp2p to 0.55 (#6889)
Updates libp2p to `0.55`.
Will address the deprecations in a subsequent PR
2025-01-31 12:21:06 +00:00
Eitan Seri-Levi
276eda3dfe POST /eth/v2/beacon/pool/attestations bugfixes (#6867) 2025-01-31 00:20:44 +00:00
Michael Sproul
d47b3e3e43 Cargo update without rust_eth_kzg (#6848)
Update cargo dependencies while keeping `rust_eth_kzg` pinned to `0.5.1` due to the regression described in:

- https://github.com/sigp/lighthouse/pull/6608

The changes from that PR were not sufficient to actually pin the dependencies of `rust_eth_kzg`, because the dependencies from the workspace Cargo.toml file were not being used anywhere. To fix this, I've added them as explicit dependencies in `crypto/kzg/Cargo.toml`. With this change, `cargo update` no longer tries to update them.
2025-01-30 08:02:04 +00:00
Eitan Seri-Levi
a1b7d616b4 Modularize beacon node backend (#4718)
#4669


  Modularize the beacon node backend to make it easier to add new database implementations
2025-01-23 02:12:16 +00:00
Age Manning
54e37096b6 Update discv5 (#6836)
* Update discv5 dep

* Handle yanked crates
2025-01-22 12:29:56 +00:00
Daniel Knopik
fb95134bd0 Merge branch 'expose-blst-internals' into into-anchor
# Conflicts:
#	Cargo.lock
#	Cargo.toml
#	account_manager/src/validator/create.rs
#	account_manager/src/validator/recover.rs
#	beacon_node/http_api/src/block_id.rs
#	beacon_node/http_api/src/lib.rs
#	consensus/types/src/payload.rs
#	validator_client/doppelganger_service/src/lib.rs
#	validator_client/http_metrics/Cargo.toml
#	validator_client/validator_services/src/preparation_service.rs
2025-01-21 17:03:38 +01:00
Eitan Seri-Levi
06329ec2d1 SingleAttestation implementation (#6488)
* First pass

* Add restrictions to RuntimeVariableList api

* Use empty_uninitialized and fix warnings

* Fix some todos

* Merge branch 'unstable' into max-blobs-preset

* Fix take impl on RuntimeFixedList

* cleanup

* Fix test compilations

* Fix some more tests

* Fix test from unstable

* Merge branch 'unstable' into max-blobs-preset

* SingleAttestation

* Add post attestation v2 endpoint logic to attestation service

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_attestation

* Implement "Bugfix and more withdrawal tests"

* Implement "Add missed exit checks to consolidation processing"

* Implement "Update initial earliest_exit_epoch calculation"

* Implement "Limit consolidating balance by validator.effective_balance"

* Implement "Use 16-bit random value in validator filter"

* Implement "Do not change creds type on consolidation"

* some tests and fixed attestqtion calc

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_attestation

* Rename PendingPartialWithdraw index field to validator_index

* Skip slots to get test to pass and add TODO

* Implement "Synchronously check all transactions to have non-zero length"

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

* Remove footgun function

* Minor simplifications

* Move from preset to config

* Fix typo

* Revert "Remove footgun function"

This reverts commit de01f923c7.

* Try fixing tests

* Implement "bump minimal preset MAX_BLOB_COMMITMENTS_PER_BLOCK and KZG_COMMITMENT_INCLUSION_PROOF_DEPTH"

* Thread through ChainSpec

* Fix release tests

* Move RuntimeFixedVector into module and rename

* Add test

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_attestation

* Added more test coverage, simplified Attestation conversion, and other minor refactors

* Removed unusued codepaths

* Fix failing test

* Implement "Remove post-altair `initialize_beacon_state_from_eth1` from specs"

* Update preset YAML

* Remove empty RuntimeVarList awefullness

* Make max_blobs_per_block a config parameter (#6329)

Squashed commit of the following:

commit 04b3743ec1
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 17:36:58 2025 +1100

    Add test

commit 440e854199
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 17:24:50 2025 +1100

    Move RuntimeFixedVector into module and rename

commit f66e179a40
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 17:17:17 2025 +1100

    Fix release tests

commit e4bfe71cd1
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 17:05:30 2025 +1100

    Thread through ChainSpec

commit 063b79c16a
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 15:32:16 2025 +1100

    Try fixing tests

commit 88bedf09bc
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 15:04:37 2025 +1100

    Revert "Remove footgun function"

    This reverts commit de01f923c7.

commit 32483d385b
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 15:04:32 2025 +1100

    Fix typo

commit 2e86585b47
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 15:04:15 2025 +1100

    Move from preset to config

commit 1095d60a40
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 14:38:40 2025 +1100

    Minor simplifications

commit de01f923c7
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 14:06:57 2025 +1100

    Remove footgun function

commit 0c2c8c4224
Merge: 21ecb58ff f51a292f7
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 14:02:50 2025 +1100

    Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

commit f51a292f77
Author: Daniel Knopik <107140945+dknopik@users.noreply.github.com>
Date:   Fri Jan 3 20:27:21 2025 +0100

    fully lint only explicitly to avoid unnecessary rebuilds (#6753)

    * fully lint only explicitly to avoid unnecessary rebuilds

commit 7e0cddef32
Author: Akihito Nakano <sora.akatsuki@gmail.com>
Date:   Tue Dec 24 10:38:56 2024 +0900

    Make sure we have fanout peers when publish (#6738)

    * Ensure that `fanout_peers` is always non-empty if it's `Some`

commit 21ecb58ff8
Merge: 2fcb2935e 9aefb5539
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Mon Oct 21 14:46:00 2024 -0700

    Merge branch 'unstable' into max-blobs-preset

commit 2fcb2935ec
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Sep 6 18:28:31 2024 -0700

    Fix test from unstable

commit 12c6ef118a
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Wed Sep 4 16:16:36 2024 -0700

    Fix some more tests

commit d37733b846
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Wed Sep 4 12:47:36 2024 -0700

    Fix test compilations

commit 52bb581e07
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Tue Sep 3 18:38:19 2024 -0700

    cleanup

commit e71020e3e6
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Tue Sep 3 17:16:10 2024 -0700

    Fix take impl on RuntimeFixedList

commit 13f9bba647
Merge: 60100fc6b 4e675cf5d
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Tue Sep 3 16:08:59 2024 -0700

    Merge branch 'unstable' into max-blobs-preset

commit 60100fc6be
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Aug 30 16:04:11 2024 -0700

    Fix some todos

commit a9cb329a22
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Aug 30 15:54:00 2024 -0700

    Use empty_uninitialized and fix warnings

commit 4dc6e6515e
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Aug 30 15:53:18 2024 -0700

    Add restrictions to RuntimeVariableList api

commit 25feedfde3
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Thu Aug 29 16:11:19 2024 -0700

    First pass

* Fix tests

* Implement max_blobs_per_block_electra

* Fix config issues

* Simplify BlobSidecarListFromRoot

* Disable PeerDAS tests

* Cleanup single attestation imports

* Fix some single attestation network plumbing

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

* Bump quota to account for new target (6)

* Remove clone

* Fix issue from review

* Try to remove ugliness

* Merge branch 'unstable' into max-blobs-preset

* Merge remote-tracking branch 'origin/unstable' into electra-alpha10

* Merge commit '04b3743ec1e0b650269dd8e58b540c02430d1c0d' into electra-alpha10

* Merge remote-tracking branch 'pawan/max-blobs-preset' into electra-alpha10

* Update tests to v1.5.0-beta.0

* Merge remote-tracking branch 'origin/electra-alpha10' into single_attestation

* Fix some tests

* Cargo fmt

* lint

* fmt

* Resolve merge conflicts

* Merge branch 'electra-alpha10' of https://github.com/sigp/lighthouse into single_attestation

* lint

* Linting

* fmt

* Merge branch 'electra-alpha10' of https://github.com/sigp/lighthouse into single_attestation

* Fmt

* Fix test and add TODO

* Gracefully handle slashed proposers in fork choice tests

* Merge remote-tracking branch 'origin/unstable' into electra-alpha10

* Keep latest changes from max_blobs_per_block PR in codec.rs

* Revert a few more regressions and add a comment

* Merge branch 'electra-alpha10' of https://github.com/sigp/lighthouse into single_attestation

* Disable more DAS tests

* Improve validator monitor test a little

* Make test more robust

* Fix sync test that didn't understand blobs

* Fill out cropped comment

* Merge remote-tracking branch 'origin/electra-alpha10' into single_attestation

* Merge remote-tracking branch 'origin/unstable' into single_attestation

* Merge remote-tracking branch 'origin/unstable' into single_attestation

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into single_attestation

* publish_attestations should accept Either<Attestation,SingleAttestation>

* log an error when failing to convert to SingleAttestation

* Use Cow to avoid clone

* Avoid reconverting to SingleAttestation

* Tweak VC error message

* update comments

* update comments

* pass in single attestation as ref to subnetid calculation method

* Improved API, new error variants and other minor tweaks

* Fix single_attestation event topic boilerplate

* fix sse event failure

* Add single_attestation event topic test coverage
2025-01-16 18:27:08 +00:00
Daniel Knopik
669932aa67 Misc. dependency cleanup (#6810)
* remove ensure_dir_exists (2 deps saved)

* group UNHANDLED_ERRORs into a generic (2 deps saved)

* Introduce separate `health_metrics` crate

* separate health_metrics crate

* remove metrics from warp_utils

* move ProcessHealth::observe and SystemHealth::observe to health_metrics

* fix errors

* nitpick `Cargo.toml`s

---------

Co-authored-by: Daniel Knopik <daniel@dknopik.de>
# Conflicts:
#	Cargo.toml
2025-01-16 01:48:50 +00:00
Daniel Knopik
997991f551 Anchor pre-PR: Dependency cleanup (#6769)
* remove ensure_dir_exists (2 deps saved)

* group UNHANDLED_ERRORs into a generic (2 deps saved)

* Introduce separate `health_metrics` crate

* separate health_metrics crate

* remove metrics from warp_utils

* move ProcessHealth::observe and SystemHealth::observe to health_metrics

* fix errors

* nitpick `Cargo.toml`s

---------

Co-authored-by: Daniel Knopik <daniel@dknopik.de>

---------

Co-authored-by: ThreeHrSleep <151536303+ThreeHrSleep@users.noreply.github.com>
2025-01-10 00:06:14 +11:00
Daniel Knopik
64eb84800a Anchor pre-PR: Modularize validator store (#6771)
* pass slots_per_epoch at runtime

* remove generic E from unrequired types

* move `validator_store` to `lighthouse_validator_store`

* make validator_store into a trait

* further reduce dependencies

* remove `environment` dependency on `beacon_node_fallback`

* Manually pull in some changes from tracing-integration (thanks sayan!)

Co-authored-by: ThreeHrSleep <threehrsleep@gmail.com>

* remove `environment` from `validator_services`

* unify boost factor accessors

* add builder for DutiesService

* Manually merge tracing PR for beacon_node_fallback

Co-authored-by: ThreeHrSleep <threehrsleep@gmail.com>

* Fix chain_spec for BlockService

* address review

* remove PhantomData from SyncDutiesMap

* fix tests

* correct test

* Add `E` to `ValidatorStore` as associated type

* fix tests

* derive Clone for ValidatorStore's Error and required sub-errors

* switch to enum for block signing to allow differing types

---------

Co-authored-by: João Oliveira <hello@jxs.pt>
Co-authored-by: ThreeHrSleep <threehrsleep@gmail.com>
Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
2025-01-09 23:56:29 +11:00
Daniel Knopik
9a4768a771 Anchor pre-PR: Decouple eth2 (#6770)
* decouple `eth2` from `store` and `lighthouse_network`

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>

* remove unused dependency

---------

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Jimmy Chen <jimmy@sigmaprime.io>
2025-01-09 23:48:18 +11:00
Pawan Dhananjay
2662dc7f8f Fix Sse client api (#6685)
* Use reqwest eventsource for get_events api

* await for Event::Open before returning stream

* fmt

* Merge branch 'unstable' into sse-client-fix

* Ignore lint
2024-12-18 00:05:58 +00:00
Jun Song
11e1d5bf14 Add CLI flag for HTTP API token path (VC) (#6577)
* Add cli flag for HTTP API token path (VC)

* Add http_token_path_flag test

* Add pre-check for directory case & Fix test utils

* Update docs

* Apply review: move http_token_path into validator_http_api config

* Lint

* Make diff lesser to replace PK_FILENAME

* Merge branch 'unstable' into feature/cli-token-path

* Applt review: help_vc.md

Co-authored-by: chonghe <44791194+chong-he@users.noreply.github.com>

* Fix help for cli

* Fix issues on ci

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path
2024-12-16 05:43:54 +00:00
Michael Sproul
a6de0d5e12 Merge remote-tracking branch 'origin/stable' into back-merge-6.0.1 2024-12-16 15:37:52 +11:00
Michael Sproul
0d90135047 Release v6.0.1 (#6659)
* Release v6.0.1
2024-12-16 03:03:22 +00:00
Daniel Knopik
a2b00090fd Remove ZeroizeString in favour of Zeroizing<String> (#6661)
* Remove ZeroizeString in favour of Zeroizing<String>

* cargo fmt

* remove unrelated line that slipped in

* Update beacon_node/store/Cargo.toml

thanks michael!

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge branch 'unstable' into remove-zeroizedstring
2024-12-11 23:51:20 +00:00
hopinheimer
fec502db9f Deprecate futures ticker (#6630)
* deprecate futures-ticker

* Merge branch 'unstable' of github.com:sigp/lighthouse into deprecate-futures-timer

* Merge branch 'unstable' into deprecate-futures-timer

* making the linter happy

* remove unrequired #[allow(unused_imports)]

* fixing minor issues

* merge commit

* minor fix

* clippy changes
2024-12-03 08:56:43 +00:00
Michael Sproul
c042dc14d7 Release v6.0.0 (#6605)
* Release v6.0.0
2024-12-02 04:00:51 +00:00
Eitan Seri-Levi
38f5f665e1 Remove error-chain dependency (#6628)
* remove error-chain dependency

* rerun CI

* rerun CI
2024-11-28 02:39:50 +00:00
Age Manning
08e8b92e50 Simple Subnet Management (#6146)
* Initial temp commit

* Merge latest unstable

* First draft without tests

* Update tests for new version

* Correct comments and reviewers comments

* Merge latest unstable

* Fix errors

* Missed a comment, corrected it

* Fix lints

* Merge latest unstable

* Fix tests

* Merge latest unstable

* Reviewers comments

* Remove sync subnets from ENR on unsubscribe

* Merge branch 'unstable' into simple-peer-mapping

* Merge branch 'unstable' into simple-peer-mapping

* Merge branch 'unstable' into simple-peer-mapping

* Merge latest unstable

* Prevent clash with pin of rust_eth_kzg
2024-11-26 01:48:07 +00:00
Age Manning
8cebc87d95 Update to latest discovery (#6486)
* Upgrade discv5 to v0.8

* Rename some logs

* Improve the NAT reporting with new discv5 metrics

* Merge branch 'unstable' into discv5-v8

* Limited Cargo.lock update

* Update yanked futures-* crates
2024-11-18 22:52:23 +00:00
Michael Sproul
9fdd53df56 Hierarchical state diffs (#5978)
* Start extracting freezer changes for tree-states

* Remove unused config args

* Add comments

* Remove unwraps

* Subjective more clear implementation

* Clean up hdiff

* Update xdelta3

* Tree states archive metrics (#6040)

* Add store cache size metrics

* Add compress timer metrics

* Add diff apply compute timer metrics

* Add diff buffer cache hit metrics

* Add hdiff buffer load times

* Add blocks replayed metric

* Move metrics to store

* Future proof some metrics

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Port and clean up forwards iterator changes

* Add and polish hierarchy-config flag

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Cleaner errors

* Fix beacon_chain test compilation

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Patch a few more freezer block roots

* Fix genesis block root bug

* Fix test failing due to pending updates

* Beacon chain tests passing

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix doc lint

* Implement DB schema upgrade for hierarchical state diffs (#6193)

* DB upgrade

* Add flag

* Delete RestorePointHash

* Update docs

* Update docs

* Implement hierarchical state diffs config migration (#6245)

* Implement hierarchical state diffs config migration

* Review PR

* Remove TODO

* Set CURRENT_SCHEMA_VERSION correctly

* Fix genesis state loading

* Re-delete some PartialBeaconState stuff

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix test compilation

* Update schema downgrade test

* Fix tests

* Fix null anchor migration

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix tree states upgrade migration (#6328)

* Towards crash safety

* Fix compilation

* Move cold summaries and state roots to new columns

* Rename StateRoots chunked field

* Update prune states

* Clean hdiff CLI flag and metrics

* Fix "staged reconstruction"

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix alloy issues

* Fix staged reconstruction logic

* Prevent weird slot drift

* Remove "allow" flag

* Update CLI help

* Remove FIXME about downgrade

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Remove some unnecessary error variants

* Fix new test

* Tree states archive - review comments and metrics (#6386)

* Review PR comments and metrics

* Comments

* Add anchor metrics

* drop prev comment

* Update metadata.rs

* Apply suggestions from code review

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/store/src/hot_cold_store.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Clarify comment and remove anchor_slot garbage

* Simplify database anchor (#6397)

* Simplify database anchor

* Update beacon_node/store/src/reconstruct.rs

* Add migration for anchor

* Fix and simplify light_client store tests

* Fix incompatible config test

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* More metrics

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* New historic state cache (#6475)

* New historic state cache

* Add more metrics

* State cache hit rate metrics

* Fix store metrics

* More logs and metrics

* Fix logger

* Ensure cached states have built caches :O

* Replay blocks in preference to diffing

* Two separate caches

* Distribute cache build time to next slot

* Re-plumb historic-state-cache flag

* Clean up metrics

* Update book

* Update beacon_node/store/src/hdiff.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update beacon_node/store/src/historic_state_cache.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update database docs

* Update diagram

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Update lockbud to work with bindgen/etc

* Correct pkg name for Debian

* Remove vestigial epochs_per_state_diff

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Markdown lint

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Address Jimmy's review comments

* Simplify ReplayFrom case

* Fix and document genesis_state_root

* Typo

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Merge branch 'unstable' into tree-states-archive

* Compute diff of validators list manually (#6556)

* Split hdiff computation

* Dedicated logic for historical roots and summaries

* Benchmark against real states

* Mutated source?

* Version the hdiff

* Add lighthouse DB config for hierarchy exponents

* Tidy up hierarchy exponents flag

* Apply suggestions from code review

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Address PR review

* Remove hardcoded paths in benchmarks

* Delete unused function in benches

* lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Test hdiff binary format stability (#6585)

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Add deprecation warning for SPRP

* Update xdelta to get rid of duplicate deps

* Document test
2024-11-18 01:51:44 +00:00
Age Manning
8e95024945 Split the VC into crates making it more modular (#6453)
* Starting to modularize the VC

* Revert changes to eth2

* More progress

* More progress

* Compiles

* Merge latest unstable and make it compile

* Fix some lints

* Tests compile

* Merge latest unstable

* Remove unnecessary deps

* Merge latest unstable

* Correct release tests

* Merge latest unstable

* Merge remote-tracking branch 'origin/unstable' into modularize-vc

* Merge branch 'unstable' into modularize-vc

* Revert unnecessary cargo lock changes

* Update validator_client/beacon_node_fallback/Cargo.toml

* Update validator_client/http_metrics/Cargo.toml

* Update validator_client/http_metrics/src/lib.rs

* Update validator_client/initialized_validators/Cargo.toml

* Update validator_client/signing_method/Cargo.toml

* Update validator_client/validator_metrics/Cargo.toml

* Update validator_client/validator_services/Cargo.toml

* Update validator_client/validator_store/Cargo.toml

* Update validator_client/validator_store/src/lib.rs

* Merge remote-tracking branch 'origin/unstable' into modularize-vc

* Fix format string

* Rename doppelganger trait

* Don't drop the tempdir

* Cargo fmt
2024-11-08 01:01:46 +00:00
chonghe
fdf456f398 Validator manager commands for the Keymanager APIs (#6261)
* Validator manager commands for standard key-manager APIs

* Merge latest unstable

* Fix Some in lib.rs

* Replace Arg::with_name with Arg::new

* Update takes_value

* Remove clap::App

* Change App to Command

* Add command in use

* Remove generic in ArgMatches

* Fix matches.get_flag

* Fixes

* fix error handling

* SetTrue in import

* Fix

* Fix builder-proposal flag (will delete the flag later)

* Minor fix

* Fix prefer_builder_proposals

* Remove unwrap

* Error handling from Michael

* Add cli help text

* Use None in import to simplify

* Delete unwrap

* Revert flags option

* Simplify help command code

* Remove flag header in move

* Merge remote-tracking branch 'origin/unstable' into pahor/validator-manager-standard-keystore

* Add log in VC when keystore is deleted

* Delete duplicated log when validator does not exist

* Simplify log code

* Rename remove to delete

* cargo-fmt

* Try to remove a function

* make-cli

* Error handling

* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm

* Update CLI hel text

* make-cli

* Fix checks

* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm

* Try to fix check errors

* Fix test

* Remove changes

* Update flag name

* CLI display order

* Move builde_proposals flag

* Add doc

* mdlint

* Update validator_manager/src/list_validators.rs

Co-authored-by: Mac L <mjladson@pm.me>

* Delete empty line

* Fix list

* Simplify delete

* Add support to delete more validators

* Fix test

* Rename response

* Add (s)

* Add test to delete multiple validators

* Book and cli

* Make cli

* Only log when keystore is deleted

* Revise deletion log

* Add validator pubkey to error message

* Merge import

* Thank you Mac

* Test

* Add flags

* Error handling for password

* make cli

* Merge remote-tracking branch 'origin/unstable' into vm

* make cli

* Fix test

* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm

* Fix test

* vm test

* Debug trait thank you Michael

* Fix test

* Merge branch 'unstable' into vm

* test

* testing

* Combine import validator(s)

* make cli

* Add requires

* Update book

* mdlint

* Only show import log when import is successful

* delete testing

* Test for standard format

* Test standard format

* Test

* fix builder_proposals flag

* Fix test for standard format

* Add requires

* Fix vm test

* make cli

* Remove flag header

* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm

* make cli

* Delete space

* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm

* Merge branch 'unstable' into vm

* Rename delete_validator to delete_validators

* Rearrange

* Remove pub in run function

* Fix grammar

* Apply suggestions from code review

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Remove description

* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm

* Close bracket

* make cli

* Revise list code and test

* Revise import flag

* make cli

* Comment out test

* Update vm test

* Simplify

* Merge remote-tracking branch 'origin/unstable' into vm

* make cli

* Add test

* Add password as a requirement for keystore file

* Correct flags in docs

* typo
2024-10-29 04:14:06 +00:00
Age Manning
e31ac508d4 Modularize tracing executor and metrics rename (#6424)
* Tracing executor and metrics rename

* Appease clippy

* Merge branch 'unstable' into modularise-task-executor
2024-10-28 09:41:45 +00:00
Eitan Seri-Levi
7091c789c9 Optimization to Redb slasher (#6481)
* first() & last() optimizations

* Fmt
2024-10-17 05:53:31 +00:00
Pawan Dhananjay
83d5c521d7 Electra updates for v1.5.0-alpha.6 (#6445)
* Update process_slashing

* Update test vectors version

* Delete Domain::Consolidation

* Rename to get_max_effective_balance

* Fix unused; lint

* Add the pre-electra slashing processing

* lint

* Change request json types

* Move requests from payload to beacon block body

* Refactor engine api

* Fix warnings

* Update engine api to latest

* engine api changed..again

* yet again

* Merge branch 'engine-requests' into electra-updates

* Fix tests

* Store reference instead of bytes in NewPayloadRequest

* Merge branch 'unstable' into electra-updates

* Update beacon_node/execution_layer/src/engine_api/json_structures.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/execution_layer/src/lib.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/execution_layer/src/test_utils/handle_rpc.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>
2024-10-15 17:38:43 +00:00
Akihito Nakano
8cf686f5c1 Add test for ActiveSamplingRequest (#6307)
* Add test for ActiveSamplingRequest

* Fix the column_indexes field from the requested ones to the responded ones

* Fix clippy errors

* Move tests to tests.rs

* Fix unused import

* Fix clippy error

* Merge branch 'unstable' into fork/add-test-for-active-sampling-request

# Conflicts:
#	beacon_node/network/Cargo.toml
#	beacon_node/network/src/sync/sampling.rs

* Merge branch 'unstable' into fork/add-test-for-active-sampling-request
2024-10-04 03:00:32 +00:00
Mac L
f870b66f49 Rework Validator Client fallback mechanism (#4393)
* Rework Validator Client fallback mechanism

* Add CI workflow for fallback simulator

* Tie-break with sync distance for non-synced nodes

* Fix simulator

* Cleanup unused code

* More improvements

* Add IsOptimistic enum for readability

* Use configurable sync distance tiers

* Fix tests

* Combine status and health and improve logging

* Fix nodes not being marked as available

* Fix simulator

* Fix tests again

* Increase fallback simulator tolerance

* Add http api endpoint

* Fix todos and tests

* Update simulator

* Merge branch 'unstable' into vc-fallback

* Add suggestions

* Add id to ui endpoint

* Remove unnecessary clones

* Formatting

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Fix flag tests

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Fix conflicts

* Merge branch 'unstable' into vc-fallback

* Remove unnecessary pubs

* Simplify `compute_distance_tier` and reduce notifier awaits

* Use the more descriptive `user_index` instead of `id`

* Combine sync distance tolerance flags into one

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* wip

* Use new simulator from unstable

* Fix cli text

* Remove leftover files

* Remove old commented code

* Merge branch 'unstable' into vc-fallback

* Update cli text

* Silence candidate errors when pre-genesis

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Retry on failure

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Remove disable_run_on_all

* Remove unused error variant

* Fix out of date comment

* Merge branch 'unstable' into vc-fallback

* Remove unnecessary as_u64

* Remove more out of date comments

* Use tokio RwLock and remove parking_lot

* Merge branch 'unstable' into vc-fallback

* Formatting

* Ensure nodes are still added to total when not available

* Allow VC to detect when BN comes online

* Fix ui endpoint

* Don't have block_service as an Option

* Merge branch 'unstable' into vc-fallback

* Clean up lifetimes and futures

* Revert "Don't have block_service as an Option"

This reverts commit b5445a09e9.

* Merge branch 'unstable' into vc-fallback

* Merge branch 'unstable' into vc-fallback

* Improve rwlock sanitation using clones

* Merge branch 'unstable' into vc-fallback

* Drop read lock immediately by cloning the vec.
2024-10-03 05:57:12 +00:00
Eitan Seri-Levi
b619f1ab5c Make BeaconChain::kzg field mandatory (#6267)
* make kzg field required

* update todo

* always load trusted setup WIP

* fmt

* use new rust_eth_kzg version

* merge conlficts

* add kzg fn with trusted setup disabled

* as_slice

* add kzg with no precomp

* ignore udep for kzg

* refactor kzg init

* fix peerdas kzg schedule

* fix

* udeps

* uuuudeps

* merge conflict resolved

* merge conflict

* merge conflicts

* resolve TODO

* update

* move kzg to a test util fn

* remove trusted setup default impl

* lint fmt

* fix failing test

* lint

* fix test

* Merge branch 'unstable' into beacon-chain-kzg-field-required
2024-09-23 04:54:32 +00:00
Age Manning
a97d77c147 Correct ENR decoding on extension trait (#6402)
* Correct enr extension encodings

* Clippy my ol friend

* Correct all encoding and comparisons

* Found some more encodings

* Fix remaining tests
2024-09-20 12:14:57 +00:00
Age Manning
e5a40fb73b Upgrade to discv5 0.7.0 (#6385)
* Upgrade to discv5 v0.7.0
2024-09-12 00:26:20 +00:00
Eitan Seri-Levi
a94b12b4d5 Persist light client bootstrap (#5915)
* persist light client updates

* update beacon chain to serve light client updates

* resolve todos

* cache best update

* extend cache parts

* is better light client update

* resolve merge conflict

* initial api changes

* add lc update db column

* fmt

* added tests

* add sim

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* fix some weird issues with the simulator

* tests

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* test changes

* merge conflict

* testing

* started work on ef tests and some code clean up

* update tests

* linting

* noop pre altair, were still failing on electra though

* allow for zeroed light client header

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-updates

* merge unstable

* remove unwraps

* remove unwraps

* fetch bootstrap without always querying for state

* storing bootstrap parts in db

* mroe code cleanup

* test

* prune sync committee branches from dropped chains

* Update light_client_update.rs

* merge unstable

* move functionality to helper methods

* refactor is best update fn

* refactor is best update fn

* improve organization of light client server cache logic

* fork diget calc, and only spawn as many blcoks as we need for the lc update test

* resovle merge conflict

* add electra bootstrap logic, add logic to cache current sync committee

* add latest sync committe branch cache

* fetch lc update from the cache if it exists

* fmt

* Fix beacon_chain tests

* Add debug code to update ranking_order ef test

* Fix compare code

* merge conflicts

* merge conflict

* add better error messaging

* resolve merge conflicts

* remove lc update from basicsim

* rename sync comittte variable and fix persist condition

* refactor get_light_client_update logic

* add better comments, return helpful error messages over http and rpc

* pruning canonical non checkpoint slots

* fix test

* rerun test

* update pruning logic, add tests

* fix tests

* fix imports

* fmt

* refactor db code

* Refactor db method

* Refactor db method

* add additional comments

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into persist-light-client-bootstrap

* fix merge

* linting

* merge conflict

* prevent overflow

* enable lc server for http api tests

* fix tests

* remove prints

* remove warning

* revert change
2024-09-10 00:27:49 +00:00
Håvard Anda Estensen
0c5e25b62a Use tikv-jemallocator instead of jemallocator (#6354)
* Use tikv-jemallocator instead of jemallocator

* Merge branch 'unstable' into tikv-jemallocator

* Bump tikv-jemallocator and tikv-jemalloc-ctl
2024-09-06 03:21:21 +00:00