Commit Graph

439 Commits

Author SHA1 Message Date
Daniel Ramirez-Chiquillo
2ecbb7f90b Remove cargo test targets, use nextest exclusively (#7874)
Fixes #7835


  - Remove cargo test-based Make targets (`test-release`, `test-debug`, `run-ef-tests`)
- Update aliases (`test`, `test-full`, `test-ef`) to use existing nextest equivalents
- Update contributing documentation to use nextest examples
- Fix example commands that previously referenced non-existing packages (`ssz`/`eth2_ssz`)


Co-Authored-By: Daniel Ramirez-Chiquillo <hi@danielrachi.com>
2025-09-10 13:52:34 +00:00
Jimmy Chen
95882bfa66 Add --telemetry-service-name CLI flag for OpenTelemetry service name override (#7903)
Allows users to customize the OpenTelemetry service name instead of using the hardcoded default `lighthouse`. Defaults to 'lighthouse-bn' for beacon node, 'lighthouse-vc' for validator client, or 'lighthouse' for other subcommands.

This is useful when analysing traces from multiple nodes, see Grafana screenshot below with service name overrides in Kurtosis (`ethereum-package` PR: https://github.com/ethpandaops/ethereum-package/pull/1160):

<img width="1148" height="627" alt="image" src="https://github.com/user-attachments/assets/7e875639-10f7-4756-837f-2006fa4b12e0" />
2025-08-20 01:16:34 +00:00
Michael Sproul
317dc0f56c Fix malloc_utils features (sysmalloc) (#7770)
Follow-up to:

- https://github.com/sigp/lighthouse/pull/7764

The `heaptrack` feature added in my previous PR was ineffective, because the jemalloc feature was turned on by the Linux target-specific dependency.

This PR tweaks the features such that:

- The jemalloc feature is just used to control whether jemalloc is compiled in. It is enabled on Linux by the target-specific dependency (see `lighthouse/Cargo.toml`), and completely disabled on Windows.
- If the `sysmalloc` feature is set on Linux then it overrides jemalloc when selecting an allocator, _even if_ the jemalloc feature is enabled (and the jemalloc dep was compiled).
2025-08-15 03:46:38 +00:00
Pawan Dhananjay
4262ad3e01 Add a flag to disable getBlobs (#7853)
N/A


  Add a flag to disable get blobs. I configured the flag to disable it regardless of version because its most likely something we use for testing anyway.
2025-08-11 23:17:00 +00:00
Jimmy Chen
40c2fd5ff4 Instrument tracing spans for block processing and import (#7816)
#7815

- removes all existing spans, so some span fields that appear in logs like `service_name` may be lost.
- instruments a few key code paths in the beacon node, starting from **root spans** named below:

* Gossip block and blobs
* `process_gossip_data_column_sidecar`
* `process_gossip_blob`
* `process_gossip_block`
* Rpc block and blobs
* `process_rpc_block`
* `process_rpc_blobs`
* `process_rpc_custody_columns`
* Rpc blocks (range and backfill)
* `process_chain_segment`
* `PendingComponents` lifecycle
* `pending_components`

To test locally:
* Run Grafana and Tempo with https://github.com/sigp/lighthouse-metrics/pull/57
* Run Lighthouse BN with `--telemetry-collector-url http://localhost:4317`

Some captured traces can be found here: https://hackmd.io/@jimmygchen/r1sLOxPPeg

Removing the old spans seem to have reduced the memory usage quite a lot - i think we were using them on long running tasks and too excessively:
<img width="910" height="495" alt="image" src="https://github.com/user-attachments/assets/5208bbe4-53b2-4ead-bc71-0b782c788669" />
2025-08-08 05:32:22 +00:00
chonghe
6409a32274 Add a guide to partially reconstruct historic states to Lighthouse book (#7679)
The main change is adding a guide to partially reconstruct historic states to the FAQ.

Other changes:
- Update the database scheme info
- Delete the Homebrew issue as it has been solved in https://github.com/Homebrew/homebrew-core/pull/225877
- Update default gas limit in: [7cbf7f1](7cbf7f1516)
- Updated the binary installation page [8076ca7](8076ca7905) as Lighthouse now supports aarch-apple binary built since v7.1.0
2025-07-14 03:24:49 +00:00
Michael Sproul
538067f1ff Merge remote-tracking branch 'origin/stable' into unstable 2025-07-10 15:53:45 +10:00
Michael Sproul
7b2f138ca7 Merge remote-tracking branch 'origin/stable' into release-v7.1.0 2025-07-09 11:19:16 +10:00
Michael Sproul
01ec2ec7ad Update LH book for v7.1.0 (#7706)
Update the book for upcoming v7.1.0 release.

This is targeted at `unstable` rather than `release-v7.1.0` because the book is built from `unstable`.
2025-07-07 04:42:34 +00:00
Pawan Dhananjay
0f895f3066 Bump default gas limit (#7695)
N/A


  Bump the default gas limit to 45 million based on recommendation from EL teams https://x.com/vdWijden/status/1939234101631856969 and pandas https://ethpandaops.io/posts/gaslimit-scaling/
2025-07-04 22:54:30 +00:00
chonghe
257d270718 Add voluntary exit via validator manager (#6612)
* #4303
* #4804


  -Add voluntary exit feature to the validator manager
-Add delete all validators by using the keyword "all"
2025-07-01 03:07:49 +00:00
Mac L
e34a9a0c65 Allow the --beacon-nodes list to be updated at runtime (#6551)
Adds a new `/lighthouse` API call to the VC which allows the list of beacon nodes to be updated dynamically at runtime.

An entirely new beacon node list is provided to the VC so it effectively adds, removes or reorders nodes to match the new list.

This can then be used in Siren, which will enable a "drag to reorder" system along with adding and removing beacon nodes while the VC is on. This will make it unnecessary to reboot the VC when users want to simply add or remove a BN from the list.
2025-06-23 03:59:34 +00:00
Pawan Dhananjay
11bcccb353 Remove all prod eth1 related code (#7133)
N/A


  After the electra fork which includes EIP 6110, the beacon node no longer needs the eth1 bridging mechanism to include new deposits as they are provided by the EL as a `deposit_request`. So after electra + a transition period where the finalized bridge deposits pre-fork are included through the old mechanism, we no longer need the elaborate machinery we had to get deposit contract data from the execution layer.

Since holesky has already forked to electra and completed the transition period, this PR basically checks to see if removing all the eth1 related logic leads to any surprises.
2025-06-23 03:00:07 +00:00
Lion - dapplion
dd98534158 Hierarchical state diffs in hot DB (#6750)
This PR implements https://github.com/sigp/lighthouse/pull/5978 (tree-states) but on the hot DB. It allows Lighthouse to massively reduce its disk footprint during non-finality and overall I/O in all cases.

Closes https://github.com/sigp/lighthouse/issues/6580

Conga into https://github.com/sigp/lighthouse/pull/6744

### TODOs

- [x] Fix OOM in CI https://github.com/sigp/lighthouse/pull/7176
- [x] optimise store_hot_state to avoid storing a duplicate state if the summary already exists (should be safe from races now that pruning is cleaner)
- [x] mispelled: get_ancenstor_state_root
- [x] get_ancestor_state_root should use state summaries
- [x] Prevent split from changing during ancestor calc
- [x] Use same hierarchy for hot and cold

### TODO Good optimization for future PRs

- [ ] On the migration, if the latest hot snapshot is aligned with the cold snapshot migrate the diffs instead of the full states.
```
align slot  time
10485760    Nov-26-2024
12582912    Sep-14-2025
14680064    Jul-02-2026
```

### TODO Maybe things good to have

- [ ] Rename anchor_slot https://github.com/sigp/lighthouse/compare/tree-states-hot-rebase-oom...dapplion:lighthouse:tree-states-hot-anchor-slot-rename?expand=1
- [ ] Make anchor fields not public such that they must be mutated through a method. To prevent un-wanted changes of the anchor_slot

### NOTTODO

- [ ] Use fork-choice and a new method [`descendants_of_checkpoint`](ca2388e196 (diff-046fbdb517ca16b80e4464c2c824cf001a74a0a94ac0065e635768ac391062a8)) to filter only the state summaries that descend of finalized checkpoint]
2025-06-19 02:43:25 +00:00
diegomrsantos
4fc0665ccd Add more context to Late Block Re-orgs (#7592)
Giving more context about late block re-orgs would make the concept easier to grasp for newcomers.


  Add more context to this section in the Lighthouse Book.
2025-06-14 03:45:35 +00:00
chonghe
f67068e1ec Update staking-deposit-cli to ethstaker-deposit-cli (#7518) 2025-06-03 04:13:57 +00:00
Mac L
0ddf9a99d6 Remove support for database migrations prior to schema version v22 (#7332)
Remove deprecated database migrations prior to v22 along with v22 migration specific code.
2025-05-28 13:47:21 +00:00
chonghe
af87135e30 Move MD059 rule to configuration file (#7484) 2025-05-20 00:12:46 +00:00
chonghe
50dbfdf612 Some updates to Lighthouse book (#7455)
- Small revision in Siren documentation in: `book/src/ui_installation.md`
- Add a section about slashing protection in web3signer, as per: https://github.com/sigp/lighthouse/issues/5310 in: `book/src/advanced_web3signer.md`
- Add a presign option in `lighthouse account validator exit` in `book/src/validator_voluntary_exit.md`

- Replace 'Holesky' with 'Hoodi' in all related parts in Lighthouse book
- Add https://ethpandaops.io/posts/kurtosis-deep-dive/ to local testnet documentation
2025-05-19 10:19:04 +00:00
Ricki Moore
e051c7ca89 Siren Pectra Feature Updates (#7396)
Add new sections explaining new features for upcoming Pectra upgrade

1.  Deposits
2. Consolidations
3. Partial Withdrawals
4. Partial Deposits / Top-ups
2025-05-12 23:50:08 +00:00
Age Manning
5b25a48af3 Siren installation improvement (#7404) 2025-05-12 23:12:52 +00:00
GarmashAlex
c8224c8d5e docs: fix broken link to voluntary exit guide (#7387) 2025-05-05 13:56:09 +00:00
Mac L
94ccd7608e Add documentation for VC API /lighthouse/beacon/health (#6653)
Changes the endpoint to get fallback health information from `/lighthouse/ui/fallback_health` to `/lighthouse/beacon/health`. This more accurately describes that the endpoint is related to the connected beacon nodes and also matched the `/lighthouse/beacon/update` endpoint being added in #6551.

Adds documentation for both fallback health and the endpoint to the Lighthouse book.
2025-04-30 02:07:07 +00:00
Roman Krasiuk
34a6c3a930 vc: increase default gas limit (#6897)
Increases default gas limit to 36M.
2025-04-30 01:15:39 +00:00
Michael Sproul
6c8770e80d Change default state cache size back to 128 (#7364)
Closes:

- https://github.com/sigp/lighthouse/issues/7363


  - Change default state cache size back to 128.
- Make state pruning properly LRU rather than MSU after skipping the cull-exempt states.
2025-04-29 01:43:25 +00:00
chonghe
2e2b0d2176 Revise consolidation info in Lighthouse book (#7351)
One of the information in the consolidation section in Lighthouse book is wrong. I realise this after reading https://ethereum.org/en/roadmap/pectra/maxeb/ and a further look at [EIP 7251](https://eips.ethereum.org/EIPS/eip-7251) which states:

`
Note: the system contract uses the EVM CALLER operation (Solidity: msg.sender) to get the address used in the consolidation request, i.e. the address that calls the system contract must match the 0x01 withdrawal credential recorded in the beacon state.
`

So the withdrawal credentials of both source and target validators need not be the same.
2025-04-28 09:43:41 +00:00
Robert Mordzon
1dd37048b9 Enable cross-compiling for riscv64 architecture (#7346)
Lighthouse currently lacks support for cross-compilation targeting the `riscv64` architecture.


  This PR introduces initial support for cross-compiling Lighthouse to `riscv64`. The following changes were made:

- **Makefile**: Updated to support `cross` with `riscv64` as a target.
- **Cross.toml**: Added configuration specific to `riscv64`.
- **Documentation**: List 'build-riscv64' in `book/src/installation_cross_compiling.md`.
2025-04-23 22:31:46 +00:00
chonghe
80fe133d2c Update Lighthouse Book for Electra features (#7280)
* #7227
2025-04-17 09:31:26 +00:00
antondlr
fb7ec0d151 Change genesis-state-url-timeout (#7112)
Timeouts sometimes occur on downloading the Holeksy genesis state from AWS, we've had reputable outside reports on this.
It's around 200MB and hosted in APAC, it makes sense to bump the default, at least for Holesky.


  Bump default timeout from 180 to 300 secs
2025-04-01 08:30:39 +00:00
ThreeHrSleep
ca8eaea116 Remove crit as an option from the CLI entirely (#7169)
https://github.com/sigp/lighthouse/issues/7165
2025-03-27 01:26:03 +00:00
chonghe
3f6c11db0e Some updates to Lighthouse book (#6995)
* #6447


  - Move some deprecated pages to a new section under `Archived`

- Remove fallback log in mev as the log will not be present after VC using `/eth/v3/validator/blocks` endpoint by default
- Add warning against using Btrfs file system (thank you @ChosunOne for the report)
- Add data shared by @mcdee  on tree states API queries time
- Rename partial withdrawals to validator sweep to differentiate it from the upcoming execution layer partial withdrawals
- Update NAT API response
- Update docs on IPv6

- Rename .md files to follow a standard prefix section name, e.g., installation_*.md, advanced_*.md
- Standardise .md files using underscore `_` instead of hyphen `-` to be consistent with other files naming conventions.
2025-03-24 02:30:20 +00:00
Eitan Seri-Levi
cbf1c04a14 resolve merge conflicts between untstable and release-v7.0.0 2025-03-23 11:09:02 -06:00
Michael Sproul
58482586f5 Support Hoodi testnet (#7145)
Hardcode config for the upcoming `hoodi` testnet so we can run with `--network hoodi`.
2025-03-18 02:10:24 +00:00
Michael Sproul
4de062626b State cache tweaks (#7095)
Backport of:

- https://github.com/sigp/lighthouse/pull/7067

For:

- https://github.com/sigp/lighthouse/issues/7039


  - Prevent writing to state cache when migrating the database
- Add `state-cache-headroom` flag to control pruning
- Prune old epoch boundary states ahead of mid-epoch states
- Never prune head block's state
- Avoid caching ancestor states unless they are on an epoch boundary
- Log when states enter/exit the cache

Co-authored-by: Eitan Seri-Levi <eserilev@ucsc.edu>
2025-03-18 02:10:21 +00:00
Eitan Seri-Levi
2c40f0b004 Set epochs-per-blob-prune default to 256 (#7113)
Partially #7100


  Set blob pruning to default to once per day
2025-03-13 02:43:07 +00:00
ThreeHrSleep
d60c24ef1c Integrate tracing (#6339)
Tracing Integration
- [reference](5bbf1859e9/projects/project-ideas.md (L297))


  - [x] replace slog & log with tracing throughout the codebase
- [x] implement custom crit log
- [x] make relevant changes in the formatter
- [x] replace sloggers
- [x] re-write SSE logging components

cc: @macladson @eserilev
2025-03-12 22:31:05 +00:00
Jimmy Chen
9c4fc6eac2 Change state cache size default to 32 (#7101)
Cherry-picking #7055 from `holesky-rescue` branch to the clean `release-v7.0.0` branch.
2025-03-11 01:21:50 +00:00
Michael Sproul
b4e79edf2a Merge remote-tracking branch 'origin/release-v7.0.0' into unstable 2025-03-10 15:21:24 +11:00
Mac L
1235d44802 Remove watch (#7048)
Removes the WIP chain indexer known as beacon watch.
2025-03-05 05:57:09 +00:00
Mac L
29a295a134 Add --long-timeouts-multiplier CLI flag (#7047)
Adds the `--long-timeouts-multiplier` flag.
Allows granular control for VC timeouts which has proved useful in Holesky.
2025-03-05 01:52:57 +00:00
Michael Sproul
80cd8bd911 Add --disable-attesting flag to validator client (#7046)
Cleaned up and isolated version of the `--disable-attesting` flag for the VC, from the `holesky-rescue` branch:

- https://github.com/sigp/lighthouse/pull/7041

I figured we don't need the `--disable-attesting` flag on the BN for now, and it was a much more invasive impl.
2025-02-26 13:07:16 +00:00
Pawan Dhananjay
522b3cbaab Fix builder API headers (#7009)
Resolves https://github.com/sigp/lighthouse/issues/7000


  Set the accept header on builder to the correct value when requesting ssz.

This PR also adds a flag to disable ssz over the builder api altogether. In the case that builders/relays have an ssz bug, we can react quickly by asking clients to restart their nodes with the `--disable-ssz-builder` flag to force json. I'm not fully convinced if this is useful so open to removing it or opening another PR for it.

Testing this currently.
2025-02-24 03:39:13 +00:00
Age Manning
62a0f25f97 IPv6 By Default (#6808) 2025-02-10 01:58:11 +00:00
Eitan Seri-Levi
afdda83798 Enable Light Client server by default (#6950) 2025-02-10 01:27:03 +00:00
Age Manning
d1061dcf59 UX Network Fixes (#6796)
There were two things I came across during some recent testing, that this PR addresses.

1 - The default port for IPv6 was set to 9090, which is confusing. I've set this to match its ipv4 counterpart (i.e 9000 and 9001). This makes more sense and is easier to firewall, for those firewalls that support both versions for a single rule.

2 - Watching the NAT status of lighthouse, I notice we only set the field to 1 once the NAT is passed. We don't give it a default 0 (false). So we only see results when its successful. On peer disconnects, i've piggy-backed a loop of the connected peers to also watch and check for NAT status updates.
2025-02-04 01:28:37 +00:00
Eitan Seri-Levi
7e4b27c922 Migrate validator client to clap derive (#6300)
Partially #5900


  Migrate the validator client cli to clap derive
2025-02-03 20:08:31 +00:00
Eitan Seri-Levi
a1b7d616b4 Modularize beacon node backend (#4718)
#4669


  Modularize the beacon node backend to make it easier to add new database implementations
2025-01-23 02:12:16 +00:00
chonghe
5022398715 Revise Siren documentation (#6553)
* revise Siren doc

* Fix broken links

* Fix broken links

* broken links

* mdlint

* mdlint

* mdlint again

* Merge branch 'unstable' into book-siren

* test whether I have the required privs :-)

* revise

* some minor siren related changes for the book

* updates re: `--net=host`

* lint

* Minor revision

* Add note

* mdlint

* Merge branch 'unstable' into book-siren

* Merge branch 'unstable' into book-siren

* Merge remote-tracking branch 'origin/unstable' into book-siren

* Fix spellcheck

* Capital letters SSL
2024-12-19 05:46:10 +00:00
chonghe
1de498340c Add spell check and update Lighthouse book (#6627)
* spellcheck config

* Merge remote-tracking branch 'origin/unstable' into spellcheck

* spellcheck update

* update spellcheck

* spell check passes

* Remove ignored and add other md files

* Remove some words in wordlist

* CI

* test spell check CI

* correct spell check

* Merge branch 'unstable' into spellcheck

* minor fix

* Merge branch 'spellcheck' of https://github.com/chong-he/lighthouse into spellcheck

* Update book

* mdlint

* delete previous_epoch_active_gwei

* Merge branch 'unstable' into spellcheck

* Tweak "container runtime" wording

* Try `BeaconState`s
2024-12-17 07:26:59 +00:00
Jun Song
11e1d5bf14 Add CLI flag for HTTP API token path (VC) (#6577)
* Add cli flag for HTTP API token path (VC)

* Add http_token_path_flag test

* Add pre-check for directory case & Fix test utils

* Update docs

* Apply review: move http_token_path into validator_http_api config

* Lint

* Make diff lesser to replace PK_FILENAME

* Merge branch 'unstable' into feature/cli-token-path

* Applt review: help_vc.md

Co-authored-by: chonghe <44791194+chong-he@users.noreply.github.com>

* Fix help for cli

* Fix issues on ci

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path
2024-12-16 05:43:54 +00:00