Commit Graph

434 Commits

Author SHA1 Message Date
chonghe
6409a32274 Add a guide to partially reconstruct historic states to Lighthouse book (#7679)
The main change is adding a guide to partially reconstruct historic states to the FAQ.

Other changes:
- Update the database scheme info
- Delete the Homebrew issue as it has been solved in https://github.com/Homebrew/homebrew-core/pull/225877
- Update default gas limit in: [7cbf7f1](7cbf7f1516)
- Updated the binary installation page [8076ca7](8076ca7905) as Lighthouse now supports aarch-apple binary built since v7.1.0
2025-07-14 03:24:49 +00:00
Michael Sproul
538067f1ff Merge remote-tracking branch 'origin/stable' into unstable 2025-07-10 15:53:45 +10:00
Michael Sproul
7b2f138ca7 Merge remote-tracking branch 'origin/stable' into release-v7.1.0 2025-07-09 11:19:16 +10:00
Michael Sproul
01ec2ec7ad Update LH book for v7.1.0 (#7706)
Update the book for upcoming v7.1.0 release.

This is targeted at `unstable` rather than `release-v7.1.0` because the book is built from `unstable`.
2025-07-07 04:42:34 +00:00
Pawan Dhananjay
0f895f3066 Bump default gas limit (#7695)
N/A


  Bump the default gas limit to 45 million based on recommendation from EL teams https://x.com/vdWijden/status/1939234101631856969 and pandas https://ethpandaops.io/posts/gaslimit-scaling/
2025-07-04 22:54:30 +00:00
chonghe
257d270718 Add voluntary exit via validator manager (#6612)
* #4303
* #4804


  -Add voluntary exit feature to the validator manager
-Add delete all validators by using the keyword "all"
2025-07-01 03:07:49 +00:00
Mac L
e34a9a0c65 Allow the --beacon-nodes list to be updated at runtime (#6551)
Adds a new `/lighthouse` API call to the VC which allows the list of beacon nodes to be updated dynamically at runtime.

An entirely new beacon node list is provided to the VC so it effectively adds, removes or reorders nodes to match the new list.

This can then be used in Siren, which will enable a "drag to reorder" system along with adding and removing beacon nodes while the VC is on. This will make it unnecessary to reboot the VC when users want to simply add or remove a BN from the list.
2025-06-23 03:59:34 +00:00
Pawan Dhananjay
11bcccb353 Remove all prod eth1 related code (#7133)
N/A


  After the electra fork which includes EIP 6110, the beacon node no longer needs the eth1 bridging mechanism to include new deposits as they are provided by the EL as a `deposit_request`. So after electra + a transition period where the finalized bridge deposits pre-fork are included through the old mechanism, we no longer need the elaborate machinery we had to get deposit contract data from the execution layer.

Since holesky has already forked to electra and completed the transition period, this PR basically checks to see if removing all the eth1 related logic leads to any surprises.
2025-06-23 03:00:07 +00:00
Lion - dapplion
dd98534158 Hierarchical state diffs in hot DB (#6750)
This PR implements https://github.com/sigp/lighthouse/pull/5978 (tree-states) but on the hot DB. It allows Lighthouse to massively reduce its disk footprint during non-finality and overall I/O in all cases.

Closes https://github.com/sigp/lighthouse/issues/6580

Conga into https://github.com/sigp/lighthouse/pull/6744

### TODOs

- [x] Fix OOM in CI https://github.com/sigp/lighthouse/pull/7176
- [x] optimise store_hot_state to avoid storing a duplicate state if the summary already exists (should be safe from races now that pruning is cleaner)
- [x] mispelled: get_ancenstor_state_root
- [x] get_ancestor_state_root should use state summaries
- [x] Prevent split from changing during ancestor calc
- [x] Use same hierarchy for hot and cold

### TODO Good optimization for future PRs

- [ ] On the migration, if the latest hot snapshot is aligned with the cold snapshot migrate the diffs instead of the full states.
```
align slot  time
10485760    Nov-26-2024
12582912    Sep-14-2025
14680064    Jul-02-2026
```

### TODO Maybe things good to have

- [ ] Rename anchor_slot https://github.com/sigp/lighthouse/compare/tree-states-hot-rebase-oom...dapplion:lighthouse:tree-states-hot-anchor-slot-rename?expand=1
- [ ] Make anchor fields not public such that they must be mutated through a method. To prevent un-wanted changes of the anchor_slot

### NOTTODO

- [ ] Use fork-choice and a new method [`descendants_of_checkpoint`](ca2388e196 (diff-046fbdb517ca16b80e4464c2c824cf001a74a0a94ac0065e635768ac391062a8)) to filter only the state summaries that descend of finalized checkpoint]
2025-06-19 02:43:25 +00:00
diegomrsantos
4fc0665ccd Add more context to Late Block Re-orgs (#7592)
Giving more context about late block re-orgs would make the concept easier to grasp for newcomers.


  Add more context to this section in the Lighthouse Book.
2025-06-14 03:45:35 +00:00
chonghe
f67068e1ec Update staking-deposit-cli to ethstaker-deposit-cli (#7518) 2025-06-03 04:13:57 +00:00
Mac L
0ddf9a99d6 Remove support for database migrations prior to schema version v22 (#7332)
Remove deprecated database migrations prior to v22 along with v22 migration specific code.
2025-05-28 13:47:21 +00:00
chonghe
af87135e30 Move MD059 rule to configuration file (#7484) 2025-05-20 00:12:46 +00:00
chonghe
50dbfdf612 Some updates to Lighthouse book (#7455)
- Small revision in Siren documentation in: `book/src/ui_installation.md`
- Add a section about slashing protection in web3signer, as per: https://github.com/sigp/lighthouse/issues/5310 in: `book/src/advanced_web3signer.md`
- Add a presign option in `lighthouse account validator exit` in `book/src/validator_voluntary_exit.md`

- Replace 'Holesky' with 'Hoodi' in all related parts in Lighthouse book
- Add https://ethpandaops.io/posts/kurtosis-deep-dive/ to local testnet documentation
2025-05-19 10:19:04 +00:00
Ricki Moore
e051c7ca89 Siren Pectra Feature Updates (#7396)
Add new sections explaining new features for upcoming Pectra upgrade

1.  Deposits
2. Consolidations
3. Partial Withdrawals
4. Partial Deposits / Top-ups
2025-05-12 23:50:08 +00:00
Age Manning
5b25a48af3 Siren installation improvement (#7404) 2025-05-12 23:12:52 +00:00
GarmashAlex
c8224c8d5e docs: fix broken link to voluntary exit guide (#7387) 2025-05-05 13:56:09 +00:00
Mac L
94ccd7608e Add documentation for VC API /lighthouse/beacon/health (#6653)
Changes the endpoint to get fallback health information from `/lighthouse/ui/fallback_health` to `/lighthouse/beacon/health`. This more accurately describes that the endpoint is related to the connected beacon nodes and also matched the `/lighthouse/beacon/update` endpoint being added in #6551.

Adds documentation for both fallback health and the endpoint to the Lighthouse book.
2025-04-30 02:07:07 +00:00
Roman Krasiuk
34a6c3a930 vc: increase default gas limit (#6897)
Increases default gas limit to 36M.
2025-04-30 01:15:39 +00:00
Michael Sproul
6c8770e80d Change default state cache size back to 128 (#7364)
Closes:

- https://github.com/sigp/lighthouse/issues/7363


  - Change default state cache size back to 128.
- Make state pruning properly LRU rather than MSU after skipping the cull-exempt states.
2025-04-29 01:43:25 +00:00
chonghe
2e2b0d2176 Revise consolidation info in Lighthouse book (#7351)
One of the information in the consolidation section in Lighthouse book is wrong. I realise this after reading https://ethereum.org/en/roadmap/pectra/maxeb/ and a further look at [EIP 7251](https://eips.ethereum.org/EIPS/eip-7251) which states:

`
Note: the system contract uses the EVM CALLER operation (Solidity: msg.sender) to get the address used in the consolidation request, i.e. the address that calls the system contract must match the 0x01 withdrawal credential recorded in the beacon state.
`

So the withdrawal credentials of both source and target validators need not be the same.
2025-04-28 09:43:41 +00:00
Robert Mordzon
1dd37048b9 Enable cross-compiling for riscv64 architecture (#7346)
Lighthouse currently lacks support for cross-compilation targeting the `riscv64` architecture.


  This PR introduces initial support for cross-compiling Lighthouse to `riscv64`. The following changes were made:

- **Makefile**: Updated to support `cross` with `riscv64` as a target.
- **Cross.toml**: Added configuration specific to `riscv64`.
- **Documentation**: List 'build-riscv64' in `book/src/installation_cross_compiling.md`.
2025-04-23 22:31:46 +00:00
chonghe
80fe133d2c Update Lighthouse Book for Electra features (#7280)
* #7227
2025-04-17 09:31:26 +00:00
antondlr
fb7ec0d151 Change genesis-state-url-timeout (#7112)
Timeouts sometimes occur on downloading the Holeksy genesis state from AWS, we've had reputable outside reports on this.
It's around 200MB and hosted in APAC, it makes sense to bump the default, at least for Holesky.


  Bump default timeout from 180 to 300 secs
2025-04-01 08:30:39 +00:00
ThreeHrSleep
ca8eaea116 Remove crit as an option from the CLI entirely (#7169)
https://github.com/sigp/lighthouse/issues/7165
2025-03-27 01:26:03 +00:00
chonghe
3f6c11db0e Some updates to Lighthouse book (#6995)
* #6447


  - Move some deprecated pages to a new section under `Archived`

- Remove fallback log in mev as the log will not be present after VC using `/eth/v3/validator/blocks` endpoint by default
- Add warning against using Btrfs file system (thank you @ChosunOne for the report)
- Add data shared by @mcdee  on tree states API queries time
- Rename partial withdrawals to validator sweep to differentiate it from the upcoming execution layer partial withdrawals
- Update NAT API response
- Update docs on IPv6

- Rename .md files to follow a standard prefix section name, e.g., installation_*.md, advanced_*.md
- Standardise .md files using underscore `_` instead of hyphen `-` to be consistent with other files naming conventions.
2025-03-24 02:30:20 +00:00
Eitan Seri-Levi
cbf1c04a14 resolve merge conflicts between untstable and release-v7.0.0 2025-03-23 11:09:02 -06:00
Michael Sproul
58482586f5 Support Hoodi testnet (#7145)
Hardcode config for the upcoming `hoodi` testnet so we can run with `--network hoodi`.
2025-03-18 02:10:24 +00:00
Michael Sproul
4de062626b State cache tweaks (#7095)
Backport of:

- https://github.com/sigp/lighthouse/pull/7067

For:

- https://github.com/sigp/lighthouse/issues/7039


  - Prevent writing to state cache when migrating the database
- Add `state-cache-headroom` flag to control pruning
- Prune old epoch boundary states ahead of mid-epoch states
- Never prune head block's state
- Avoid caching ancestor states unless they are on an epoch boundary
- Log when states enter/exit the cache

Co-authored-by: Eitan Seri-Levi <eserilev@ucsc.edu>
2025-03-18 02:10:21 +00:00
Eitan Seri-Levi
2c40f0b004 Set epochs-per-blob-prune default to 256 (#7113)
Partially #7100


  Set blob pruning to default to once per day
2025-03-13 02:43:07 +00:00
ThreeHrSleep
d60c24ef1c Integrate tracing (#6339)
Tracing Integration
- [reference](5bbf1859e9/projects/project-ideas.md (L297))


  - [x] replace slog & log with tracing throughout the codebase
- [x] implement custom crit log
- [x] make relevant changes in the formatter
- [x] replace sloggers
- [x] re-write SSE logging components

cc: @macladson @eserilev
2025-03-12 22:31:05 +00:00
Jimmy Chen
9c4fc6eac2 Change state cache size default to 32 (#7101)
Cherry-picking #7055 from `holesky-rescue` branch to the clean `release-v7.0.0` branch.
2025-03-11 01:21:50 +00:00
Michael Sproul
b4e79edf2a Merge remote-tracking branch 'origin/release-v7.0.0' into unstable 2025-03-10 15:21:24 +11:00
Mac L
1235d44802 Remove watch (#7048)
Removes the WIP chain indexer known as beacon watch.
2025-03-05 05:57:09 +00:00
Mac L
29a295a134 Add --long-timeouts-multiplier CLI flag (#7047)
Adds the `--long-timeouts-multiplier` flag.
Allows granular control for VC timeouts which has proved useful in Holesky.
2025-03-05 01:52:57 +00:00
Michael Sproul
80cd8bd911 Add --disable-attesting flag to validator client (#7046)
Cleaned up and isolated version of the `--disable-attesting` flag for the VC, from the `holesky-rescue` branch:

- https://github.com/sigp/lighthouse/pull/7041

I figured we don't need the `--disable-attesting` flag on the BN for now, and it was a much more invasive impl.
2025-02-26 13:07:16 +00:00
Pawan Dhananjay
522b3cbaab Fix builder API headers (#7009)
Resolves https://github.com/sigp/lighthouse/issues/7000


  Set the accept header on builder to the correct value when requesting ssz.

This PR also adds a flag to disable ssz over the builder api altogether. In the case that builders/relays have an ssz bug, we can react quickly by asking clients to restart their nodes with the `--disable-ssz-builder` flag to force json. I'm not fully convinced if this is useful so open to removing it or opening another PR for it.

Testing this currently.
2025-02-24 03:39:13 +00:00
Age Manning
62a0f25f97 IPv6 By Default (#6808) 2025-02-10 01:58:11 +00:00
Eitan Seri-Levi
afdda83798 Enable Light Client server by default (#6950) 2025-02-10 01:27:03 +00:00
Age Manning
d1061dcf59 UX Network Fixes (#6796)
There were two things I came across during some recent testing, that this PR addresses.

1 - The default port for IPv6 was set to 9090, which is confusing. I've set this to match its ipv4 counterpart (i.e 9000 and 9001). This makes more sense and is easier to firewall, for those firewalls that support both versions for a single rule.

2 - Watching the NAT status of lighthouse, I notice we only set the field to 1 once the NAT is passed. We don't give it a default 0 (false). So we only see results when its successful. On peer disconnects, i've piggy-backed a loop of the connected peers to also watch and check for NAT status updates.
2025-02-04 01:28:37 +00:00
Eitan Seri-Levi
7e4b27c922 Migrate validator client to clap derive (#6300)
Partially #5900


  Migrate the validator client cli to clap derive
2025-02-03 20:08:31 +00:00
Eitan Seri-Levi
a1b7d616b4 Modularize beacon node backend (#4718)
#4669


  Modularize the beacon node backend to make it easier to add new database implementations
2025-01-23 02:12:16 +00:00
chonghe
5022398715 Revise Siren documentation (#6553)
* revise Siren doc

* Fix broken links

* Fix broken links

* broken links

* mdlint

* mdlint

* mdlint again

* Merge branch 'unstable' into book-siren

* test whether I have the required privs :-)

* revise

* some minor siren related changes for the book

* updates re: `--net=host`

* lint

* Minor revision

* Add note

* mdlint

* Merge branch 'unstable' into book-siren

* Merge branch 'unstable' into book-siren

* Merge remote-tracking branch 'origin/unstable' into book-siren

* Fix spellcheck

* Capital letters SSL
2024-12-19 05:46:10 +00:00
chonghe
1de498340c Add spell check and update Lighthouse book (#6627)
* spellcheck config

* Merge remote-tracking branch 'origin/unstable' into spellcheck

* spellcheck update

* update spellcheck

* spell check passes

* Remove ignored and add other md files

* Remove some words in wordlist

* CI

* test spell check CI

* correct spell check

* Merge branch 'unstable' into spellcheck

* minor fix

* Merge branch 'spellcheck' of https://github.com/chong-he/lighthouse into spellcheck

* Update book

* mdlint

* delete previous_epoch_active_gwei

* Merge branch 'unstable' into spellcheck

* Tweak "container runtime" wording

* Try `BeaconState`s
2024-12-17 07:26:59 +00:00
Jun Song
11e1d5bf14 Add CLI flag for HTTP API token path (VC) (#6577)
* Add cli flag for HTTP API token path (VC)

* Add http_token_path_flag test

* Add pre-check for directory case & Fix test utils

* Update docs

* Apply review: move http_token_path into validator_http_api config

* Lint

* Make diff lesser to replace PK_FILENAME

* Merge branch 'unstable' into feature/cli-token-path

* Applt review: help_vc.md

Co-authored-by: chonghe <44791194+chong-he@users.noreply.github.com>

* Fix help for cli

* Fix issues on ci

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path

* Merge branch 'unstable' into feature/cli-token-path
2024-12-16 05:43:54 +00:00
Michael Sproul
d49e1be35d Remove heading that isn't rendered correctly (#6650)
* Remove heading that isn't rendered correctly
2024-12-13 05:44:41 +00:00
Michael Sproul
1fd86f8b59 Add a security section to the book (#6581)
* Add a security section to the book

* Update book/src/security.md

Co-authored-by: chonghe <44791194+chong-he@users.noreply.github.com>

* Update book/src/security.md

Co-authored-by: chonghe <44791194+chong-he@users.noreply.github.com>
2024-12-03 00:08:53 +00:00
Michael Sproul
79de61b624 Update DB migrations in book (#6611)
* Update DB migrations in book

* Merge branch 'unstable' into db-migrations-v22
2024-11-26 05:08:24 +00:00
Michael Sproul
c5007eaa1c Deprecate eth1 and dummy-eth1 flags (#6566)
* Deprecate eth1/dummy-eth1 flags

* Update book

* Simplify + make staking conflict with disable-deposit-contract
2024-11-18 08:36:01 +00:00
Michael Sproul
9fdd53df56 Hierarchical state diffs (#5978)
* Start extracting freezer changes for tree-states

* Remove unused config args

* Add comments

* Remove unwraps

* Subjective more clear implementation

* Clean up hdiff

* Update xdelta3

* Tree states archive metrics (#6040)

* Add store cache size metrics

* Add compress timer metrics

* Add diff apply compute timer metrics

* Add diff buffer cache hit metrics

* Add hdiff buffer load times

* Add blocks replayed metric

* Move metrics to store

* Future proof some metrics

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Port and clean up forwards iterator changes

* Add and polish hierarchy-config flag

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Cleaner errors

* Fix beacon_chain test compilation

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Patch a few more freezer block roots

* Fix genesis block root bug

* Fix test failing due to pending updates

* Beacon chain tests passing

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix doc lint

* Implement DB schema upgrade for hierarchical state diffs (#6193)

* DB upgrade

* Add flag

* Delete RestorePointHash

* Update docs

* Update docs

* Implement hierarchical state diffs config migration (#6245)

* Implement hierarchical state diffs config migration

* Review PR

* Remove TODO

* Set CURRENT_SCHEMA_VERSION correctly

* Fix genesis state loading

* Re-delete some PartialBeaconState stuff

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix test compilation

* Update schema downgrade test

* Fix tests

* Fix null anchor migration

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix tree states upgrade migration (#6328)

* Towards crash safety

* Fix compilation

* Move cold summaries and state roots to new columns

* Rename StateRoots chunked field

* Update prune states

* Clean hdiff CLI flag and metrics

* Fix "staged reconstruction"

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix alloy issues

* Fix staged reconstruction logic

* Prevent weird slot drift

* Remove "allow" flag

* Update CLI help

* Remove FIXME about downgrade

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Remove some unnecessary error variants

* Fix new test

* Tree states archive - review comments and metrics (#6386)

* Review PR comments and metrics

* Comments

* Add anchor metrics

* drop prev comment

* Update metadata.rs

* Apply suggestions from code review

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/store/src/hot_cold_store.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Clarify comment and remove anchor_slot garbage

* Simplify database anchor (#6397)

* Simplify database anchor

* Update beacon_node/store/src/reconstruct.rs

* Add migration for anchor

* Fix and simplify light_client store tests

* Fix incompatible config test

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* More metrics

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* New historic state cache (#6475)

* New historic state cache

* Add more metrics

* State cache hit rate metrics

* Fix store metrics

* More logs and metrics

* Fix logger

* Ensure cached states have built caches :O

* Replay blocks in preference to diffing

* Two separate caches

* Distribute cache build time to next slot

* Re-plumb historic-state-cache flag

* Clean up metrics

* Update book

* Update beacon_node/store/src/hdiff.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update beacon_node/store/src/historic_state_cache.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update database docs

* Update diagram

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Update lockbud to work with bindgen/etc

* Correct pkg name for Debian

* Remove vestigial epochs_per_state_diff

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Markdown lint

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Address Jimmy's review comments

* Simplify ReplayFrom case

* Fix and document genesis_state_root

* Typo

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Merge branch 'unstable' into tree-states-archive

* Compute diff of validators list manually (#6556)

* Split hdiff computation

* Dedicated logic for historical roots and summaries

* Benchmark against real states

* Mutated source?

* Version the hdiff

* Add lighthouse DB config for hierarchy exponents

* Tidy up hierarchy exponents flag

* Apply suggestions from code review

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Address PR review

* Remove hardcoded paths in benchmarks

* Delete unused function in benches

* lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Test hdiff binary format stability (#6585)

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Add deprecation warning for SPRP

* Update xdelta to get rid of duplicate deps

* Document test
2024-11-18 01:51:44 +00:00