gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-03 00:31:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
7,031 Commits 45 Branches 140 Tags
811eccdf3434e2491b89703affdc88141110a439
Commit Graph

267 Commits

Author SHA1 Message Date
Jimmy Chen
78b4cca46b Run sync tests on CI by default. (#7929) 
This PR enables some sync tests by default on CI - this will help catch breakages on sync happy paths, e.g. this would have caught the bug #7926 if lighthouse is not able to sync OR serve sync requests.

The enabled tests are genesis-sync with 120s / 300s offline time, and should cover both serving / consuming by root and by range requests.

I'm leaving the checkpoint sync tests optional, as it has external dependencies on checkpoint server (which may cause CI instability) and may cause extra loads on them.
 2025-08-26 02:49:50 +00:00
antondlr
5ebb44e222 Try using sccache instead of disabling (#7873) 
We temporarily can't build sccache on windows runners, but it's still available on linux.
this smol change lets us use it when available, instead of disabling across the board.


  The Windows runners now have a conditional check to disable (unset the `rustc-wrapper` env var) sccache in their entrypoint, just like the Linux ones have.
Also the workflows no longer fail when `sccache --show-stats` fails.
 2025-08-14 00:10:13 +00:00
Michael Sproul
bd6b8b6a65 Disable sccache to fix Windows builds (#7867) 
Quick fix to unblock Windows CI. I have a hammer and I'm using it.
 2025-08-13 01:51:19 +00:00
antondlr
6604fd10b4 Deprecate macOS-x86 binaries (#7862) 
Rust is demoting x86 for macOS: https://blog.rust-lang.org/2025/08/07/Rust-1.89.0/
This makes it unfeasible to maintain such a build going forward.


  Stop publishing `x86_64-apple-darwin` binaries.
 2025-08-12 07:23:31 +00:00
Eitan Seri-Levi
d6de8a7484 Add additional broadcast validation tests for Fulu/PeerDAS (#7325) 
Closes #6855


  Add PeerDAS broadcast validation tests and fix a small bug where `sampling_columns_indices` is none (indicating that we've already sampled the necessary columns) and `process_gossip_data_columns` gets called
 2025-07-17 07:50:28 +00:00
Michael Sproul
7b2f138ca7 Merge remote-tracking branch 'origin/stable' into release-v7.1.0 2025-07-09 11:19:16 +10:00
Michael Sproul
25ea8a83b7 Add Michael as codeowner for store crate (#7667) 
I'm adding myself as a codeowner for the `store` crate so that I can more easily keep track of database-related PRs.
 2025-06-30 10:47:52 +00:00
chonghe
7416d06dce Add genesis sync test to CI (#7561) 
* #7550

Use existing code from  @jimmygchen in #7530 and modify for genesis sync test. Thanks @jimmygchen !
 2025-06-11 09:51:37 +00:00
Jimmy Chen
9a4972053e Add e2e sync tests to CI (#7530) 
This PR adds the following sync tests to CI workflow - triggered when a PR is labeled `syncing` - to ensure we have some e2e coverage on basic sync scenarios:
- [x] checkpoint sync to a live network (covers range and backfill sync for _current_ fork)
- [x] checkpoint sync to a running devnet (covers range and backfill sync for _next_ fork)

It seems to work fine running on github hosted runners - but if performance become an issue we could switch to using self hosted runners for sepolia sync test. (standard CPU runners have 4 CPU, 16 GB ram - i think it _should_ be enough on sepolia / devnet networks)

The following tests have been **removed** from this PR and moved to a separate issue *(#7550)
- [x] genesis sync on a local devnet (covers current and next fork)
- [x] brief shutdown and restart (covers lookup sync)
- [x] longer shutdown and restart (covers range sync)

I'm hoping to keep these e2e test maintenance effort to a minimum - hopefully longer term we could have some generic e2e tests that works for all clients and the maintenance effort can be spread across teams.

### Latest test run:
https://github.com/sigp/lighthouse/actions/runs/15411744248

### Results:
<img width="687" alt="image" src="https://github.com/user-attachments/assets/c7178291-7b39-4f3b-a339-d3715eb16081" />
<img width="693" alt="image" src="https://github.com/user-attachments/assets/a8fc3520-296c-4baf-ae1e-1e887e660a3c" />

#### logs are available as artifacts:
<img width="629" alt="image" src="https://github.com/user-attachments/assets/3c0e1cd7-9c94-4d0c-be62-5e45179ab8f3" />
 2025-06-05 08:31:55 +00:00
Akihito Nakano
886ceb7e25 Run Assertoor tests in CI (#6882) 
Added Assertoor tests to the local-testnet CI.


  - The assertoor logs are included in the `logs-local-testnet` that is uploaded to GitHub Artifacts.
- Use `start_local_testnet.sh` so that we can also easily run the test locally.
 2025-06-02 00:47:06 +00:00
Jimmy Chen
9e9c51be6f Remove redundant and 2025-05-28 10:59:17 +10:00
Jimmy Chen
b7fc03437b Fix condition 2025-05-28 10:55:14 +10:00
Jimmy Chen
8dde5bdb44 Update mergify rules so that I can add waiting-on-author on a PR that's passing CI. Remove noisy comments. 2025-05-27 23:15:07 +10:00
Akihito Nakano
537fc5bde8 Revive network-test logs files in CI (#7459) 
https://github.com/sigp/lighthouse/issues/7187


  This PR adds a writer that implements `tracing_subscriber::fmt::MakeWriter`, which writes logs to separate files for each test.
 2025-05-22 02:51:22 +00:00
Jimmy Chen
7759cb8f91 Update mergify rule to not evaluate PRs that are not ready for review - to reduce noise and avoid updating stale PRs. (#7494) 2025-05-21 18:18:48 +10:00
Jimmy Chen
e21198c08b One more attempt to fix mergify condition. (#7472) 2025-05-16 13:55:06 +10:00
Jimmy Chen
e0ee148d6a Prevent mergify from updating labels while CI is still running. (#7470) 2025-05-16 11:46:21 +10:00
Jimmy Chen
cc6ae9d3f0 Fix mergify infinite loop. (#7463) 
* Fix mergify infinite loop.

* Update rule for `ready-for-review` label.

* More fix to prevent infinite loop
 2025-05-15 16:54:56 +10:00
Jimmy Chen
b1138c28fb Add additional mergify rules to automate triaging (#7451) 
* Add additional mergify rules to automate triaging.

* Update mergify config.
 2025-05-15 15:54:24 +10:00
Akihito Nakano
e0c1f27e13 simulator: Persist beacon logs (#7394) 
Beacon logs in the simulator are printed only to stdout. The logs are usually large, so persisting them would be helpful for debugging.


  Added `--log-dir` parameter to the simulators and a step to upload the logs to Artifacts.

(Update)
Added `--disable-stdout-logging` to disable stdout logging, making the CI page cleaner.
 2025-05-09 10:50:08 +00:00
Jimmy Chen
4b9c16fc71 Add Electra forks to basic sim tests (#7199) 
This PR adds transitions to Electra ~~and Fulu~~ fork epochs in the simulator tests.

~~It also covers blob inclusion verification and data column syncing on a full node in Fulu.~~

UPDATE: Remove fulu fork from sim tests due to https://github.com/sigp/lighthouse/pull/7199#issuecomment-2852281176
 2025-05-08 08:43:44 +00:00
chonghe
e90fcbe657 Add ARM binary for macOS in release (#7416) 
* #5410
 2025-05-08 06:12:57 +00:00
Tim Fan
5527125f5e Fix GitHub releases page looks bad in GitHub dark theme (#7340) 
#2573


  Change release page display in dark theme.

Before

<img width="1028" alt="image" src="https://user-images.githubusercontent.com/574696/132262479-c1e5c904-576f-4878-8a10-6012e0c51620.png" />

After:

<img width="1028" alt="image" src="https://github.com/user-attachments/assets/e6f42090-f9eb-4da7-9567-521124ea2f10" />

Others stay unchanged
 2025-04-22 09:40:36 +00:00
Michael Sproul
e61e92b926 Merge remote-tracking branch 'origin/stable' into unstable 2025-04-22 18:55:06 +10:00
chonghe
80fe133d2c Update Lighthouse Book for Electra features (#7280) 
* #7227
 2025-04-17 09:31:26 +00:00
Michael Sproul
c7acffc171 Merge remote-tracking branch 'origin/stable' into release-v7.0.0 2025-04-17 17:40:00 +10:00
Mac L
1235d44802 Remove watch (#7048) 
Removes the WIP chain indexer known as beacon watch.
 2025-03-05 05:57:09 +00:00
João Oliveira
01df433dfd update codeowners, to be more specific (#7021) 
I keep being notified for PR's like https://github.com/sigp/lighthouse/pull/7009 where it doesn't touch the specified directories in the `CODEOWNERS` file.
After reading the [docs](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-code-owners) not having a forward slash in beginning of the path means:
>  In this example, @octocat owns any file in an apps directory
> anywhere in your repository.

whereas with the slashes:
> In this example, @doctocat owns any file in the `/docs`
> directory in the root of your repository and any of its
> subdirectories.

this update makes it more rigid for the files in the jxs ownership
 2025-02-24 03:39:18 +00:00
Pawan Dhananjay
b3b6aea1c5 Rust 1.85 lints (#7019) 
N/A


  2 changes:
1. Replace Option::map_or(true, ...) with is_none_or(...)
2. Remove unnecessary `Into::into` blocks where the type conversion is apparent from the types
 2025-02-24 02:36:13 +00:00
Jimmy Chen
bf955c7543 Update mergify conditions for trivial and ready-for-merge labels to satisfy if base is not stable (#6997) 2025-02-14 10:23:38 +11:00
Eitan Seri-Levi
56f201a257 Add check to Lockbud CI job (#6898) 2025-02-04 02:00:37 +00:00
Jimmy Chen
e4183f8e4d Fix mdbook build. (#6891) 
https://github.com/sigp/lighthouse/actions/runs/13063781937/job/36452383133

`mdbook` ci job above is failing because the latest release now requires a newer version of glibc:

> Updated the Linux pre-built binaries which requires a newer version of glibc (2.34). https://github.com/rust-lang/mdBook/pull/2523

https://github.com/rust-lang/mdBook/blob/master/CHANGELOG.md

Updating to latest ubuntu to fix this.
 2025-01-31 03:39:34 +00:00
João Oliveira
029b4f2104 Improve mergify config (#6852) 
* improve mergify config

* negate conflict
 2025-01-24 11:43:51 +11:00
Michael Sproul
6b6f2beb7d Fix branch/tag names in mergify config (#6842) 2025-01-23 11:01:11 +11:00
João Oliveira
dc73791f35 update script for new mergify syntax (#6597) 2025-01-23 09:55:55 +11:00
Eitan Seri-Levi
a244aa3a69 Add libssl install to udeps task (#6777) 
* Add libssl install to udeps task

* Use HTTPS
 2025-01-10 01:13:32 +00:00
realbigsean
84519010f2 add joao CODEOWNERS (#6762) 
* add joao CODEOWNERS
 2025-01-08 00:39:48 +00:00
Daniel Knopik
f51a292f77 fully lint only explicitly to avoid unnecessary rebuilds (#6753) 
* fully lint only explicitly to avoid unnecessary rebuilds
 2025-01-03 19:27:21 +00:00
Mac L
b2b1faad4e Enforce alphabetically ordered cargo deps (#6678) 
* Enforce alphabetically ordered cargo deps

* Fix test-suite

* Another CI fix

* Merge branch 'unstable' into cargo-sort

* Fix conflicts

* Merge remote-tracking branch 'origin/unstable' into cargo-sort
 2024-12-19 05:46:03 +00:00
chonghe
1de498340c Add spell check and update Lighthouse book (#6627) 
* spellcheck config

* Merge remote-tracking branch 'origin/unstable' into spellcheck

* spellcheck update

* update spellcheck

* spell check passes

* Remove ignored and add other md files

* Remove some words in wordlist

* CI

* test spell check CI

* correct spell check

* Merge branch 'unstable' into spellcheck

* minor fix

* Merge branch 'spellcheck' of https://github.com/chong-he/lighthouse into spellcheck

* Update book

* mdlint

* delete previous_epoch_active_gwei

* Merge branch 'unstable' into spellcheck

* Tweak "container runtime" wording

* Try `BeaconState`s
 2024-12-17 07:26:59 +00:00
antondlr
b7ffcc8229 Fix: Docker CI to use org tokens (#6655) 
* update Dockerhub creds to new scheme

* Merge branch 'release-v6.0.1' into fix-docker-ci
 2024-12-12 00:24:58 +00:00
Michael Sproul
e9ec67e78a Fix Kurtosis, web3signer and cargo-audit for CI (#6671) 
* Update kurtosis-cli

* Fix name of Kurtosis artefact used in doppelganger tests

* Ignore idna vuln

* Set Java Version to 21 (required since Web3Signer 24.12.0).
 2024-12-10 02:14:13 +00:00
Michael Sproul
9fdd53df56 Hierarchical state diffs (#5978) 
* Start extracting freezer changes for tree-states

* Remove unused config args

* Add comments

* Remove unwraps

* Subjective more clear implementation

* Clean up hdiff

* Update xdelta3

* Tree states archive metrics (#6040)

* Add store cache size metrics

* Add compress timer metrics

* Add diff apply compute timer metrics

* Add diff buffer cache hit metrics

* Add hdiff buffer load times

* Add blocks replayed metric

* Move metrics to store

* Future proof some metrics

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Port and clean up forwards iterator changes

* Add and polish hierarchy-config flag

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Cleaner errors

* Fix beacon_chain test compilation

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Patch a few more freezer block roots

* Fix genesis block root bug

* Fix test failing due to pending updates

* Beacon chain tests passing

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix doc lint

* Implement DB schema upgrade for hierarchical state diffs (#6193)

* DB upgrade

* Add flag

* Delete RestorePointHash

* Update docs

* Update docs

* Implement hierarchical state diffs config migration (#6245)

* Implement hierarchical state diffs config migration

* Review PR

* Remove TODO

* Set CURRENT_SCHEMA_VERSION correctly

* Fix genesis state loading

* Re-delete some PartialBeaconState stuff

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix test compilation

* Update schema downgrade test

* Fix tests

* Fix null anchor migration

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix tree states upgrade migration (#6328)

* Towards crash safety

* Fix compilation

* Move cold summaries and state roots to new columns

* Rename StateRoots chunked field

* Update prune states

* Clean hdiff CLI flag and metrics

* Fix "staged reconstruction"

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix alloy issues

* Fix staged reconstruction logic

* Prevent weird slot drift

* Remove "allow" flag

* Update CLI help

* Remove FIXME about downgrade

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Remove some unnecessary error variants

* Fix new test

* Tree states archive - review comments and metrics (#6386)

* Review PR comments and metrics

* Comments

* Add anchor metrics

* drop prev comment

* Update metadata.rs

* Apply suggestions from code review

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/store/src/hot_cold_store.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Clarify comment and remove anchor_slot garbage

* Simplify database anchor (#6397)

* Simplify database anchor

* Update beacon_node/store/src/reconstruct.rs

* Add migration for anchor

* Fix and simplify light_client store tests

* Fix incompatible config test

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* More metrics

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* New historic state cache (#6475)

* New historic state cache

* Add more metrics

* State cache hit rate metrics

* Fix store metrics

* More logs and metrics

* Fix logger

* Ensure cached states have built caches :O

* Replay blocks in preference to diffing

* Two separate caches

* Distribute cache build time to next slot

* Re-plumb historic-state-cache flag

* Clean up metrics

* Update book

* Update beacon_node/store/src/hdiff.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update beacon_node/store/src/historic_state_cache.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update database docs

* Update diagram

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Update lockbud to work with bindgen/etc

* Correct pkg name for Debian

* Remove vestigial epochs_per_state_diff

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Markdown lint

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Address Jimmy's review comments

* Simplify ReplayFrom case

* Fix and document genesis_state_root

* Typo

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Merge branch 'unstable' into tree-states-archive

* Compute diff of validators list manually (#6556)

* Split hdiff computation

* Dedicated logic for historical roots and summaries

* Benchmark against real states

* Mutated source?

* Version the hdiff

* Add lighthouse DB config for hierarchy exponents

* Tidy up hierarchy exponents flag

* Apply suggestions from code review

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Address PR review

* Remove hardcoded paths in benchmarks

* Delete unused function in benches

* lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Test hdiff binary format stability (#6585)

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Add deprecation warning for SPRP

* Update xdelta to get rid of duplicate deps

* Document test
 2024-11-18 01:51:44 +00:00
Jimmy Chen
ae160ebf07 Remove yq installation on CI (#6574) 
* Use `snap` to install `yq` on CI.

* Remove yq install.
 2024-11-07 22:19:43 +00:00
Jimmy Chen
11260585d7 Pin kurtosis-cli version (#6555) 
* Test old version of kurtosis.
 2024-10-31 07:35:53 +00:00
Jimmy Chen
ad229a63c0 Use make cli-local in CI test suite to remove redundant docker (#6531) 
* Remove docker command from `make cli`.

* Run `cli-local` on CI.

* Update Makefile

Co-authored-by: Mac L <mjladson@pm.me>
 2024-10-22 22:51:42 +00:00
Eitan Seri-Levi
352a9cf054 Add lockbud task to CI (#6470) 
* Add lockbud task to CI

* Allow unknown lint

* Merge branch 'unstable' of https://github.com/sigp/lighthouse into lockbud

* remove potential deadlock

* ignore tokio util crate

* Update image
 2024-10-09 23:11:24 +00:00
Akihito Nakano
a4a673b780 Output network-test logs into files in CI (#6355) 
* Add ci_logger

* Update artifact name

* Add env var

* Add fork_name

* Fix clippy error

* Add comments
 2024-10-03 13:53:36 +00:00
Jimmy Chen
1e4d0e53c7 Fix docker & release workflows (#6316) 
* Fix release workflow - remove portable builds which no longer exists.
 2024-08-28 10:15:01 +00:00
chonghe
b6d15bc299 Make portable the default in Lighthouse book and Makefile, update workflows (#6247) 
* Remove portable in book

* Add back modern in docker.md

* Update Makefile

* Update release.yml

* Update docker.yml

* Update book

* Minor revision

* delete portable tarballs

* delete portable in Make command

* Fix link in book

* mdlint

* Remove modernity in docker

* Merge branch 'unstable' into update-book-portable

* Merge branch 'unstable' into update-book-portable

* Remove `-dev` docker images

* Merge remote-tracking branch 'origin/unstable' into update-book-portable
 2024-08-19 07:56:33 +00:00