This PR adds the following sync tests to CI workflow - triggered when a PR is labeled `syncing` - to ensure we have some e2e coverage on basic sync scenarios:
- [x] checkpoint sync to a live network (covers range and backfill sync for _current_ fork)
- [x] checkpoint sync to a running devnet (covers range and backfill sync for _next_ fork)
It seems to work fine running on github hosted runners - but if performance become an issue we could switch to using self hosted runners for sepolia sync test. (standard CPU runners have 4 CPU, 16 GB ram - i think it _should_ be enough on sepolia / devnet networks)
The following tests have been **removed** from this PR and moved to a separate issue *(#7550)
- [x] genesis sync on a local devnet (covers current and next fork)
- [x] brief shutdown and restart (covers lookup sync)
- [x] longer shutdown and restart (covers range sync)
I'm hoping to keep these e2e test maintenance effort to a minimum - hopefully longer term we could have some generic e2e tests that works for all clients and the maintenance effort can be spread across teams.
### Latest test run:
https://github.com/sigp/lighthouse/actions/runs/15411744248
### Results:
<img width="687" alt="image" src="https://github.com/user-attachments/assets/c7178291-7b39-4f3b-a339-d3715eb16081" />
<img width="693" alt="image" src="https://github.com/user-attachments/assets/a8fc3520-296c-4baf-ae1e-1e887e660a3c" />
#### logs are available as artifacts:
<img width="629" alt="image" src="https://github.com/user-attachments/assets/3c0e1cd7-9c94-4d0c-be62-5e45179ab8f3" />
Added Assertoor tests to the local-testnet CI.
- The assertoor logs are included in the `logs-local-testnet` that is uploaded to GitHub Artifacts.
- Use `start_local_testnet.sh` so that we can also easily run the test locally.
Beacon logs in the simulator are printed only to stdout. The logs are usually large, so persisting them would be helpful for debugging.
Added `--log-dir` parameter to the simulators and a step to upload the logs to Artifacts.
(Update)
Added `--disable-stdout-logging` to disable stdout logging, making the CI page cleaner.
This PR adds transitions to Electra ~~and Fulu~~ fork epochs in the simulator tests.
~~It also covers blob inclusion verification and data column syncing on a full node in Fulu.~~
UPDATE: Remove fulu fork from sim tests due to https://github.com/sigp/lighthouse/pull/7199#issuecomment-2852281176
N/A
2 changes:
1. Replace Option::map_or(true, ...) with is_none_or(...)
2. Remove unnecessary `Into::into` blocks where the type conversion is apparent from the types
* Update kurtosis-cli
* Fix name of Kurtosis artefact used in doppelganger tests
* Ignore idna vuln
* Set Java Version to 21 (required since Web3Signer 24.12.0).
* Remove portable in book
* Add back modern in docker.md
* Update Makefile
* Update release.yml
* Update docker.yml
* Update book
* Minor revision
* delete portable tarballs
* delete portable in Make command
* Fix link in book
* mdlint
* Remove modernity in docker
* Merge branch 'unstable' into update-book-portable
* Merge branch 'unstable' into update-book-portable
* Remove `-dev` docker images
* Merge remote-tracking branch 'origin/unstable' into update-book-portable
* Kurtosis local testnet.
* Remove unused `lcli` subcommands
* Migrate doppelganger_protection test to kurtosis and further cleanup.
* Fix lint
* Add missing download image step and remove unused `lcli` dependencies.
* doppelganger success case working
* Run tests on hosted runner and improve error handling.
* Start the dp vc only after epoch 1
* Add more logging to test results.
* Fix exit code and speed up docker build.
* Fix incorrect exit codes and split doppelganger tests on CI.
* Missing the escape for double quotes 😫
* Remove unnecessary vc params in kurtosis config.
* fix lib.rs and tests.rs
* update decode.rs
* auto-delete in Cargo.lock
* delete milagro in cargo.toml
* remove milagro from makefile
* remove milagro from the name
* delete milagro in comment
* delete milagro in cargo.toml
* delete in /testing/ef_tests/cargo.toml
* delete milagro in the logical OR
* delete milagro in /lighthouse/src/main.rs
* delete milagro in /crypto/bls/tests/tests.rs
* delete milagro in comment
* delete milagro in /testing//ef_test/src//cases/bls_eth_aggregate_pubkeys.rs
* delete milagro
* delete more in lib.rs
* delete more in lib.rs
* delete more in lib.rs
* delete milagro in /crypto/bls/src/lib.rs
* delete milagro in crypto/bls/src/mod.rs
* delete milagro.rs
* Add mergify.yml.
* Abstract out CI jobs to a "success" job so that a change in the CI jobs don't require modification to mergify configuration on stable branch.
* Add new jobs to the `needs` list of `test-suite-success`.
* Set `batch_max_wait_time` to 60 s.
* Add cli.sh file
* update bash script
* update Makefile
* update
* modified test-suite
* fix path
* Fix cli.sh permissions
* update cmd
* cli_manual
* Revise to update
* Update directory in Github
* Correct cli.txt directory
* test old cli_manual
* change exit 1
* Update cli and makefile
* Move cli.sh
* remove files
* fix permission
* Indentation and revision
* Fixed permission
* Create new cli folder
* remove dummy
* put a dummy file
* Revise cli.sh
* comment
* function
* remove vm.md
* test make cli
* test
* testing
* testing
* update
* update
* test
* test
* add vm
* change back non-debug mode
* add exist and update for future debug
* revise
* remove troubleshooting part
* update
* add summary.md
* test
* test
* Update Makefile
Co-authored-by: Mac L <mjladson@pm.me>
* Update Makefile
Co-authored-by: Mac L <mjladson@pm.me>
* Remove help-cli.md and rearrange
* Remove help-cli.md
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* remove maxperf
* move then to same line as if
* Fix indent and echo file not found
* To be explicit in replacing the old file
* Add logging when there are changes
* Add local variables
* spacing
* remove cargo fmt
* update .md files
* Edit exit message to avoid confusion
* Remove am and add vm subcommands
* Add cargo-fmt
* Revise test-suite.yml
* Update SUMMARY.md
* Add set -e
* Add vm
* Fix
* Add vm
* set -e
* Remove return 1 and add :
* Small revision
* Fix typo
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Indent
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Remove .exe in Windows
* Fix period with \.
* test
* check diff
* linux commit
* Add cli.sh file
* update bash script
* update Makefile
* update
* modified test-suite
* fix path
* Fix cli.sh permissions
* update cmd
* cli_manual
* Revise to update
* Update directory in Github
* Correct cli.txt directory
* test old cli_manual
* change exit 1
* Update cli and makefile
* Move cli.sh
* remove files
* fix permission
* Indentation and revision
* Fixed permission
* Create new cli folder
* remove dummy
* put a dummy file
* Revise cli.sh
* comment
* function
* remove vm.md
* test make cli
* test
* testing
* testing
* update
* update
* test
* test
* add vm
* change back non-debug mode
* add exist and update for future debug
* revise
* remove troubleshooting part
* update
* add summary.md
* test
* test
* Update Makefile
Co-authored-by: Mac L <mjladson@pm.me>
* Update Makefile
Co-authored-by: Mac L <mjladson@pm.me>
* Remove help-cli.md and rearrange
* Remove help-cli.md
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* remove maxperf
* move then to same line as if
* Fix indent and echo file not found
* To be explicit in replacing the old file
* Add logging when there are changes
* Add local variables
* spacing
* remove cargo fmt
* Edit exit message to avoid confusion
* update .md files
* Remove am and add vm subcommands
* Add cargo-fmt
* Revise test-suite.yml
* Update SUMMARY.md
* Add set -e
* Add vm
* Fix
* Add vm
* set -e
* Remove return 1 and add :
* Small revision
* Fix typo
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Indent
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Remove .exe in Windows
* Fix period with \.
* test
* check diff
* Revert "Merge branch 'book-cli' of https://github.com/chong-he/lighthouse into book-cli"
This reverts commit 314005d3f8, reversing
changes made to a007f61378.
* update
* update
* Remove echo diff
* Dockerize
* Remove `-ti`
* take ownership inside container
* fix mistake
* proper escaping, restore ownership afterwards
* try without taking ownership of repo
* update
* add diff for troubleshooting
* binary
* update using linux
* binary
* make file
* remove diff
* add diff
* update progressive balance help text
* Remove diff
---------
Co-authored-by: Mac L <mjladson@pm.me>
Co-authored-by: antondlr <anton@delaruelle.net>
