* fix lib.rs and tests.rs
* update decode.rs
* auto-delete in Cargo.lock
* delete milagro in cargo.toml
* remove milagro from makefile
* remove milagro from the name
* delete milagro in comment
* delete milagro in cargo.toml
* delete in /testing/ef_tests/cargo.toml
* delete milagro in the logical OR
* delete milagro in /lighthouse/src/main.rs
* delete milagro in /crypto/bls/tests/tests.rs
* delete milagro in comment
* delete milagro in /testing//ef_test/src//cases/bls_eth_aggregate_pubkeys.rs
* delete milagro
* delete more in lib.rs
* delete more in lib.rs
* delete more in lib.rs
* delete milagro in /crypto/bls/src/lib.rs
* delete milagro in crypto/bls/src/mod.rs
* delete milagro.rs
* Add mergify.yml.
* Abstract out CI jobs to a "success" job so that a change in the CI jobs don't require modification to mergify configuration on stable branch.
* Add new jobs to the `needs` list of `test-suite-success`.
* Set `batch_max_wait_time` to 60 s.
* Add cli.sh file
* update bash script
* update Makefile
* update
* modified test-suite
* fix path
* Fix cli.sh permissions
* update cmd
* cli_manual
* Revise to update
* Update directory in Github
* Correct cli.txt directory
* test old cli_manual
* change exit 1
* Update cli and makefile
* Move cli.sh
* remove files
* fix permission
* Indentation and revision
* Fixed permission
* Create new cli folder
* remove dummy
* put a dummy file
* Revise cli.sh
* comment
* function
* remove vm.md
* test make cli
* test
* testing
* testing
* update
* update
* test
* test
* add vm
* change back non-debug mode
* add exist and update for future debug
* revise
* remove troubleshooting part
* update
* add summary.md
* test
* test
* Update Makefile
Co-authored-by: Mac L <mjladson@pm.me>
* Update Makefile
Co-authored-by: Mac L <mjladson@pm.me>
* Remove help-cli.md and rearrange
* Remove help-cli.md
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* remove maxperf
* move then to same line as if
* Fix indent and echo file not found
* To be explicit in replacing the old file
* Add logging when there are changes
* Add local variables
* spacing
* remove cargo fmt
* update .md files
* Edit exit message to avoid confusion
* Remove am and add vm subcommands
* Add cargo-fmt
* Revise test-suite.yml
* Update SUMMARY.md
* Add set -e
* Add vm
* Fix
* Add vm
* set -e
* Remove return 1 and add :
* Small revision
* Fix typo
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Indent
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Remove .exe in Windows
* Fix period with \.
* test
* check diff
* linux commit
* Add cli.sh file
* update bash script
* update Makefile
* update
* modified test-suite
* fix path
* Fix cli.sh permissions
* update cmd
* cli_manual
* Revise to update
* Update directory in Github
* Correct cli.txt directory
* test old cli_manual
* change exit 1
* Update cli and makefile
* Move cli.sh
* remove files
* fix permission
* Indentation and revision
* Fixed permission
* Create new cli folder
* remove dummy
* put a dummy file
* Revise cli.sh
* comment
* function
* remove vm.md
* test make cli
* test
* testing
* testing
* update
* update
* test
* test
* add vm
* change back non-debug mode
* add exist and update for future debug
* revise
* remove troubleshooting part
* update
* add summary.md
* test
* test
* Update Makefile
Co-authored-by: Mac L <mjladson@pm.me>
* Update Makefile
Co-authored-by: Mac L <mjladson@pm.me>
* Remove help-cli.md and rearrange
* Remove help-cli.md
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* remove maxperf
* move then to same line as if
* Fix indent and echo file not found
* To be explicit in replacing the old file
* Add logging when there are changes
* Add local variables
* spacing
* remove cargo fmt
* Edit exit message to avoid confusion
* update .md files
* Remove am and add vm subcommands
* Add cargo-fmt
* Revise test-suite.yml
* Update SUMMARY.md
* Add set -e
* Add vm
* Fix
* Add vm
* set -e
* Remove return 1 and add :
* Small revision
* Fix typo
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Indent
* Update scripts/cli.sh
Co-authored-by: Mac L <mjladson@pm.me>
* Remove .exe in Windows
* Fix period with \.
* test
* check diff
* Revert "Merge branch 'book-cli' of https://github.com/chong-he/lighthouse into book-cli"
This reverts commit 314005d3f8, reversing
changes made to a007f61378.
* update
* update
* Remove echo diff
* Dockerize
* Remove `-ti`
* take ownership inside container
* fix mistake
* proper escaping, restore ownership afterwards
* try without taking ownership of repo
* update
* add diff for troubleshooting
* binary
* update using linux
* binary
* make file
* remove diff
* add diff
* update progressive balance help text
* Remove diff
---------
Co-authored-by: Mac L <mjladson@pm.me>
Co-authored-by: antondlr <anton@delaruelle.net>
## Issue Addressed
Addresses the recent CI failures caused by caching `blst` for the wrong CPU type.
## Proposed Changes
- Use `FEATURES: jemalloc,portable` when building Lighthouse & `lcli` in tests
- Add a new `TEST_FEATURES` and set to `portable` for all CI test jobs.
- Updated Makefiles to read the `TEST_FEATURES` environment variable, and default to none.
## Issue Addressed
Related to https://github.com/sigp/lighthouse/issues/4676.
Deneb-specifc CI code to be removed before merging to `unstable`. Dot not merge until we're ready to merge into `unstable`, as we may need to release deneb docker images before merging.
Keep in mind that most of the changes in the below PR (to `unstable`) have already
been merged to `deneb-free-blobs`, so merging `deneb-free-blobs` into `unstable` would include those changes - it would be ok if the release runners are ready, otherwise we may want to exclude them before merging.
- https://github.com/sigp/lighthouse/pull/4592
## Issue Addressed
CI is currently blocked by persistently failing integration tests.
## Proposed Changes
Use latest Nethermind release and apply the appropriate fixes as there have been breaking changes.
Also increase the timeout since I had some local timeouts.
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
Co-authored-by: antondlr <anton@delaruelle.net>
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
Attempting to improve our CI speeds as its recently been a pain point.
Major changes:
- Use a github action to pull stable/nightly rust rather than building it each run
- Shift test suite to `nexttest` https://github.com/nextest-rs/nextest for CI
UPDATE:
So I've iterated on some changes, and although I think its still not optimal I think this is a good base to start from. Some extra things in this PR:
- Shifted where we pull rust from. We're now using this thing: https://github.com/moonrepo/setup-rust . It's got some interesting cache's built in, but was not seeing the gains that Jimmy managed to get. In either case tho, it can pull rust, cargofmt, clippy, cargo nexttest all in < 5s. So I think it's worthwhile.
- I've grouped a few of the check-like tests into a single test called `code-test`. Although we were using github runners in parallel which may be faster, it just seems wasteful. There were like 4-5 tests, where we would pull lighthouse, compile it, then run an action, like clippy, cargo-audit or fmt. I've grouped these into a single action, so we only compile lighthouse once, then in each step we run the checks. This avoids compiling lighthouse like 5 times.
- Ive made doppelganger tests run on our local machines to avoid pulling foundry, building and making lcli which are all now baked into the images.
- We have sccache and do not incremental compile lighthouse
Misc bonus things:
- Cargo update
- Fix web3 signer openssl keys which is required after a cargo update
- Use mock_instant in an LRU cache test to avoid non-deterministic test
- Remove race condition in building web3signer tests
There's still some things we could improve on. Such as downloading the EF tests every run and the web3-signer binary, but I've left these to be out of scope of this PR. I think the above are meaningful improvements.
Co-authored-by: Paul Hauner <paul@paulhauner.com>
Co-authored-by: realbigsean <seananderson33@gmail.com>
Co-authored-by: antondlr <anton@delaruelle.net>
## Issue Addressed
I noticed a node.js version warning on our CI, and thought about updating the version to get rid of this warning, but then realized we may not need node.js anymore now we're using `anvil` instead of `ganache`.
> The following actions uses node12 which is deprecated and will be forced to run on node16: actions/setup-node@v2. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
* remove protoc and token from network tests github action
* delete unused beacon chain methods
* downgrade writing blobs to store log
* reduce diff in block import logic
* remove some todo's and deneb built in network
* remove unnecessary error, actually use some added metrics
* remove some metrics, fix missing components on publish funcitonality
* fix status tests
* rename sidecar by root to blobs by root
* clean up some metrics
* remove unnecessary feature gate from attestation subnet tests, clean up blobs by range response code
* pawan's suggestion in `protocol_info`, peer score in matching up batch sync block and blobs
* fix range tests for deneb
* pub block and blob db cache behind the same mutex
* remove unused errs and an empty file
* move sidecar trait to new file
* move types from payload to eth2 crate
* update comment and add flag value name
* make function private again, remove allow unused
* use reth rlp for tx decoding
* fix compile after merge
* rename kzg commitments
* cargo fmt
* remove unused dep
* Update beacon_node/execution_layer/src/lib.rs
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
* Update beacon_node/beacon_processor/src/lib.rs
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
* pawan's suggestiong for vec capacity
* cargo fmt
* Revert "use reth rlp for tx decoding"
This reverts commit 5181837d81.
* remove reth rlp
---------
Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
## Issue Addressed
Upgrade libp2p to v0.52
## Proposed Changes
- **Workflows**: remove installation of `protoc`
- **Book**: remove installation of `protoc`
- **`Dockerfile`s and `cross`**: remove custom base `Dockerfile` for cross since it's no longer needed. Remove `protoc` from remaining `Dockerfiles`s
- **Upgrade `discv5` to `v0.3.1`:** we have some cool stuff in there: no longer needs `protoc` and faster ip updates on cold start
- **Upgrade `prometheus` to `0.21.0`**, now it no longer needs encoding checks
- **things that look like refactors:** bunch of api types were renamed and need to be accessed in a different (clearer) way
- **Lighthouse network**
- connection limits is now a behaviour
- banned peers no longer exist on the swarm level, but at the behaviour level
- `connection_event_buffer_size` now is handled per connection with a buffer size of 4
- `mplex` is deprecated and was removed
- rpc handler now logs the peer to which it belongs
## Additional Info
Tried to keep as much behaviour unchanged as possible. However, there is a great deal of improvements we can do _after_ this upgrade:
- Smart connection limits: Connection limits have been checked only based on numbers, we can now use information about the incoming peer to decide if we want it
- More powerful peer management: Dial attempts from other behaviours can be rejected early
- Incoming connections can be rejected early
- Banning can be returned exclusively to the peer management: We should not get connections to banned peers anymore making use of this
- TCP Nat updates: We might be able to take advantage of confirmed external addresses to check out tcp ports/ips
Co-authored-by: Age Manning <Age@AgeManning.com>
Co-authored-by: Akihito Nakano <sora.akatsuki@gmail.com>
## Issue Addressed
NA
## Proposed Changes
Carries on from #4115, with the following modifications:
1. Self-hosted runners are only enabled if `github.repository == sigp/lighthouse`.
- This allows forks to still have Github-hosted CI.
- This gives us a method to switch back to Github-runners if we have extended downtime on self-hosted.
1. Does not remove any existing dependency builds for Github-hosted runners (e.g., installing the latest Rust).
1. Adds the `WATCH_HOST` environment variable which defines where we expect to find the postgres db in the `watch` tests. This should be set to `host.docker.internal` for the tests to pass on self-hosted runners.
## Additional Info
NA
Co-authored-by: antondlr <anton@delaruelle.net>
## Issue Addressed
Speed up CI by installing foundry with Github action instead of building Anvil from source.
Building anvil from source on GItHub hosted runners currently takes about 10 mins. Using the `foundry-toolchain` action to install only takes about 2 seconds.
