#6296: Deterministic RNG in peer DAS publish block tests
Made test functions to call publish-block APIs with true for the deterministic RNG boolean parameter while production code with false. This will deterministically shuffle columns for unit tests under broadcast_validation_tests.rs.
Having merged the drop-headtracker PR we now have a DB schema change in `unstable` compared to `release-v7.0.0`:
- https://github.com/sigp/lighthouse/pull/6744
There is a DB downgrade available, however this needs to be applied manually and it's usually a bit of a hassle.
This PR bumps the version on `unstable` to `v7.1.0-beta.0` _without_ actually cutting a `v7.1.0-beta.0` release, so that we can tell at a glance which schema version a node is using.
#7153#7146#7147#7148 -> Thanks to @ackintosh
This PR does the following:
1. Disable logging to file when using either `--logfile-max-number 0` or `--logfile-max-size 0`. Note that disabling the log file in this way will also disable `discv5` and `libp2p` logging.
1. `discv5` and `libp2p` logging will be disabled by default unless running `beacon_node` or `boot_node`. This also should fix the VC panic we were seeing.
1. Removes log rotation and compression from `libp2p` and `discv5` logs. It is now limited to 1 file and will rotate based on the value of the `--logfile-max-size` flag. We could potentially add flags specifically to control the size/number of these, however I felt a single log file was sufficient. Perhaps @AgeManning has opinions about this?
1. Removes all dependency logging and references to `dep_log`.
1. Introduces workspace filtering to file and stdout. This explicitly allows logs from members of the Lighthouse workspace, disallowing all others. It uses a proc macro which pulls the member list from cargo metadata at compile time. This might be over-engineered but my hope is that this list will not require maintenance.
1. Unifies file and stdout JSON format. With slog, the formats were slightly different. @threehrsleep worked to maintain that format difference, to ensure there was no breaking changes. If these format differences are actually problematic we can restore it, however I felt the added complexity wasn't worth it.
1. General code improvements and cleanup.
Cross builds were failing since:
- https://github.com/sigp/lighthouse/pull/7086
This seems to have been due to a regression upstream in `ring` which is noted in the v0.17.14 release notes. I'm hoping that updating remedies it.
> Compatibility with GNU binutils 2.29 (used on Amazon Linux 2), and probably even earlier versions, was restored. It is expected that ring 0.17.14 will build on all the systems that 0.17.12 would build on.
https://github.com/briansmith/ring/blob/main/RELEASES.md#version-01714-2025-03-11
I feel it's preferable to do this explicitly by updating the revision on `Cargo.toml` rather than implicitly by letting `Cargo.lock` control the revision of the branch.
- #6452 (partially)
Remove dependencies on `store` and `lighthouse_network` from `eth2`. This was achieved as follows:
- depend on `enr` and `multiaddr` directly instead of using `lighthouse_network`'s reexports.
- make `lighthouse_network` responsible for converting between API and internal types.
- in two cases, remove complex internal types and use the generic `serde_json::Value` instead - this is not ideal, but should be fine for now, as this affects two internal non-spec endpoints which are meant for debugging, unstable, and subject to change without notice anyway. Inspired by #6679. The alternative is to move all relevant types to `eth2` or `types` instead - what do you think?
Resolves#7091
The `prometheus` crate pulls in `protobuf 2.x` which fails cargo audit. We actually dont use any `protobuf` related features in LH. By disabling default features for `prometheus`, we no longer pull in the `protobuf` crate
We forked `gossipsub` into the lighthouse repo sometime ago so that we could iterate quicker on implementing back pressure and IDONTWANT.
Meanwhile we have pushed all our changes upstream and we are now the main maintainers of `rust-libp2p` this allows us to use upstream `gossipsub` again.
Nonetheless we still use our forked repo to give us freedom to experiment with features before submitting them upstream
NA
Bumps the `ethereum_ssz` version, along with other crates that share the dep.
Primarily, this give us bitfields which can store 128 bytes on the stack before allocating, rather than 32 bytes (https://github.com/sigp/ethereum_ssz/pull/38). The validator count has increase massively since we set it at 32 bytes, so aggregation bitfields (et al) now require a heap allocation. This new value of 128 should get us to ~2m active validators.
Currently we have very poor coverage of range sync with unit tests. With the event driven test framework we could cover much more ground and be confident when modifying the code.
Add two basic cases:
- Happy path, complete a finalized sync for 2 epochs
- Post-PeerDAS case where we start without enough custody peers and later we find enough
⚠️ If you have ideas for more test cases, please let me know! I'll write them
Run `cargo update` to address [RUSTSEC-2025-0004](https://rustsec.org/advisories/RUSTSEC-2025-0004), a vulnerability in `openssl`. I don't think we are affected, but this PR is required for us to pass `cargo audit` and unblock CI.
Update cargo dependencies while keeping `rust_eth_kzg` pinned to `0.5.1` due to the regression described in:
- https://github.com/sigp/lighthouse/pull/6608
The changes from that PR were not sufficient to actually pin the dependencies of `rust_eth_kzg`, because the dependencies from the workspace Cargo.toml file were not being used anywhere. To fix this, I've added them as explicit dependencies in `crypto/kzg/Cargo.toml`. With this change, `cargo update` no longer tries to update them.
* Use reqwest eventsource for get_events api
* await for Event::Open before returning stream
* fmt
* Merge branch 'unstable' into sse-client-fix
* Ignore lint
* Add cli flag for HTTP API token path (VC)
* Add http_token_path_flag test
* Add pre-check for directory case & Fix test utils
* Update docs
* Apply review: move http_token_path into validator_http_api config
* Lint
* Make diff lesser to replace PK_FILENAME
* Merge branch 'unstable' into feature/cli-token-path
* Applt review: help_vc.md
Co-authored-by: chonghe <44791194+chong-he@users.noreply.github.com>
* Fix help for cli
* Fix issues on ci
* Merge branch 'unstable' into feature/cli-token-path
* Merge branch 'unstable' into feature/cli-token-path
* Merge branch 'unstable' into feature/cli-token-path
* Merge branch 'unstable' into feature/cli-token-path
* Remove ZeroizeString in favour of Zeroizing<String>
* cargo fmt
* remove unrelated line that slipped in
* Update beacon_node/store/Cargo.toml
thanks michael!
Co-authored-by: Michael Sproul <micsproul@gmail.com>
* Merge branch 'unstable' into remove-zeroizedstring
