Squashed commit of the following:
commit 974b3359f8
Merge: ac205b7ba480309fb9
Author: Michael Sproul <michael@sigmaprime.io>
Date: Wed Jan 18 10:01:26 2023 +1100
Merge remote-tracking branch 'origin/unstable' into jemalloc
commit 480309fb96
Author: aliask <aliask@gmail.com>
Date: Tue Jan 17 05:13:49 2023 +0000
Fix some dead links in markdown files (#3885)
## Issue Addressed
No issue has been raised for these broken links.
## Proposed Changes
Update links with the new URLs for the same document.
## Additional Info
~The link for the [Lighthouse Development Updates](https://eepurl.com/dh9Lvb/) mailing list is also broken, but I can't find the correct link.~
Co-authored-by: Paul Hauner <paul@paulhauner.com>
commit b4d9fc03ee
Author: GeemoCandama <geemo@tutanota.com>
Date: Tue Jan 17 05:13:48 2023 +0000
add logging for starting request and receiving block (#3858)
## Issue Addressed
#3853
## Proposed Changes
Added `INFO` level logs for requesting and receiving the unsigned block.
## Additional Info
Logging for successfully publishing the signed block is already there. And seemingly there is a log for when "We realize we are going to produce a block" in the `start_update_service`: `info!(log, "Block production service started");
`. Is there anywhere else you'd like to see logging around this event?
Co-authored-by: GeemoCandama <104614073+GeemoCandama@users.noreply.github.com>
commit 9a970ce3a2
Author: David Theodore <prodigalsonsolutions@gmail.com>
Date: Tue Jan 17 05:13:47 2023 +0000
add better err reporting UnableToOpenVotingKeystore (#3781)
## Issue Addressed
#3780
## Proposed Changes
Add error reporting that notifies the node operator that the `voting_keystore_path` in their `validator_definitions.yml` file may be incorrect.
## Additional Info
There is more info in issue #3780
Co-authored-by: Paul Hauner <paul@paulhauner.com>
commit ac205b7bab
Merge: 93457d85bbf533c8e4
Author: Michael Sproul <michael@sigmaprime.io>
Date: Fri Nov 25 16:32:33 2022 +1100
Merge remote-tracking branch 'origin/unstable' into jemalloc
commit 93457d85b7
Author: Michael Sproul <michael@sigmaprime.io>
Date: Wed Nov 9 11:53:59 2022 +1100
Fix cargo-udeps
commit 6c42aef1b5
Author: Michael Sproul <micsproul@gmail.com>
Date: Tue Nov 8 19:12:19 2022 +1100
Fixups
commit f14b87bb88
Author: Michael Sproul <michael@sigmaprime.io>
Date: Tue Nov 8 16:28:16 2022 +1100
Update docs
commit 5005dc3b65
Author: Michael Sproul <michael@sigmaprime.io>
Date: Tue Nov 8 16:22:42 2022 +1100
Fix lcli
commit a082ba5904
Author: Michael Sproul <michael@sigmaprime.io>
Date: Tue Nov 8 16:17:10 2022 +1100
Remove check-consensus
commit 81441e9cea
Author: Michael Sproul <micsproul@gmail.com>
Date: Tue Nov 8 15:28:11 2022 +1100
Disable jemalloc on Windows
commit 41eac5d0c1
Author: Michael Sproul <micsproul@gmail.com>
Date: Tue Nov 8 13:46:17 2022 +1100
Compatibility with macOS
commit 69ecba7876
Author: Michael Sproul <michael@sigmaprime.io>
Date: Mon Nov 7 18:48:31 2022 +1100
Add jemalloc support
## Proposed Changes
I did some gardening 🌳 in our dependency tree:
- Remove duplicate versions of `warp` (git vs patch)
- Remove duplicate versions of lots of small deps: `cpufeatures`, `ethabi`, `ethereum-types`, `bitvec`, `nix`, `libsecp256k1`.
- Update MDBX (should resolve#3028). I tested and Lighthouse compiles on Windows 11 now.
- Restore `psutil` back to upstream
- Make some progress updating everything to rand 0.8. There are a few crates stuck on 0.7.
Hopefully this puts us on a better footing for future `cargo audit` issues, and improves compile times slightly.
## Additional Info
Some crates are held back by issues with `zeroize`. libp2p-noise depends on [`chacha20poly1305`](https://crates.io/crates/chacha20poly1305) which depends on zeroize < v1.5, and we can only have one version of zeroize because it's post 1.0 (see https://github.com/rust-lang/cargo/issues/6584). The latest version of `zeroize` is v1.5.4, which is used by the new versions of many other crates (e.g. `num-bigint-dig`). Once a new version of chacha20poly1305 is released we can update libp2p-noise and upgrade everything to the latest `zeroize` version.
I've also opened a PR to `blst` related to zeroize: https://github.com/supranational/blst/pull/111
## Proposed Changes
Lots of lint updates related to `flat_map`, `unwrap_or_else` and string patterns. I did a little more creative refactoring in the op pool, but otherwise followed Clippy's suggestions.
## Additional Info
We need this PR to unblock CI.
## Issue Addressed
Closes https://github.com/sigp/lighthouse/issues/2857
## Proposed Changes
Explicitly set GNU malloc's MMAP_THRESHOLD to 128KB, disabling dynamic adjustments. For rationale see the linked issue.
## Proposed Changes
Add `mallinfo2` behind a feature flag so that we can get accurate memory metrics during debugging. It can be enabled when building Lighthouse like so (so long as the platform supports it):
```
cargo install --path lighthouse --features "malloc_utils/mallinfo2"
```
## Proposed Changes
While investigating memory usage I noticed that the malloc metrics were going negative once they passed 2GiB. This is because the underlying `mallinfo` function returns a `i32`, and we were casting it straight to an `i64`, preserving the sign.
The long-term fix will be to move to `mallinfo2`, but it's still not yet widely available.
## Issue Addressed
NA
## Proposed Changes
I've noticed some of the SigP Prater nodes struggling on v1.4.0-rc.0. I suspect this is due to the changes in #2296. Specifically, the trade-off which lowered the memory footprint whilst increasing runtime on some functions.
Presently, this PR is documenting my testing on Prater.
## Additional Info
NA
## Issue Addressed
NA
## Proposed Changes
Modify the configuration of [GNU malloc](https://www.gnu.org/software/libc/manual/html_node/The-GNU-Allocator.html) to reduce memory footprint.
- Set `M_ARENA_MAX` to 4.
- This reduces memory fragmentation at the cost of contention between threads.
- Set `M_MMAP_THRESHOLD` to 2mb
- This means that any allocation >= 2mb is allocated via an anonymous mmap, instead of on the heap/arena. This reduces memory fragmentation since we don't need to keep growing the heap to find big contiguous slabs of free memory.
- ~~Run `malloc_trim` every 60 seconds.~~
- ~~This shaves unused memory from the top of the heap, preventing the heap from constantly growing.~~
- Removed, see: https://github.com/sigp/lighthouse/pull/2299#issuecomment-825322646
*Note: this only provides memory savings on the Linux (glibc) platform.*
## Additional Info
I'm going to close#2288 in favor of this for the following reasons:
- I've managed to get the memory footprint *smaller* here than with jemalloc.
- This PR seems to be less of a dramatic change than bringing in the jemalloc dep.
- The changes in this PR are strictly runtime changes, so we can create CLI flags which disable them completely. Since this change is wide-reaching and complex, it's nice to have an easy "escape hatch" if there are undesired consequences.
## TODO
- [x] Allow configuration via CLI flags
- [x] Test on Mac
- [x] Test on RasPi.
- [x] Determine if GNU malloc is present?
- I'm not quite sure how to detect for glibc.. This issue suggests we can't really: https://github.com/rust-lang/rust/issues/33244
- [x] Make a clear argument regarding the affect of this on CPU utilization.
- [x] Test with higher `M_ARENA_MAX` values.
- [x] Test with longer trim intervals
- [x] Add some stats about memory savings
- [x] Remove `malloc_trim` calls & code
