mirror of
https://github.com/sigp/lighthouse.git
synced 2026-03-06 10:11:44 +00:00
Tune GNU malloc (#2299)
## Issue Addressed NA ## Proposed Changes Modify the configuration of [GNU malloc](https://www.gnu.org/software/libc/manual/html_node/The-GNU-Allocator.html) to reduce memory footprint. - Set `M_ARENA_MAX` to 4. - This reduces memory fragmentation at the cost of contention between threads. - Set `M_MMAP_THRESHOLD` to 2mb - This means that any allocation >= 2mb is allocated via an anonymous mmap, instead of on the heap/arena. This reduces memory fragmentation since we don't need to keep growing the heap to find big contiguous slabs of free memory. - ~~Run `malloc_trim` every 60 seconds.~~ - ~~This shaves unused memory from the top of the heap, preventing the heap from constantly growing.~~ - Removed, see: https://github.com/sigp/lighthouse/pull/2299#issuecomment-825322646 *Note: this only provides memory savings on the Linux (glibc) platform.* ## Additional Info I'm going to close #2288 in favor of this for the following reasons: - I've managed to get the memory footprint *smaller* here than with jemalloc. - This PR seems to be less of a dramatic change than bringing in the jemalloc dep. - The changes in this PR are strictly runtime changes, so we can create CLI flags which disable them completely. Since this change is wide-reaching and complex, it's nice to have an easy "escape hatch" if there are undesired consequences. ## TODO - [x] Allow configuration via CLI flags - [x] Test on Mac - [x] Test on RasPi. - [x] Determine if GNU malloc is present? - I'm not quite sure how to detect for glibc.. This issue suggests we can't really: https://github.com/rust-lang/rust/issues/33244 - [x] Make a clear argument regarding the affect of this on CPU utilization. - [x] Test with higher `M_ARENA_MAX` values. - [x] Test with longer trim intervals - [x] Add some stats about memory savings - [x] Remove `malloc_trim` calls & code
This commit is contained in:
@@ -1,5 +1,5 @@
|
||||
use clap::ArgMatches;
|
||||
use clap_utils::BAD_TESTNET_DIR_MESSAGE;
|
||||
use clap_utils::{flags::DISABLE_MALLOC_TUNING_FLAG, BAD_TESTNET_DIR_MESSAGE};
|
||||
use client::{ClientConfig, ClientGenesis};
|
||||
use directory::{DEFAULT_BEACON_NODE_DIR, DEFAULT_NETWORK_DIR, DEFAULT_ROOT_DIR};
|
||||
use eth2_libp2p::{multiaddr::Protocol, Enr, Multiaddr, NetworkConfig, PeerIdSerialized};
|
||||
@@ -156,6 +156,11 @@ pub fn get_config<E: EthSpec>(
|
||||
);
|
||||
}
|
||||
|
||||
// Do not scrape for malloc metrics if we've disabled tuning malloc as it may cause panics.
|
||||
if cli_args.is_present(DISABLE_MALLOC_TUNING_FLAG) {
|
||||
client_config.http_metrics.allocator_metrics_enabled = false;
|
||||
}
|
||||
|
||||
/*
|
||||
* Eth1
|
||||
*/
|
||||
|
||||
Reference in New Issue
Block a user