Tree hash cache arena (#836)

* Start adding interop genesis state to lcli

* Use more efficient method to generate genesis state

* Remove duplicate int_to_bytes32

* Add lcli command to change state genesis time

* Add option to allow VC to start with unsynced BN

* Set VC to do parallel key loading

* Don't default to dummy eth1 backend

* Add endpoint to dump operation pool

* Add metrics for op pool

* Remove state clone for slot notifier

* Add mem size approximation for tree hash cache

* Avoid cloning tree hash when getting head

* Avoid cloning tree hash when getting head

* Add working arena-based cached tree hash

* Add another benchmark

* Add pre-allocation for caches

* Make cache nullable

* Fix bugs in cache tree hash

* Add validator tree hash optimization

* Optimize hash_concat

* Make hash32_concat return fixed-len array

* Fix failing API tests

* Add new beacon state cache struct

* Add validator-specific cache

* Separate list and values arenas

* Add parallel validator registry hashing

* Remove MultiTreeHashCache

* Remove cached tree hash macro

* Fix failing tree hash test

* Address Michael's comments

* Add CachedTreeHash impl for ef tests

* Fix messy merge conflict

* Rename cache struct, add comments

* Rename cache struct, add comments

* Remove unnecessary mutability

* Wrap iter in result

* Tidy cached tree hash

* Address Michael comments

* Address more comments

* Use ring::Context
This commit is contained in:
Paul Hauner
2020-02-07 12:42:49 +11:00
committed by GitHub
parent f267bf2afe
commit c3182e3c1c
20 changed files with 1341 additions and 378 deletions

View File

@@ -1,12 +1,13 @@
mod cache;
mod cache_arena;
mod impls;
mod multi_cache;
#[cfg(test)]
mod test;
pub type CacheArena = cache_arena::CacheArena<Hash256>;
pub use crate::cache::TreeHashCache;
pub use crate::impls::int_log;
pub use crate::multi_cache::MultiTreeHashCache;
use ethereum_types::H256 as Hash256;
use tree_hash::TreeHash;
@@ -19,13 +20,26 @@ pub enum Error {
CannotShrink,
/// Cache is inconsistent with the list of dirty indices provided.
CacheInconsistent,
CacheArenaError(cache_arena::Error),
/// Unable to find left index in Merkle tree.
MissingLeftIdx(usize),
}
impl From<cache_arena::Error> for Error {
fn from(e: cache_arena::Error) -> Error {
Error::CacheArenaError(e)
}
}
/// Trait for types which can make use of a cache to accelerate calculation of their tree hash root.
pub trait CachedTreeHash<Cache>: TreeHash {
/// Create a new cache appropriate for use with values of this type.
fn new_tree_hash_cache() -> Cache;
fn new_tree_hash_cache(&self, arena: &mut CacheArena) -> Cache;
/// Update the cache and use it to compute the tree hash root for `self`.
fn recalculate_tree_hash_root(&self, cache: &mut Cache) -> Result<Hash256, Error>;
fn recalculate_tree_hash_root(
&self,
arena: &mut CacheArena,
cache: &mut Cache,
) -> Result<Hash256, Error>;
}