Implement freezer database (#508)

* Implement freezer database for state vectors

* Improve BeaconState safe accessors

And fix a bug in the compact committees accessor.

* Banish dodgy type bounds back to gRPC

* Clean up

* Switch to exclusive end points in chunked vec

* Cleaning up and start of tests

* Randao fix, more tests

* Fix unsightly hack

* Resolve test FIXMEs

* Config file support

* More clean-ups, migrator beginnings

* Finish migrator, integrate into BeaconChain

* Fixups

* Fix store tests

* Fix BeaconChain tests

* Fix LMD GHOST tests

* Address review comments, delete 'static bounds

* Cargo format

* Address review comments

* Fix LMD ghost tests

* Update to spec v0.9.0

* Update to v0.9.1

* Bump spec tags for v0.9.1

* Formatting, fix CI failures

* Resolve accidental KeyPair merge conflict

* Document new BeaconState functions

* Fix incorrect cache drops in `advance_caches`

* Update fork choice for v0.9.1

* Clean up some FIXMEs

* Fix a few docs/logs

* Update for new builder paradigm, spec changes

* Freezer DB integration into BeaconNode

* Cleaning up

* This works, clean it up

* Cleanups

* Fix and improve store tests

* Refine store test

* Delete unused beacon_chain_builder.rs

* Fix CLI

* Store state at split slot in hot database

* Make fork choice lookup fast again

* Store freezer DB split slot in the database

* Handle potential div by 0 in chunked_vector

* Exclude committee caches from freezer DB

* Remove FIXME about long-running test
This commit is contained in:
Michael Sproul
2019-11-27 10:54:46 +11:00
committed by Paul Hauner
parent a514968155
commit bf2eeae3f2
36 changed files with 2486 additions and 580 deletions

View File

@@ -1,4 +1,5 @@
use super::*;
use crate::impls::beacon_state::{get_full_state, store_full_state};
use crate::metrics;
use db_key::Key;
use leveldb::database::kv::KV;
@@ -6,14 +7,10 @@ use leveldb::database::Database;
use leveldb::error::Error as LevelDBError;
use leveldb::options::{Options, ReadOptions, WriteOptions};
use std::path::Path;
use std::sync::Arc;
/// A wrapped leveldb database.
#[derive(Clone)]
pub struct LevelDB {
// Note: this `Arc` is only included because of an artificial constraint by gRPC. Hopefully we
// can remove this one day.
db: Arc<Database<BytesKey>>,
db: Database<BytesKey>,
}
impl LevelDB {
@@ -23,7 +20,7 @@ impl LevelDB {
options.create_if_missing = true;
let db = Arc::new(Database::open(path, options)?);
let db = Database::open(path, options)?;
Ok(Self { db })
}
@@ -111,6 +108,24 @@ impl Store for LevelDB {
.delete(self.write_options(), column_key)
.map_err(Into::into)
}
/// Store a state in the store.
fn put_state<E: EthSpec>(
&self,
state_root: &Hash256,
state: &BeaconState<E>,
) -> Result<(), Error> {
store_full_state(self, state_root, state)
}
/// Fetch a state from the store.
fn get_state<E: EthSpec>(
&self,
state_root: &Hash256,
_: Option<Slot>,
) -> Result<Option<BeaconState<E>>, Error> {
get_full_state(self, state_root)
}
}
impl From<LevelDBError> for Error {