Commit Graph

39 Commits

Author SHA1 Message Date
Michael Sproul
c1a2238f1a Implement tree hash caching (#584)
* Implement basic tree hash caching

* Use spaces to indent top-level Cargo.toml

* Optimize BLS tree hash by hashing bytes directly

* Implement tree hash caching for validator registry

* Persist BeaconState tree hash cache to disk

* Address Paul's review comments
2019-11-05 15:46:52 +11:00
Paul Hauner
bcffe42712 Bulk signature verification (#507)
* Add basic block processing benches

* Start reviving state processing benches

* Fix old block builders

* Add optimization for faster pubkey add

* Tidy benches, add another

* Add extra block processing bench

* Start working on faster BLS scheme

* Add partially complete sig verify optimization

* Add .gitignore to state processing

* Add progress on faster signature verification

* Fix SignatureSet for fake_crypto

* Tidy attester slashings sig set

* Tidy bulk signature verifier

* Refactor signature sets to be cleaner

* Start threading SignatureStrategy through code

* Add (empty) test dir

* Move BenchingBlockBuilder

* Add initial block signature verification tests

* Add tests for bulk signature verification

* Start threading SignatureStrategy in block proc.

* Refactor per_block_processing errors

* Use sig set tuples instead of lists of two

* Remove dead code

* Thread VerifySignatures through per_block_processing

* Add bulk signature verification

* Introduce parallel bulk signature verification

* Expand state processing benches

* Fix additional compile errors

* Fix issue where par iter chunks is 0

* Update milagro_bls dep

* Remove debugs, code fragment in beacon chain

* Tidy, add comments to block sig verifier

* Fix various PR comments

* Add block_root option to per_block_processing

* Fix comment in block signature verifier

* Fix comments from PR review

* Remove old comment

* Fix comment
2019-08-29 11:34:25 +10:00
Paul Hauner
8cfa36fedd Publish ssz_types (and deps) to crates.io (#468)
* Rename `hashing` crate to `eth2_hashing`

* Add license, desc to eth2_hashing Cargo.toml

* Remove merkle root from eth2 hashing

* Remove old benches folder (zombied from old branch)

* Add docs to eth2_hashing

* Prepare tree_hash for publishing on crates.io

* Update deps to use crates.io instead of paths

* Update all crates to pull ssz from crates.io

* Remove cached_tree_hash, add patches to manifest

* Fix compile error in benches

* Remove unused code

* Fix fake_crypto compile error
2019-08-08 11:39:47 +10:00
Michael Sproul
a236003a7b Update to frozen spec ❄️ (v0.8.1) (#444)
* types: first updates for v0.8

* state_processing: epoch processing v0.8.0

* state_processing: block processing v0.8.0

* tree_hash_derive: support generics in SignedRoot

* types v0.8: update to use ssz_types

* state_processing v0.8: use ssz_types

* ssz_types: add bitwise methods and from_elem

* types: fix v0.8 FIXMEs

* ssz_types: add bitfield shift_up

* ssz_types: iterators and DerefMut for VariableList

* types,state_processing: use VariableList

* ssz_types: fix BitVector Decode impl

Fixed a typo in the implementation of ssz::Decode for BitVector, which caused it
to be considered variable length!

* types: fix test modules for v0.8 update

* types: remove slow type-level arithmetic

* state_processing: fix tests for v0.8

* op_pool: update for v0.8

* ssz_types: Bitfield difference length-independent

Allow computing the difference of two bitfields of different lengths.

* Implement compact committee support

* epoch_processing: committee & active index roots

* state_processing: genesis state builder v0.8

* state_processing: implement v0.8.1

* Further improve tree_hash

* Strip examples, tests from cached_tree_hash

* Update TreeHash, un-impl CachedTreeHash

* Update bitfield TreeHash, un-impl CachedTreeHash

* Update FixedLenVec TreeHash, unimpl CachedTreeHash

* Update update tree_hash_derive for new TreeHash

* Fix TreeHash, un-impl CachedTreeHash for ssz_types

* Remove fixed_len_vec, ssz benches

SSZ benches relied upon fixed_len_vec -- it is easier to just delete
them and rebuild them later (when necessary)

* Remove boolean_bitfield crate

* Fix fake_crypto BLS compile errors

* Update ef_tests for new v.8 type params

* Update ef_tests submodule to v0.8.1 tag

* Make fixes to support parsing ssz ef_tests

* `compact_committee...` to `compact_committees...`

* Derive more traits for `CompactCommittee`

* Flip bitfield byte-endianness

* Fix tree_hash for bitfields

* Modify CLI output for ef_tests

* Bump ssz crate version

* Update ssz_types doc comment

* Del cached tree hash tests from ssz_static tests

* Tidy SSZ dependencies

* Rename ssz_types crate to eth2_ssz_types

* validator_client: update for v0.8

* ssz_types: update union/difference for bit order swap

* beacon_node: update for v0.8, EthSpec

* types: disable cached tree hash, update min spec

* state_processing: fix slot bug in committee update

* tests: temporarily disable fork choice harness test

See #447

* committee cache: prevent out-of-bounds access

In the case where we tried to access the committee of a shard that didn't have a committee in the
current epoch, we were accessing elements beyond the end of the shuffling vector and panicking! This
commit adds a check to make the failure safe and explicit.

* fix bug in get_indexed_attestation and simplify

There was a bug in our implementation of get_indexed_attestation whereby
incorrect "committee indices" were used to index into the custody bitfield. The
bug was only observable in the case where some bits of the custody bitfield were
set to 1. The implementation has been simplified to remove the bug, and a test
added.

* state_proc: workaround for compact committees bug

https://github.com/ethereum/eth2.0-specs/issues/1315

* v0.8: updates to make the EF tests pass

* Remove redundant max operation checks.
* Always supply both messages when checking attestation signatures -- allowing
  verification of an attestation with no signatures.
* Swap the order of the fork and domain constant in `get_domain`, to match
  the spec.

* rustfmt

* ef_tests: add new epoch processing tests

* Integrate v0.8 into master (compiles)

* Remove unused crates, fix clippy lints

* Replace v0.6.3 tags w/ v0.8.1

* Remove old comment

* Ensure lmd ghost tests only run in release

* Update readme
2019-07-30 12:44:51 +10:00
John Adler
fec7168512 Fix lots of typos. 2019-07-26 15:26:06 -04:00
Michael Sproul
6bf6ba337d bls: as_hex_string for aggregate signatures/pubkeys 2019-06-11 18:00:05 +10:00
Paul Hauner
10a7936c37 Update BLS library 2019-06-03 13:26:22 +10:00
Paul Hauner
a0148b5aae Rename SSZ traits (Encodable -> Encode) 2019-05-13 15:12:19 +10:00
Paul Hauner
5550d14d62 Upgrade BLS to new SSZ 2019-05-06 15:03:20 +10:00
Paul Hauner
f65e981f6f Begin implementing cached hashing in types 2019-04-26 15:24:18 +10:00
Paul Hauner
10a5d2657c Encode bitfield as list not vector 2019-04-17 11:57:57 +10:00
Paul Hauner
f69b56ad60 Add new tree_hash crate project wide. 2019-04-16 14:25:43 +10:00
Paul Hauner
b8c4c3308a Update types to new tree_hash crate 2019-04-16 14:14:38 +10:00
Paul Hauner
f61db9cac8 Fix merge conflict in bls
Issue came about from Michael and Kirk doing simultaneous work.
2019-04-02 15:33:38 +11:00
Paul Hauner
206f651895 Merge pull request #293 from michaelsproul/operation-pool
Implement Operation Pool
2019-04-02 13:53:22 +11:00
Kirk Baird
fd2f9d0d15 Remove ssz encoding of length from; Signature, AggregateSiganture, PublicKey, SecretKey 2019-03-26 16:45:25 +11:00
Kirk Baird
864ef137ef Merge ssz little endian branch 2019-03-26 15:44:01 +11:00
Kirk Baird
116d995960 Merge current master and fix ssz-fuzzing 2019-03-26 14:53:50 +11:00
Kirk Baird
d76baa1cc1 Allow downloading of large tests from EF github and fix issues with serde 2019-03-25 18:02:37 +11:00
Michael Sproul
0bf8cb953e BLS: wrap AggregateSignature::add_aggregate 2019-03-25 12:44:30 +11:00
Kirk Baird
9cef6a5814 Finish implementing Darrens migrate-ssz-little-endian and add wrapper to check for ssz underflow 2019-03-18 18:11:46 +11:00
Paul Hauner
236b97476a Marge fixes to test_harness, add serdehex crate 2019-03-15 13:31:30 +11:00
Paul Hauner
123650efbb Merge pull request #290 from mjkeating/tree_hash_spec_update
Updated TreeHash logic as per revised spec
2019-03-13 14:04:42 +13:00
Paul Hauner
9057b436f3 Run rustfmt 2019-03-12 17:19:35 +11:00
Kirk Baird
c92f867cd8 Upgrade to signature scheme 0.6.0 2019-03-12 16:01:09 +11:00
mjkeating
d4f3bab68d Updated TreeHash logic as per revised spec 2019-03-08 15:24:07 -08:00
Paul Hauner
35ae1b6745 Add agg_pub to bls, add agg_sig.verify_multiple
- Adds a new-type wrapper for `AggregatePublicKey`, just like all the
other types.
- Adds the `verify_multiple` method to the `AggregateSignature` newtype,
as was introduced in a recent version of signature-schemes.
2019-03-03 11:10:38 +11:00
Kirk Baird
673c8ea388 Merge current master 2019-02-20 13:59:11 +11:00
Kirk Baird
9c4a1f1d1f Update to signature-scheme 0.5.2 2019-02-18 10:50:40 +11:00
mjkeating
6fa141181b Updated TreeHash to spec - added padding 2019-02-17 09:30:18 -08:00
Kirk Baird
977f3edfb6 Add domain to all signature funcitons, modify validate_proof_of_possession() 2019-02-15 13:58:14 +11:00
Paul Hauner
35c914baa6 Revert "Merge pull request #200 from sigp/new-structure"
This reverts commit d7a3545be1, reversing
changes made to 1da06c156c.
2019-02-14 12:09:18 +11:00
Paul Hauner
1d5ff4359a Strip out old code
All of these files have been moved to either:

- https://github.com/sigp/lighthouse-beacon
- https://github.com/sigp/lighthouse-validator
- https://github.com/sigp/lighthouse-common

For rationale, see: https://github.com/sigp/lighthouse/issues/197
2019-02-13 14:15:53 +11:00
Paul Hauner
90ae2298ab Impl serde::Serialize for all types 2019-01-26 08:25:31 +11:00
Paul Hauner
6b4c0fe1a3 Merge branch 'master' into lmd-ghost 2019-01-25 17:23:14 +11:00
Paul Hauner
5ef02688d5 Remove pre-2018 extern crate uses 2019-01-25 16:46:16 +11:00
Paul Hauner
ea523c8658 Merge spec crate into types 2019-01-25 13:52:21 +11:00
Kirk Baird
9c9b07c182 Add TreeHash trait to all types and structs 2019-01-25 12:22:56 +11:00
Paul Hauner
e16f9e0aec Rename beacon_chain/ -> eth2/ 2019-01-22 16:16:02 +11:00