add HTTP API to the beacon node to retrieve IL from the EL.
add IL service in the validator client to download the IL from the beacon node.
add logic to the beacon node to package the IL for the validator client.
add HTTP API to the beacon node to gossip signed ILs.
the validator client will sign the ILs from the beacon node and resubmit to the beacon node to gossip.
* Compute blob rpc limits in static block
* Fix min size
* Use MainnetEthSpec in rpc tests
* Revert MainnetEthSpec; add another constant for blob size minimal
* Fix db query and add some additional metrics
* fmt
* Update beacon_node/beacon_chain/src/metrics.rs
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
* Update beacon_node/beacon_chain/src/metrics.rs
Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
* Get blobs from EL.
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
* Avoid cloning blobs after fetching blobs.
* Address review comments and refactor code.
* Fix lint.
* Move blob computation metric to the right spot.
* Merge branch 'unstable' into das-fetch-blobs
* Merge branch 'unstable' into das-fetch-blobs
# Conflicts:
# beacon_node/beacon_chain/src/beacon_chain.rs
# beacon_node/beacon_chain/src/block_verification.rs
# beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
* Merge branch 'unstable' into das-fetch-blobs
# Conflicts:
# beacon_node/beacon_chain/src/beacon_chain.rs
* Gradual publication of data columns for supernodes.
* Recompute head after importing block with blobs from the EL.
* Fix lint
* Merge branch 'unstable' into das-fetch-blobs
* Use blocking task instead of async when computing cells.
* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs
* Merge remote-tracking branch 'origin/unstable' into das-fetch-blobs
* Fix semantic conflicts
* Downgrade error log.
* Merge branch 'unstable' into das-fetch-blobs
# Conflicts:
# beacon_node/beacon_chain/src/data_availability_checker.rs
# beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
# beacon_node/execution_layer/src/engine_api.rs
# beacon_node/execution_layer/src/engine_api/json_structures.rs
# beacon_node/network/src/network_beacon_processor/gossip_methods.rs
# beacon_node/network/src/network_beacon_processor/mod.rs
# beacon_node/network/src/network_beacon_processor/sync_methods.rs
* Merge branch 'unstable' into das-fetch-blobs
* Publish block without waiting for blob and column proof computation.
* Address review comments and refactor.
* Merge branch 'unstable' into das-fetch-blobs
* Fix test and docs.
* Comment cleanups.
* Merge branch 'unstable' into das-fetch-blobs
* Address review comments and cleanup
* Address review comments and cleanup
* Refactor to de-duplicate gradual publication logic.
* Add more logging.
* Merge remote-tracking branch 'origin/unstable' into das-fetch-blobs
# Conflicts:
# Cargo.lock
* Fix incorrect comparison on `num_fetched_blobs`.
* Implement gradual blob publication.
* Merge branch 'unstable' into das-fetch-blobs
* Inline `publish_fn`.
* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs
* Gossip verify blobs before publishing
* Avoid queries for 0 blobs and error for duplicates
* Gossip verified engine blob before processing them, and use observe cache to detect duplicates before publishing.
* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs
# Conflicts:
# beacon_node/network/src/network_beacon_processor/mod.rs
* Merge branch 'unstable' into das-fetch-blobs
* Fix invalid commitment inclusion proofs in blob sidecars created from EL blobs.
* Only publish EL blobs triggered from gossip block, and not RPC block.
* Downgrade gossip blob log to `debug`.
* Merge branch 'unstable' into das-fetch-blobs
* Merge branch 'unstable' into das-fetch-blobs
* Grammar
* Send `IDONTWANT` on publish to avoid downloading data we already have.
* Merge branch 'unstable' into send-idontwant-on-publish
* Move broadcast of `IDONTWANT` to after publishing.
* make `execution-endpoint` mandatory
* use parse_required instead
* make test pass
* Merge branch 'unstable' into make_ee_required
* fix test
* Merge branch 'unstable' into make_ee_required
* Fix cli help text
* Fix tests
* Merge branch 'unstable' into make_ee_required
* Add comment
* Clarification
* Merge remote-tracking branch 'origin/unstable' into make_ee_required
* Validator manager commands for standard key-manager APIs
* Merge latest unstable
* Fix Some in lib.rs
* Replace Arg::with_name with Arg::new
* Update takes_value
* Remove clap::App
* Change App to Command
* Add command in use
* Remove generic in ArgMatches
* Fix matches.get_flag
* Fixes
* fix error handling
* SetTrue in import
* Fix
* Fix builder-proposal flag (will delete the flag later)
* Minor fix
* Fix prefer_builder_proposals
* Remove unwrap
* Error handling from Michael
* Add cli help text
* Use None in import to simplify
* Delete unwrap
* Revert flags option
* Simplify help command code
* Remove flag header in move
* Merge remote-tracking branch 'origin/unstable' into pahor/validator-manager-standard-keystore
* Add log in VC when keystore is deleted
* Delete duplicated log when validator does not exist
* Simplify log code
* Rename remove to delete
* cargo-fmt
* Try to remove a function
* make-cli
* Error handling
* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm
* Update CLI hel text
* make-cli
* Fix checks
* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm
* Try to fix check errors
* Fix test
* Remove changes
* Update flag name
* CLI display order
* Move builde_proposals flag
* Add doc
* mdlint
* Update validator_manager/src/list_validators.rs
Co-authored-by: Mac L <mjladson@pm.me>
* Delete empty line
* Fix list
* Simplify delete
* Add support to delete more validators
* Fix test
* Rename response
* Add (s)
* Add test to delete multiple validators
* Book and cli
* Make cli
* Only log when keystore is deleted
* Revise deletion log
* Add validator pubkey to error message
* Merge import
* Thank you Mac
* Test
* Add flags
* Error handling for password
* make cli
* Merge remote-tracking branch 'origin/unstable' into vm
* make cli
* Fix test
* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm
* Fix test
* vm test
* Debug trait thank you Michael
* Fix test
* Merge branch 'unstable' into vm
* test
* testing
* Combine import validator(s)
* make cli
* Add requires
* Update book
* mdlint
* Only show import log when import is successful
* delete testing
* Test for standard format
* Test standard format
* Test
* fix builder_proposals flag
* Fix test for standard format
* Add requires
* Fix vm test
* make cli
* Remove flag header
* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm
* make cli
* Delete space
* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm
* Merge branch 'unstable' into vm
* Rename delete_validator to delete_validators
* Rearrange
* Remove pub in run function
* Fix grammar
* Apply suggestions from code review
Co-authored-by: Michael Sproul <micsproul@gmail.com>
* Remove description
* Merge branch 'vm' of https://github.com/chong-he/lighthouse into vm
* Close bracket
* make cli
* Revise list code and test
* Revise import flag
* make cli
* Comment out test
* Update vm test
* Simplify
* Merge remote-tracking branch 'origin/unstable' into vm
* make cli
* Add test
* Add password as a requirement for keystore file
* Correct flags in docs
* typo
* Better assert message in lookup sampling test
* Export status
* Merge remote-tracking branch 'sigp/unstable' into lookup-sampling-test-assert
* Drop unused
* Use slice
* Move reconstruction logic out of `overflow_lru_cache` to simplify the code and avoids having to pass `DataColumnsToPublish` around and blocking other processing.
* Publish reconstructed cells before recomputing head. Remove duplicate functions.
* Merge branch 'unstable' into non-blocking-reconstruction
* Merge branch 'unstable' into non-blocking-reconstruction
# Conflicts:
# beacon_node/beacon_chain/src/beacon_chain.rs
# beacon_node/beacon_chain/src/data_availability_checker.rs
# beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
# beacon_node/network/src/network_beacon_processor/sync_methods.rs
* Spawn a blocking task for reconstruction.
* Merge branch 'unstable' into non-blocking-reconstruction
# Conflicts:
# beacon_node/network/src/network_beacon_processor/mod.rs
* Fix fmt
* Merge branch 'unstable' into non-blocking-reconstruction
# Conflicts:
# beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
* Fix race condition by making check and mutation atomic as suggested by Lion. Also added error handling to reconstruction failure.
* Add reconstruction reason metric and more debug logging to da checker.
* Add comment and logging.
* Rename `NotRequired` to `NotStarted`.
* Remove extra character added.
