Commit Graph

5417 Commits

Author SHA1 Message Date
Emilia Hane
4de523fb75 fixup! Allow user to set an epoch margin for pruning 2023-02-08 11:44:40 +01:00
Emilia Hane
1812301c9c Allow user to set an epoch margin for pruning 2023-02-08 11:44:40 +01:00
Emilia Hane
d7fc24a9d5 Plug in running blob pruning in migrator, related bug fixes and add todos 2023-02-08 11:44:40 +01:00
Emilia Hane
d1b75e281f Fix typo 2023-02-08 11:44:39 +01:00
Emilia Hane
0bdc291490 Only store non-empty orphaned blobs 2023-02-08 11:44:39 +01:00
Emilia Hane
caa04db58a Run prune blobs on migrator thread 2023-02-08 11:44:39 +01:00
Emilia Hane
a875bec5f2 Fix blobs store bug 2023-02-08 11:44:39 +01:00
Emilia Hane
3bede06c9b Fix typo 2023-02-08 11:44:38 +01:00
Emilia Hane
54699f808c fixup! Clarify hybrid blob prune solution and fix error handling 2023-02-08 11:44:38 +01:00
Emilia Hane
83a9520761 Clarify hybrid blob prune solution and fix error handling 2023-02-08 11:44:38 +01:00
Emilia Hane
74172ed160 Ignore IDE file 2023-02-08 11:44:38 +01:00
Emilia Hane
3d93dad0e2 Fix type bug
Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2023-02-08 11:44:37 +01:00
Emilia Hane
44ec331452 fixup! Simplify conceptual design 2023-02-08 11:44:37 +01:00
Emilia Hane
20567750c1 fixup! Simplify conceptual design 2023-02-08 11:44:37 +01:00
Emilia Hane
7103a257ce Simplify conceptual design 2023-02-08 11:44:37 +01:00
Emilia Hane
0d13932663 Fix epoch constructor misconception 2023-02-08 11:44:37 +01:00
Emilia Hane
b5abfe620a Convert epochs_per_blob_prune to Epoch once 2023-02-08 11:44:36 +01:00
Emilia Hane
fb2ce909f6 Avoid repeteadly updating blob info for multiple head candidates 2023-02-08 11:44:36 +01:00
Emilia Hane
d58a30b3de fixup! Store orphan block roots 2023-02-08 11:44:36 +01:00
Emilia Hane
6346c30158 Enable skipping blob pruning at each epoch 2023-02-08 11:44:35 +01:00
Emilia Hane
2f565d25b2 Prune blobs in bg after canonical head update 2023-02-08 11:44:35 +01:00
Emilia Hane
8752deeced Store orphan block roots 2023-02-08 11:44:35 +01:00
Emilia Hane
c7f53a9062 Delete blobs that conflict with finalization 2023-02-08 11:44:34 +01:00
Emilia Hane
94aa2cef67 Log info loaded from disk 2023-02-08 11:44:34 +01:00
Emilia Hane
a2b8c6ee69 Save fetching state for blobs pruning 2023-02-08 11:44:33 +01:00
Emilia Hane
6f5ca02ac9 Improve syntax
Co-authored-by: Michael Sproul <micsproul@gmail.com>
2023-02-08 11:44:33 +01:00
Emilia Hane
667cca5cf2 Fix try_prune_blobs to use state root 2023-02-08 11:44:33 +01:00
Emilia Hane
d67468d737 Prune blobs on migration in addition to start-up 2023-02-08 11:44:32 +01:00
Emilia Hane
82ffec378a Fix typo
Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2023-02-08 11:44:32 +01:00
Emilia Hane
ce2db355de Fix rebase conflict 2023-02-08 11:44:32 +01:00
Emilia Hane
a211e6afee Fix rebase conflict 2023-02-08 11:44:31 +01:00
Emilia Hane
28e1e635c3 Fix rebase conflict 2023-02-08 11:44:31 +01:00
Emilia Hane
d3b94d8617 fixup! Prune blobs before data availability breakpoint 2023-02-08 11:44:31 +01:00
Emilia Hane
934f3ab587 Remove inaccurate guess for db index 2023-02-08 11:44:31 +01:00
Emilia Hane
d21c66ddf4 fixup! Plug in pruning of blobs into app 2023-02-08 11:44:31 +01:00
Emilia Hane
b88d888145 fixup! Plug in pruning of blobs into app 2023-02-08 11:44:30 +01:00
Emilia Hane
2a41f25d68 fixup! Prune blobs before data availability breakpoint 2023-02-08 11:44:30 +01:00
Emilia Hane
fe0c911402 Plug in pruning of blobs into app 2023-02-08 11:44:30 +01:00
Emilia Hane
7bf88c2336 Prune blobs before data availability breakpoint 2023-02-08 11:44:30 +01:00
Emilia Hane
e2a6da4274 Boiler plate code for blobs pruning 2023-02-08 11:44:20 +01:00
realbigsean
8661477675 use hex decode instead of parse 2023-02-07 21:51:19 -05:00
Michael Sproul
1cae98856c Update dependencies (#3946)
## Issue Addressed

Resolves the cargo-audit failure caused by https://rustsec.org/advisories/RUSTSEC-2023-0010.

I also removed the ignore for `RUSTSEC-2020-0159` as we are no longer using a vulnerable version of `chrono`. We still need the other ignore for `time 0.1` because we depend on it via `sloggers -> chrono -> time 0.1`.
2023-02-08 02:18:54 +00:00
Divma
ceb986549d Self rate limiting dev flag (#3928)
## Issue Addressed
Adds self rate limiting options, mainly with the idea to comply with peer's rate limits in small testnets

## Proposed Changes
Add a hidden flag `self-limiter` this can take no value, or customs values to configure quotas per protocol

## Additional Info
### How to use
`--self-limiter` will turn on the self rate limiter applying the same params we apply to inbound requests (requests from other peers)
`--self-limiter "beacon_blocks_by_range:64/1"` will turn on the self rate limiter for ALL protocols, but change the quota for bbrange to 64 requested blocks per 1 second.
`--self-limiter "beacon_blocks_by_range:64/1;ping:1/10"` same as previous one, changing the quota for ping as well.

### Caveats
- The rate limiter is either on or off for all protocols. I added the custom values to be able to change the quotas per protocol so that some protocols can be given extremely loose or tight quotas. I think this should satisfy every need even if we can't technically turn off rate limits per protocol.
- This reuses the rate limiter struct for the inbound requests so there is this ugly part of the code in which we need to deal with the inbound only protocols (light client stuff) if this becomes too ugly as we add lc protocols, we might want to split the rate limiters. I've checked this and looks doable with const generics to avoid so much code duplication

### Knowing if this is on
```
Feb 06 21:12:05.493 DEBG Using self rate limiting params         config: OutboundRateLimiterConfig { ping: 2/10s, metadata: 1/15s, status: 5/15s, goodbye: 1/10s, blocks_by_range: 1024/10s, blocks_by_root: 128/10s }, service: libp2p_rpc, service: libp2p
```
2023-02-08 02:18:53 +00:00
Nazar Hussain
7934485aef Update the docker build to include features based images (#3875)
## Proposed Changes

There are some features that are enabled/disabled with the `FEATURES` env variable. This PR would introduce a pattern to introduce docker images based on those features. This can be useful later on to have specific images for some experimental features in the future.

## Additional Info

We at Lodesart need to have `minimal` spec support for some cross-client network testing. To make it efficient on the CI, we tend to use minimal preset.
2023-02-08 02:18:51 +00:00
Diva M
493784366f self rate limiting 2023-02-07 13:00:35 -05:00
realbigsean
a42d07592c fix compilation issues after merge 2023-02-07 12:33:29 -05:00
realbigsean
26a296246d Merge branch 'capella' of https://github.com/sigp/lighthouse into eip4844
# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	beacon_node/beacon_chain/src/block_verification.rs
#	beacon_node/beacon_chain/src/test_utils.rs
#	beacon_node/execution_layer/src/engine_api.rs
#	beacon_node/execution_layer/src/engine_api/http.rs
#	beacon_node/execution_layer/src/lib.rs
#	beacon_node/execution_layer/src/test_utils/handle_rpc.rs
#	beacon_node/http_api/src/lib.rs
#	beacon_node/http_api/tests/fork_tests.rs
#	beacon_node/network/src/beacon_processor/mod.rs
#	beacon_node/network/src/beacon_processor/work_reprocessing_queue.rs
#	beacon_node/network/src/beacon_processor/worker/sync_methods.rs
#	beacon_node/operation_pool/src/bls_to_execution_changes.rs
#	beacon_node/operation_pool/src/lib.rs
#	beacon_node/operation_pool/src/persistence.rs
#	consensus/serde_utils/src/u256_hex_be_opt.rs
#	testing/antithesis/Dockerfile.libvoidstar
2023-02-07 12:12:56 -05:00
realbigsean
21e5b7fecd Merge pull request #3941 from realbigsean/blob-decoding
Blob decoding
2023-02-07 08:38:20 -05:00
realbigsean
3533ed418e pr feedback and bugfixes 2023-02-07 08:36:35 -05:00
naviechan
9547ac069c Implement block_rewards API (per-validator reward) (#3907)
## Issue Addressed

[#3661](https://github.com/sigp/lighthouse/issues/3661)

## Proposed Changes

`/eth/v1/beacon/rewards/blocks/{block_id}`

```
{
  "execution_optimistic": false,
  "finalized": false,
  "data": {
    "proposer_index": "123",
    "total": "123",
    "attestations": "123",
    "sync_aggregate": "123",
    "proposer_slashings": "123",
    "attester_slashings": "123"
  }
}
```

The issue contains the implementation of three per-validator reward APIs:
* `sync_committee_rewards`
* [`attestation_rewards`](https://github.com/sigp/lighthouse/pull/3822)
* `block_rewards`

This PR only implements the `block_rewards`.

The endpoints can be viewed in the Ethereum Beacon nodes API browser: [https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Rewards](https://ethereum.github.io/beacon-APIs/?urls.primaryName=dev#/Rewards)

## Additional Info

The implementation of [consensus client reward APIs](https://github.com/eth-protocol-fellows/cohort-three/blob/master/projects/project-ideas.md#consensus-client-reward-apis) is part of the [EPF](https://github.com/eth-protocol-fellows/cohort-three).

Co-authored-by: kevinbogner <kevbogner@gmail.com>
Co-authored-by: navie <naviechan@gmail.com>
2023-02-07 08:33:23 +00:00