Self rate limiting dev flag (#3928)

## Issue Addressed
Adds self rate limiting options, mainly with the idea to comply with peer's rate limits in small testnets

## Proposed Changes
Add a hidden flag `self-limiter` this can take no value, or customs values to configure quotas per protocol

## Additional Info
### How to use
`--self-limiter` will turn on the self rate limiter applying the same params we apply to inbound requests (requests from other peers)
`--self-limiter "beacon_blocks_by_range:64/1"` will turn on the self rate limiter for ALL protocols, but change the quota for bbrange to 64 requested blocks per 1 second.
`--self-limiter "beacon_blocks_by_range:64/1;ping:1/10"` same as previous one, changing the quota for ping as well.

### Caveats
- The rate limiter is either on or off for all protocols. I added the custom values to be able to change the quotas per protocol so that some protocols can be given extremely loose or tight quotas. I think this should satisfy every need even if we can't technically turn off rate limits per protocol.
- This reuses the rate limiter struct for the inbound requests so there is this ugly part of the code in which we need to deal with the inbound only protocols (light client stuff) if this becomes too ugly as we add lc protocols, we might want to split the rate limiters. I've checked this and looks doable with const generics to avoid so much code duplication

### Knowing if this is on
```
Feb 06 21:12:05.493 DEBG Using self rate limiting params         config: OutboundRateLimiterConfig { ping: 2/10s, metadata: 1/15s, status: 5/15s, goodbye: 1/10s, blocks_by_range: 1024/10s, blocks_by_root: 128/10s }, service: libp2p_rpc, service: libp2p
```
This commit is contained in:
Divma
2023-02-08 02:18:53 +00:00
parent 7934485aef
commit ceb986549d
10 changed files with 528 additions and 39 deletions

View File

@@ -1,6 +1,7 @@
use crate::rpc::{InboundRequest, Protocol};
use crate::rpc::Protocol;
use fnv::FnvHashMap;
use libp2p::PeerId;
use serde_derive::{Deserialize, Serialize};
use std::convert::TryInto;
use std::future::Future;
use std::hash::Hash;
@@ -47,12 +48,31 @@ type Nanosecs = u64;
/// n*`replenish_all_every`/`max_tokens` units of time since their last request.
///
/// To produce hard limits, set `max_tokens` to 1.
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct Quota {
/// How often are `max_tokens` fully replenished.
replenish_all_every: Duration,
pub(super) replenish_all_every: Duration,
/// Token limit. This translates on how large can an instantaneous batch of
/// tokens be.
max_tokens: u64,
pub(super) max_tokens: u64,
}
impl Quota {
/// A hard limit of one token every `seconds`.
pub const fn one_every(seconds: u64) -> Self {
Quota {
replenish_all_every: Duration::from_secs(seconds),
max_tokens: 1,
}
}
/// Allow `n` tokens to be use used every `seconds`.
pub const fn n_every(n: u64, seconds: u64) -> Self {
Quota {
replenish_all_every: Duration::from_secs(seconds),
max_tokens: n,
}
}
}
/// Manages rate limiting of requests per peer, with differentiated rates per protocol.
@@ -78,6 +98,7 @@ pub struct RPCRateLimiter {
}
/// Error type for non conformant requests
#[derive(Debug)]
pub enum RateLimitedErr {
/// Required tokens for this request exceed the maximum
TooLarge,
@@ -86,7 +107,7 @@ pub enum RateLimitedErr {
}
/// User-friendly builder of a `RPCRateLimiter`
#[derive(Default)]
#[derive(Default, Clone)]
pub struct RPCRateLimiterBuilder {
/// Quota for the Goodbye protocol.
goodbye_quota: Option<Quota>,
@@ -105,13 +126,8 @@ pub struct RPCRateLimiterBuilder {
}
impl RPCRateLimiterBuilder {
/// Get an empty `RPCRateLimiterBuilder`.
pub fn new() -> Self {
Default::default()
}
/// Set a quota for a protocol.
fn set_quota(mut self, protocol: Protocol, quota: Quota) -> Self {
pub fn set_quota(mut self, protocol: Protocol, quota: Quota) -> Self {
let q = Some(quota);
match protocol {
Protocol::Ping => self.ping_quota = q,
@@ -191,11 +207,40 @@ impl RPCRateLimiterBuilder {
}
}
pub trait RateLimiterItem {
fn protocol(&self) -> Protocol;
fn expected_responses(&self) -> u64;
}
impl<T: EthSpec> RateLimiterItem for super::InboundRequest<T> {
fn protocol(&self) -> Protocol {
self.protocol()
}
fn expected_responses(&self) -> u64 {
self.expected_responses()
}
}
impl<T: EthSpec> RateLimiterItem for super::OutboundRequest<T> {
fn protocol(&self) -> Protocol {
self.protocol()
}
fn expected_responses(&self) -> u64 {
self.expected_responses()
}
}
impl RPCRateLimiter {
pub fn allows<T: EthSpec>(
/// Get a builder instance.
pub fn builder() -> RPCRateLimiterBuilder {
RPCRateLimiterBuilder::default()
}
pub fn allows<Item: RateLimiterItem>(
&mut self,
peer_id: &PeerId,
request: &InboundRequest<T>,
request: &Item,
) -> Result<(), RateLimitedErr> {
let time_since_start = self.init_time.elapsed();
let tokens = request.expected_responses().max(1);