Commit Graph

1154 Commits

Author SHA1 Message Date
realbigsean
4c17993928 Update beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs 2023-05-30 09:33:37 -04:00
Mark Mackey
dc201550c5 Check disk on in-memory cache miss 2023-05-27 14:45:06 -05:00
realbigsean
98d7a1014f get tests compiling 2023-05-23 12:10:54 -04:00
realbigsean
8ea98c596c merge unknown parent messages before current slot lookup 2023-05-23 11:33:00 -04:00
realbigsean
c7aad773a8 lots of refactoring 2023-05-19 17:47:44 -04:00
realbigsean
bef63e42f7 compile after merge 2023-05-18 16:42:13 -04:00
realbigsean
4dd1184968 Merge branch 'deneb-free-blobs' of https://github.com/sigp/lighthouse into some-blob-reprocessing-work 2023-05-18 15:27:59 -04:00
realbigsean
542e3c16db add tests and fix some things 2023-05-18 15:20:56 -04:00
ethDreamer
46db30416d Implement Overflow LRU Cache for Pending Blobs (#4203)
* All Necessary Objects Implement Encode/Decode

* Major Components for LRUOverflowCache Implemented

* Finish Database Code

* Add Maintenance Methods

* Added Maintenance Service

* Persist Blobs on Shutdown / Reload on Startup

* Address Clippy Complaints

* Add (emum_behaviour = "tag") to ssz_derive

* Convert Encode/Decode Implementations to "tag"

* Started Adding Tests

* Added a ton of tests

* 1 character fix

* Feature Guard Minimal Spec Tests

* Update beacon_node/beacon_chain/src/data_availability_checker.rs

Co-authored-by: realbigsean <seananderson33@GMAIL.com>

* Address Sean's Comments

* Add iter_raw_keys method

* Remove TODOs

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2023-05-12 10:08:24 -04:00
ethDreamer
a22e4bf636 Implement KZG EF Tests (#4274) 2023-05-08 15:58:23 -04:00
realbigsean
e3f4218624 error refactoring 2023-05-02 14:09:53 -04:00
realbigsean
8edefb7e0d make tests work for all forks 2023-05-02 10:00:07 -04:00
realbigsean
93bcd6281c some bug fixes and the start of deneb only tests 2023-04-28 15:56:54 -04:00
realbigsean
a9b24bb9df add random blobs 2023-04-27 17:39:52 -04:00
realbigsean
185b7615a2 fix compile after merge 2023-04-27 14:30:07 -04:00
realbigsean
eadf293419 Merge branch 'deneb-free-blobs' of https://github.com/sigp/lighthouse into some-blob-reprocessing-work 2023-04-27 14:19:42 -04:00
ethDreamer
c1d47da02d Update engine_api to latest version (#4223)
* Update Engine API to Latest

* Get Mock EE Working

* Fix Mock EE

* Update Engine API Again

* Rip out get_blobs_bundle Stuff

* Fix Test Harness

* Fix Clippy Complaints

* Fix Beacon Chain Tests
2023-04-27 14:18:21 -04:00
realbigsean
7614abf6df fix 2023-04-26 17:55:21 -04:00
realbigsean
3c9fb78aa1 compilation fix in beacon chain tests 2023-04-26 17:28:24 -04:00
realbigsean
2cad638f41 cargo fmt 2023-04-26 16:33:08 -04:00
realbigsean
0978848caa Merge branch 'deneb-free-blobs' of https://github.com/sigp/lighthouse into some-blob-reprocessing-work 2023-04-26 14:59:48 -04:00
Justin Traglia
aa34339298 Rename to MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS (#4206)
Co-authored-by: realbigsean <sean@sigmaprime.io>
2023-04-26 14:53:06 -04:00
realbigsean
d850788278 fix after merge 2023-04-26 14:48:50 -04:00
realbigsean
13f7217172 Merge branch 'deneb-free-blobs' of https://github.com/sigp/lighthouse into some-blob-reprocessing-work 2023-04-26 14:45:50 -04:00
realbigsean
69e5e00350 renamings 2023-04-26 14:45:07 -04:00
Pawan Dhananjay
a632969695 Gossip verification cleanup (#4219)
* Add ObservedBlobSidecar tests

* Add logging for tricky verification cases

* Update beacon_node/beacon_chain/src/blob_verification.rs

---------

Co-authored-by: realbigsean <seananderson33@GMAIL.com>
2023-04-26 10:44:58 -04:00
realbigsean
14e5bfae29 fix lints 2023-04-25 18:12:16 -04:00
realbigsean
9c2aceed10 fix some todos 2023-04-25 09:25:24 -04:00
realbigsean
381044abe7 add peer usefulness enum 2023-04-24 12:27:49 -04:00
realbigsean
b2a4c4e73e Merge branch 'deneb-free-blobs' of https://github.com/sigp/lighthouse into some-blob-reprocessing-work 2023-04-22 09:23:01 -04:00
Pawan Dhananjay
b6c0e91c05 Merge branch 'eip4844' into deneb-free-blobs 2023-04-21 14:34:50 -07:00
Pawan Dhananjay
689c0f76d3 Merge branch 'unstable' into eip4844 2023-04-21 14:13:25 -07:00
realbigsean
3e854ae2d1 fix compile 2023-04-21 16:57:53 -04:00
Pawan Dhananjay
a78285db5e Fix Rust 1.69 lints (#4222)
## Issue Addressed

N/A

## Proposed Changes

Fixes lints mostly `extra-unused-type-parameters` https://rust-lang.github.io/rust-clippy/master/index.html#extra_unused_type_paramete
2023-04-21 18:29:28 +00:00
realbigsean
bacec52017 parent blob lookups 2023-04-20 19:42:33 -04:00
realbigsean
0cc1704edf Merge branch 'deneb-free-blobs' of https://github.com/sigp/lighthouse into some-blob-reprocessing-work 2023-04-20 18:56:25 -04:00
Pawan Dhananjay
895bbd6c03 Gossip conditions deneb (#4164)
* Add all gossip conditions

* Handle some gossip errors

* Update beacon_node/beacon_chain/src/blob_verification.rs

Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>

* Add an ObservedBlobSidecars cache

---------

Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>
2023-04-20 18:26:20 -04:00
realbigsean
c7142495fd get things compiling 2023-04-20 13:38:05 -04:00
realbigsean
374ec4800a much work 2023-04-19 16:44:19 -04:00
realbigsean
0ad9fdfbbf fix compilation in main block lookup mod 2023-04-19 14:02:41 -04:00
Michael Sproul
1d92e3f77c Use efficient payload reconstruction for HTTP API (#4102)
## Proposed Changes

Builds on #4028 to use the new payload bodies methods in the HTTP API as well.

## Caveats

The payloads by range method only works for the finalized chain, so it can't be used in the execution engine integration tests because we try to reconstruct unfinalized payloads there.
2023-04-18 02:47:35 +00:00
realbigsean
195d802931 start fixing some compile errors 2023-04-17 16:58:18 -04:00
realbigsean
8618c301b5 add delayed processing logic and combine some requests 2023-04-14 16:50:41 -04:00
Paul Hauner
dd124b2d68 Address observed proposers behaviour (#4192)
## Issue Addressed

NA

## Proposed Changes

Apply two changes to code introduced in #4179:

1. Remove the `ERRO` log for when we error on `proposer_has_been_observed()`. We were seeing a lot of this in our logs for finalized blocks and it's a bit noisy.
1. Use `false` rather than `true` for `proposal_already_known` when there is an error. If a block raises an error in `proposer_has_been_observed()` then the block must be invalid, so we should process (and reject) it now rather than queuing it.

For reference, here is one of the offending `ERRO` logs:

```
ERRO Failed to check observed proposers block_root: 0x5845…878e, source: rpc, error: FinalizedBlock { slot: Slot(5410983), finalized_slot: Slot(5411232) }
```

## Additional Info

NA
2023-04-14 06:37:16 +00:00
Paul Hauner
2b3084f578 Use head state for exit verification (#4183)
## Issue Addressed

NA

## Proposed Changes

Similar to #4181 but without the version bump and a more nuanced fix.

Patches the high CPU usage seen after the Capella fork which was caused by processing exits when there are skip slots.

## Additional Info

~~This is an imperfect solution that will cause us to drop some exits at the fork boundary. This is tracked at #4184.~~
2023-04-14 01:11:46 +00:00
Michael Sproul
a3669abac5 Avoid processing redundant RPC blocks (#4179)
## Proposed Changes

We already make some attempts to avoid processing RPC blocks when a block from the same proposer is already being processed through gossip. This PR strengthens that guarantee by using the existing cache for `observed_block_producers` to inform whether an RPC block's processing should be delayed.
2023-04-13 07:05:02 +00:00
Michael Sproul
b90c0c3fb1 Make re-org strat more cautious and add more config (#4151)
## Proposed Changes

This change attempts to prevent failed re-orgs by:

1. Lowering the re-org cutoff from 2s to 1s. This is informed by a failed re-org attempted by @yorickdowne's node. The failed block was requested in the 1.5-2s window due to a Vouch failure, and failed to propagate to the majority of the network before the attestation deadline at 4s.
2. Allow users to adjust their re-org cutoff depending on observed network conditions and their risk profile. The static 2 second cutoff was too rigid.
3. Add a `--proposer-reorg-disallowed-offsets` flag which can be used to prohibit reorgs at certain slots. This is intended to help workaround an issue whereby reorging blocks at slot 1 are currently taking ~1.6s to propagate on gossip rather than ~500ms. This is suspected to be due to a cache miss in current versions of Prysm, which should be fixed in their next release.

## Additional Info

I'm of two minds about removing the `shuffling_stable` check which checks for blocks at slot 0 in the epoch. If we removed it users would be able to configure Lighthouse to try reorging at slot 0, which likely wouldn't work very well due to interactions with the proposer index cache. I think we could leave it for now and revisit it later.
2023-04-13 07:05:01 +00:00
realbigsean
2f8c8852ba Merge branch 'deneb-free-blobs' of https://github.com/sigp/lighthouse into some-blob-reprocessing-work 2023-04-12 16:29:38 -04:00
realbigsean
25ff6e8a5f more work 2023-04-11 13:13:13 -04:00
Diva M
911a63559b Merge branch 'eip4844' into deneb-free-blobs 2023-04-05 13:33:33 -05:00