getBlobsV2 response and avoid reprocessing imported data columns (#7493)
#7461 and partly #6439. Desired behaviour after receiving `engine_getBlobs` response: 1. Gossip verify the blobs and proofs, but don't mark them as observed yet. This is because not all blobs are published immediately (due to staggered publishing). If we mark them as observed and not publish them, we could end up blocking the gossip propagation. 2. Blobs are marked as observed _either_ when: * They are received from gossip and forwarded to the network . * They are published by the node. Current behaviour: - ❗ We only gossip verify `engine_getBlobsV1` responses, but not `engine_getBlobsV2` responses (PeerDAS). - ❗ After importing EL blobs AND before they're published, if the same blobs arrive via gossip, they will get re-processed, which may result in a re-import. 1. Perform gossip verification on data columns computed from EL `getBlobsV2` response. We currently only do this for `getBlobsV1` to prevent importing blobs with invalid proofs into the `DataAvailabilityChecker`, this should be done on V2 responses too. 2. Add additional gossip verification to make sure we don't re-process a ~~blob~~ or data column that was imported via the EL `getBlobs` but not yet "seen" on the gossip network. If an "unobserved" gossip blob is found in the availability cache, then we know it has passed verification so we can immediately propagate the `ACCEPT` result and forward it to the network, but without re-processing it. **UPDATE:** I've left blobs out for the second change mentioned above, as the likelihood and impact is very slow and we haven't seen it enough, but under PeerDAS this issue is a regular occurrence and we do see the same block getting imported many times.
Lighthouse: Ethereum consensus client
An open-source Ethereum consensus client, written in Rust and maintained by Sigma Prime.
Overview
Lighthouse is:
- Ready for use on Ethereum consensus mainnet.
- Fully open-source, licensed under Apache 2.0.
- Security-focused. Fuzzing techniques have been continuously applied and several external security reviews have been performed.
- Built in Rust, a modern language providing unique safety guarantees and excellent performance (comparable to C++).
- Funded by various organisations, including Sigma Prime, the Ethereum Foundation, Consensys, the Decentralization Foundation and private individuals.
- Actively involved in the specification and security analysis of the Ethereum proof-of-stake consensus specification.
Staking Deposit Contract
The Lighthouse team acknowledges
0x00000000219ab540356cBB839Cbe05303d7705Fa
as the canonical staking deposit contract address.
Documentation
The Lighthouse Book contains information for users and developers.
The Lighthouse team maintains a blog at https://blog.sigmaprime.io/tag/lighthouse which contains periodic progress updates, roadmap insights and interesting findings.
Branches
Lighthouse maintains two permanent branches:
stable: Always points to the latest stable release.- This is ideal for most users.
unstable: Used for development, contains the latest PRs.- Developers should base their PRs on this branch.
Contributing
Lighthouse welcomes contributors.
If you are looking to contribute, please head to the Contributing section of the Lighthouse book.
Contact
The best place for discussion is the Lighthouse Discord server.
Sign up to the Lighthouse Development Updates mailing list for email notifications about releases, network status and other important information.
Encrypt sensitive messages using our PGP key.
Donations
Lighthouse is an open-source project and a public good. Funding public goods is hard and we're grateful for the donations we receive from the community via:
- Gitcoin Grants.
- Ethereum address:
0x25c4a76E7d118705e7Ea2e9b7d8C59930d8aCD3b(donation.sigmaprime.eth).
