Commit Graph

3268 Commits

Author SHA1 Message Date
jking-aus
4fd8e521a4 Use existing peer count metrics loop to check for open_nat toggle (#6800)
* implement update_nat_open function in network_behaviour for tracking incoming peers below a given threshold count

* implement update_nat_open function in network_behaviour for tracking incoming peers below a given threshold count

* tidy logic and comments

* move logic to existing metrics loop

* revert change to network_behaviour protocol check

* clippy

* clippy matches! macro

* pull nat_open check outside of peercounting loop

* missing close bracket

* make threshold const
2025-01-15 03:53:40 +00:00
Pawan Dhananjay
587c3e2b8c Implement changes for EIP 7691 (#6803)
* Add new config options

* Use electra_enabled
2025-01-14 06:22:30 +00:00
Pawan Dhananjay
93f9c2c718 Execution requests with prefix (#6801)
* Exclude empty requests and add back prefix

* cleanup

* fix after rebase
2025-01-14 02:06:27 +00:00
Michael Sproul
06e4d22d49 Electra spec changes for v1.5.0-beta.0 (#6731)
* First pass

* Add restrictions to RuntimeVariableList api

* Use empty_uninitialized and fix warnings

* Fix some todos

* Merge branch 'unstable' into max-blobs-preset

* Fix take impl on RuntimeFixedList

* cleanup

* Fix test compilations

* Fix some more tests

* Fix test from unstable

* Merge branch 'unstable' into max-blobs-preset

* Implement "Bugfix and more withdrawal tests"

* Implement "Add missed exit checks to consolidation processing"

* Implement "Update initial earliest_exit_epoch calculation"

* Implement "Limit consolidating balance by validator.effective_balance"

* Implement "Use 16-bit random value in validator filter"

* Implement "Do not change creds type on consolidation"

* Rename PendingPartialWithdraw index field to validator_index

* Skip slots to get test to pass and add TODO

* Implement "Synchronously check all transactions to have non-zero length"

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

* Remove footgun function

* Minor simplifications

* Move from preset to config

* Fix typo

* Revert "Remove footgun function"

This reverts commit de01f923c7.

* Try fixing tests

* Implement "bump minimal preset MAX_BLOB_COMMITMENTS_PER_BLOCK and KZG_COMMITMENT_INCLUSION_PROOF_DEPTH"

* Thread through ChainSpec

* Fix release tests

* Move RuntimeFixedVector into module and rename

* Add test

* Implement "Remove post-altair `initialize_beacon_state_from_eth1` from specs"

* Update preset YAML

* Remove empty RuntimeVarList awefullness

* Make max_blobs_per_block a config parameter (#6329)

Squashed commit of the following:

commit 04b3743ec1
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 17:36:58 2025 +1100

    Add test

commit 440e854199
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 17:24:50 2025 +1100

    Move RuntimeFixedVector into module and rename

commit f66e179a40
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 17:17:17 2025 +1100

    Fix release tests

commit e4bfe71cd1
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 17:05:30 2025 +1100

    Thread through ChainSpec

commit 063b79c16a
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 15:32:16 2025 +1100

    Try fixing tests

commit 88bedf09bc
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 15:04:37 2025 +1100

    Revert "Remove footgun function"

    This reverts commit de01f923c7.

commit 32483d385b
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 15:04:32 2025 +1100

    Fix typo

commit 2e86585b47
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 15:04:15 2025 +1100

    Move from preset to config

commit 1095d60a40
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 14:38:40 2025 +1100

    Minor simplifications

commit de01f923c7
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 14:06:57 2025 +1100

    Remove footgun function

commit 0c2c8c4224
Merge: 21ecb58ff f51a292f7
Author: Michael Sproul <michael@sigmaprime.io>
Date:   Mon Jan 6 14:02:50 2025 +1100

    Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

commit f51a292f77
Author: Daniel Knopik <107140945+dknopik@users.noreply.github.com>
Date:   Fri Jan 3 20:27:21 2025 +0100

    fully lint only explicitly to avoid unnecessary rebuilds (#6753)

    * fully lint only explicitly to avoid unnecessary rebuilds

commit 7e0cddef32
Author: Akihito Nakano <sora.akatsuki@gmail.com>
Date:   Tue Dec 24 10:38:56 2024 +0900

    Make sure we have fanout peers when publish (#6738)

    * Ensure that `fanout_peers` is always non-empty if it's `Some`

commit 21ecb58ff8
Merge: 2fcb2935e 9aefb5539
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Mon Oct 21 14:46:00 2024 -0700

    Merge branch 'unstable' into max-blobs-preset

commit 2fcb2935ec
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Sep 6 18:28:31 2024 -0700

    Fix test from unstable

commit 12c6ef118a
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Wed Sep 4 16:16:36 2024 -0700

    Fix some more tests

commit d37733b846
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Wed Sep 4 12:47:36 2024 -0700

    Fix test compilations

commit 52bb581e07
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Tue Sep 3 18:38:19 2024 -0700

    cleanup

commit e71020e3e6
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Tue Sep 3 17:16:10 2024 -0700

    Fix take impl on RuntimeFixedList

commit 13f9bba647
Merge: 60100fc6b 4e675cf5d
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Tue Sep 3 16:08:59 2024 -0700

    Merge branch 'unstable' into max-blobs-preset

commit 60100fc6be
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Aug 30 16:04:11 2024 -0700

    Fix some todos

commit a9cb329a22
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Aug 30 15:54:00 2024 -0700

    Use empty_uninitialized and fix warnings

commit 4dc6e6515e
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Fri Aug 30 15:53:18 2024 -0700

    Add restrictions to RuntimeVariableList api

commit 25feedfde3
Author: Pawan Dhananjay <pawandhananjay@gmail.com>
Date:   Thu Aug 29 16:11:19 2024 -0700

    First pass

* Fix tests

* Implement max_blobs_per_block_electra

* Fix config issues

* Simplify BlobSidecarListFromRoot

* Disable PeerDAS tests

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

* Bump quota to account for new target (6)

* Remove clone

* Fix issue from review

* Try to remove ugliness

* Merge branch 'unstable' into max-blobs-preset

* Merge remote-tracking branch 'origin/unstable' into electra-alpha10

* Merge commit '04b3743ec1e0b650269dd8e58b540c02430d1c0d' into electra-alpha10

* Merge remote-tracking branch 'pawan/max-blobs-preset' into electra-alpha10

* Update tests to v1.5.0-beta.0

* Resolve merge conflicts

* Linting

* fmt

* Fix test and add TODO

* Gracefully handle slashed proposers in fork choice tests

* Merge remote-tracking branch 'origin/unstable' into electra-alpha10

* Keep latest changes from max_blobs_per_block PR in codec.rs

* Revert a few more regressions and add a comment

* Disable more DAS tests

* Improve validator monitor test a little

* Make test more robust

* Fix sync test that didn't understand blobs

* Fill out cropped comment
2025-01-13 23:17:00 +00:00
Mac L
348fbdb838 Add missing crates to cargo workspace (#6774)
* Add the remaining crates to cargo workspace

* Merge branch 'unstable' into add-remaining-crates-workspace
2025-01-10 07:35:05 +00:00
Pawan Dhananjay
05727290fb Make max_blobs_per_block a config parameter (#6329)
* First pass

* Add restrictions to RuntimeVariableList api

* Use empty_uninitialized and fix warnings

* Fix some todos

* Merge branch 'unstable' into max-blobs-preset

* Fix take impl on RuntimeFixedList

* cleanup

* Fix test compilations

* Fix some more tests

* Fix test from unstable

* Merge branch 'unstable' into max-blobs-preset

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

* Remove footgun function

* Minor simplifications

* Move from preset to config

* Fix typo

* Revert "Remove footgun function"

This reverts commit de01f923c7.

* Try fixing tests

* Thread through ChainSpec

* Fix release tests

* Move RuntimeFixedVector into module and rename

* Add test

* Remove empty RuntimeVarList awefullness

* Fix tests

* Simplify BlobSidecarListFromRoot

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset

* Bump quota to account for new target (6)

* Remove clone

* Fix issue from review

* Try to remove ugliness

* Merge branch 'unstable' into max-blobs-preset

* Fix max value

* Fix doctest

* Fix formatting

* Fix max check

* Delete hardcoded max_blobs_per_block in RPC limits

* Merge remote-tracking branch 'origin/unstable' into max-blobs-preset
2025-01-10 06:34:58 +00:00
Mac L
ecdf2d891f Add Fulu boilerplate (#6695)
* Add Fulu boilerplate

* Add more boilerplate

* Change fulu_time to osaka_time

* Merge branch 'unstable' into fulu-boilerplate

* Fix tests

* Merge branch 'unstable' into fulu-boilerplate

* More test fixes

* Apply suggestions

* Remove `get_payload` boilerplate

* Add lightclient fulu types and fix beacon-chain-tests

* Disable Fulu in ef-tests

* Reduce boilerplate for future forks

* Small fixes

* One more fix

* Apply suggestions

* Merge branch 'unstable' into fulu-boilerplate

* Fix lints
2025-01-10 05:25:23 +00:00
Lion - dapplion
722573f7ed Use oldest_block_slot to break off pruning payloads (#6745)
* Use oldest_block_slot to break of pruning payloads

* Update beacon_node/store/src/hot_cold_store.rs

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Merge remote-tracking branch 'origin/unstable' into anchor_slot_pruning
2025-01-10 05:25:20 +00:00
Pawan Dhananjay
1f6850fae2 Rust 1.84 lints (#6781)
* Fix few lints

* Fix remaining lints

* Use fully qualified syntax
2025-01-10 01:13:29 +00:00
Jimmy Chen
80cfbea7fe Fix incorrect data column metric name (#6761)
* Fix incorrect data column metric name.
2025-01-08 03:12:37 +00:00
Jimmy Chen
7ec748a108 Implement getBlobSidecars support for PeerDAS (#6755)
* Implement getBlobSidecars endpoint for PeerDAS.

* Merge branch 'unstable' into peerdas-get-blob-sidecars

* Fix incorrect logging.

* Replace `and_then` usage.
2025-01-08 03:12:34 +00:00
Ekaterina Riazantseva
57141d8b4b Add 'beacon_' prefix to PeerDAS metrics names (#6537)
* Add 'beacon_' prefix to PeerDAS metrics names

* Merge remote-tracking branch 'origin/unstable' into peerdas-metrics

* Merge 'origin/unstable' into peerdas-metrics

* Merge remote-tracking branch 'origin/unstable/ into peerdas-metrics

* Add 'beacon_' prefix to 'kzg_data_column' metrics
2025-01-08 00:39:53 +00:00
Akihito Nakano
7e0cddef32 Make sure we have fanout peers when publish (#6738)
* Ensure that `fanout_peers` is always non-empty if it's `Some`
2024-12-24 01:38:56 +00:00
Michael Sproul
07e82dabc0 Delete OTB verification service (#6631)
* Delete OTB verification service

* Merge branch 'unstable' into delete-otb
2024-12-19 05:46:06 +00:00
Mac L
b2b1faad4e Enforce alphabetically ordered cargo deps (#6678)
* Enforce alphabetically ordered cargo deps

* Fix test-suite

* Another CI fix

* Merge branch 'unstable' into cargo-sort

* Fix conflicts

* Merge remote-tracking branch 'origin/unstable' into cargo-sort
2024-12-19 05:46:03 +00:00
Pawan Dhananjay
2662dc7f8f Fix Sse client api (#6685)
* Use reqwest eventsource for get_events api

* await for Event::Open before returning stream

* fmt

* Merge branch 'unstable' into sse-client-fix

* Ignore lint
2024-12-18 00:05:58 +00:00
Age Manning
1315c94adb Unsaturate dial negotiation queue (#6711)
* Unsaturate dial-negotiation count
2024-12-17 20:10:53 +00:00
Pawan Dhananjay
d74b2d96f5 Electra alpha8 spec updates (#6496)
* Fix partial withdrawals count

* Remove get_active_balance

* Remove queue_entire_balance_and_reset_validator

* Switch to compounding when consolidating with source==target

* Queue deposit requests and apply them during epoch processing

* Fix ef tests

* Clear todos

* Fix engine api formatting issues

* Merge branch 'unstable' into electra-alpha7

* Make add_validator_to_registry more in line with the spec

* Address some review comments

* Cleanup

* Update initialize_beacon_state_from_eth1

* Merge branch 'unstable' into electra-alpha7

* Fix rpc decoding for blobs by range/root

* Fix block hash computation

* Fix process_deposits bug

* Merge branch 'unstable' into electra-alpha7

* Fix topup deposit processing bug

* Update builder api for electra

* Refactor mock builder to separate functionality

* Merge branch 'unstable' into electra-alpha7

* Address review comments

* Use copied for reference rather than cloned

* Optimise and simplify PendingDepositsContext::new

* Merge remote-tracking branch 'origin/unstable' into electra-alpha7

* Fix processing of deposits with invalid signatures

* Remove redundant code in genesis init

* Revert "Refactor mock builder to separate functionality"

This reverts commit 6d10456912.

* Revert "Update builder api for electra"

This reverts commit c5c9aca6db.

* Simplify pre-activation sorting

* Fix stale validators used in upgrade_to_electra

* Merge branch 'unstable' into electra-alpha7
2024-12-17 02:14:24 +00:00
Daniel Knopik
02cb2d68ff Enable lints for tests only running optimized (#6664)
* enable linting optimized-only tests

* fix automatically fixable or obvious lints

* fix suspicious_open_options by removing manual options

* fix `await_holding_lock`s

* avoid failing lint due to now disabled `#[cfg(debug_assertions)]`

* reduce future sizes in tests

* fix accidently flipped assert logic

* restore holding lock for web3signer download

* Merge branch 'unstable' into lint-opt-tests
2024-12-17 00:40:35 +00:00
Jimmy Chen
847c8019c7 Fix peer down-scoring behaviour when gossip blobs/columns are received after getBlobs or reconstruction (#6686)
* Fix peer disconnection when gossip blobs/columns are received after they are recieved from the EL or available via column reconstruction.
2024-12-16 05:44:14 +00:00
Lion - dapplion
1c5be34def Write range sync tests in external event-driven form (#6618)
* Write range sync tests in external event-driven form

* Fix remaining test

* Drop unused generics

* Merge branch 'unstable' into range-sync-tests

* Add reference to test author

* Use async await

* Fix failing test. Not sure how it was passing before without an EL.
2024-12-16 05:44:10 +00:00
Akihito Nakano
75d90795be Remove req_id from CustodyId (#6589)
* Remove req_id from CustodyId because it's not used
2024-12-16 05:44:06 +00:00
ethDreamer
86891e6d0f builder gas limit & some refactoring (#6583)
* Cache gas_limit

* Payload Parameters Refactor

* Enforce Proposer Gas Limit

* Fixed and Added New Tests

* Fix Beacon Chain Tests
2024-12-16 05:43:58 +00:00
Michael Sproul
a6de0d5e12 Merge remote-tracking branch 'origin/stable' into back-merge-6.0.1 2024-12-16 15:37:52 +11:00
Lion - dapplion
c92c07ff49 Track beacon processor import result metrics (#6541)
* Track beacon processor import result metrics

* Update metric name
2024-12-16 04:33:33 +00:00
Michael Sproul
0d90135047 Release v6.0.1 (#6659)
* Release v6.0.1
2024-12-16 03:03:22 +00:00
Age Manning
c3a0757ad2 Correct /nat API for libp2p (#6677)
* Fix nat API
2024-12-15 23:16:53 +00:00
Michael Sproul
f3b78889e5 Compact more when pruning states (#6667)
* Compact more when pruning states

* Merge branch 'release-v6.0.1' into compact-more
2024-12-14 08:43:00 +00:00
Lion - dapplion
775fa6730b Stuck lookup v6 (#6658)
* Fix stuck lookups if no peers on v6

* Merge branch 'release-v6.0.1' into stuck-lookup-v6
2024-12-13 05:02:10 +00:00
Povilas Liubauskas
4946343990 Fix subscribing to attestation subnets for aggregating (#6681) (#6682)
* Fix subscribing to attestation subnets for aggregating (#6681)

* Prevent scheduled subnet subscriptions from being overwritten by other subscriptions from same subnet with additional scoping by slot
2024-12-12 08:36:34 +00:00
Michael Sproul
fc0e0ae613 Prevent reconstruction starting prematurely (#6669)
* Prevent reconstruction starting prematurely

* Simplify condition

* Merge remote-tracking branch 'origin/release-v6.0.1' into dont-start-reconstruction-early
2024-12-12 01:58:41 +00:00
Michael Sproul
b2590bcb37 Tweak reconstruction batch size (#6668)
* Tweak reconstruction batch size

* Merge branch 'release-v6.0.1' into reconstruction-batch-size
2024-12-11 22:51:46 +00:00
Michael Sproul
c5a48a9dff Merge remote-tracking branch 'origin/release-v6.0.1' into unstable 2024-12-10 15:53:35 +11:00
Age Manning
3b8254a8ec Correct flakey CI tests (#6646)
* Correct flakey CI tests

* Correct clippy

* Extend timeout for events
2024-12-10 04:24:55 +00:00
hopinheimer
fec502db9f Deprecate futures ticker (#6630)
* deprecate futures-ticker

* Merge branch 'unstable' of github.com:sigp/lighthouse into deprecate-futures-timer

* Merge branch 'unstable' into deprecate-futures-timer

* making the linter happy

* remove unrequired #[allow(unused_imports)]

* fixing minor issues

* merge commit

* minor fix

* clippy changes
2024-12-03 08:56:43 +00:00
Michael Sproul
c042dc14d7 Release v6.0.0 (#6605)
* Release v6.0.0
2024-12-02 04:00:51 +00:00
Age Manning
770d677a4e Increase idle connection timeout (#6604)
* Increase idle connection timeout

* Update beacon_node/lighthouse_network/src/service/mod.rs

Co-authored-by: João Oliveira <hello@jxs.pt>
2024-12-02 01:44:58 +00:00
Pawan Dhananjay
f8e31f6272 Increase rpc rate limits (#6626)
* Increase rate limits for byrange requests

* Merge branch 'unstable' into reduce-blob-limits

* Update limits
2024-12-01 23:31:10 +00:00
Michael Sproul
1c8161f92b Fetch blobs from EL prior to block verification (#6600)
* Fetch blobs from EL prior to block verification

* Run fetch blobs in parallel with block import

* Merge branch 'unstable' into fetch-blobs-earlier

* Merge branch 'unstable' into fetch-blobs-earlier
2024-11-29 02:58:19 +00:00
Michael Sproul
fa6c4c02a3 Fix Rust 1.83 Clippy lints (#6629)
* Fix Rust 1.83 Clippy lints

* Cargo fmt
2024-11-29 02:23:54 +00:00
Eitan Seri-Levi
38f5f665e1 Remove error-chain dependency (#6628)
* remove error-chain dependency

* rerun CI

* rerun CI
2024-11-28 02:39:50 +00:00
Age Manning
08e8b92e50 Simple Subnet Management (#6146)
* Initial temp commit

* Merge latest unstable

* First draft without tests

* Update tests for new version

* Correct comments and reviewers comments

* Merge latest unstable

* Fix errors

* Missed a comment, corrected it

* Fix lints

* Merge latest unstable

* Fix tests

* Merge latest unstable

* Reviewers comments

* Remove sync subnets from ENR on unsubscribe

* Merge branch 'unstable' into simple-peer-mapping

* Merge branch 'unstable' into simple-peer-mapping

* Merge branch 'unstable' into simple-peer-mapping

* Merge latest unstable

* Prevent clash with pin of rust_eth_kzg
2024-11-26 01:48:07 +00:00
Pawan Dhananjay
6e1945fc5d Avoid computing rpc_blob_limits multiple times (#6595)
* Compute blob rpc limits in static block

* Fix min size

* Use MainnetEthSpec in rpc tests

* Revert MainnetEthSpec; add another constant for blob size minimal
2024-11-20 23:04:45 +00:00
Pawan Dhananjay
94311c6516 Add additional metrics for idontwant (#6578)
* Add additional metrics for idontwant

* Resolve issues from review

* Fix tests

* Don't exceed capacity

* Apply suggestions from code review

Co-authored-by: João Oliveira <hello@jxs.pt>

* Return early on failure

* Add comment
2024-11-20 22:27:13 +00:00
Michael Sproul
b1e9f69460 Fix v22 schema upgrade (#6591)
* Fix v22 schema upgrade

* Ownership
2024-11-19 22:43:18 +00:00
Age Manning
8cebc87d95 Update to latest discovery (#6486)
* Upgrade discv5 to v0.8

* Rename some logs

* Improve the NAT reporting with new discv5 metrics

* Merge branch 'unstable' into discv5-v8

* Limited Cargo.lock update

* Update yanked futures-* crates
2024-11-18 22:52:23 +00:00
Michael Sproul
c5007eaa1c Deprecate eth1 and dummy-eth1 flags (#6566)
* Deprecate eth1/dummy-eth1 flags

* Update book

* Simplify + make staking conflict with disable-deposit-contract
2024-11-18 08:36:01 +00:00
Michael Sproul
9fdd53df56 Hierarchical state diffs (#5978)
* Start extracting freezer changes for tree-states

* Remove unused config args

* Add comments

* Remove unwraps

* Subjective more clear implementation

* Clean up hdiff

* Update xdelta3

* Tree states archive metrics (#6040)

* Add store cache size metrics

* Add compress timer metrics

* Add diff apply compute timer metrics

* Add diff buffer cache hit metrics

* Add hdiff buffer load times

* Add blocks replayed metric

* Move metrics to store

* Future proof some metrics

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Port and clean up forwards iterator changes

* Add and polish hierarchy-config flag

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Cleaner errors

* Fix beacon_chain test compilation

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Patch a few more freezer block roots

* Fix genesis block root bug

* Fix test failing due to pending updates

* Beacon chain tests passing

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix doc lint

* Implement DB schema upgrade for hierarchical state diffs (#6193)

* DB upgrade

* Add flag

* Delete RestorePointHash

* Update docs

* Update docs

* Implement hierarchical state diffs config migration (#6245)

* Implement hierarchical state diffs config migration

* Review PR

* Remove TODO

* Set CURRENT_SCHEMA_VERSION correctly

* Fix genesis state loading

* Re-delete some PartialBeaconState stuff

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix test compilation

* Update schema downgrade test

* Fix tests

* Fix null anchor migration

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix tree states upgrade migration (#6328)

* Towards crash safety

* Fix compilation

* Move cold summaries and state roots to new columns

* Rename StateRoots chunked field

* Update prune states

* Clean hdiff CLI flag and metrics

* Fix "staged reconstruction"

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Fix alloy issues

* Fix staged reconstruction logic

* Prevent weird slot drift

* Remove "allow" flag

* Update CLI help

* Remove FIXME about downgrade

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Remove some unnecessary error variants

* Fix new test

* Tree states archive - review comments and metrics (#6386)

* Review PR comments and metrics

* Comments

* Add anchor metrics

* drop prev comment

* Update metadata.rs

* Apply suggestions from code review

---------

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Update beacon_node/store/src/hot_cold_store.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Clarify comment and remove anchor_slot garbage

* Simplify database anchor (#6397)

* Simplify database anchor

* Update beacon_node/store/src/reconstruct.rs

* Add migration for anchor

* Fix and simplify light_client store tests

* Fix incompatible config test

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* More metrics

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* New historic state cache (#6475)

* New historic state cache

* Add more metrics

* State cache hit rate metrics

* Fix store metrics

* More logs and metrics

* Fix logger

* Ensure cached states have built caches :O

* Replay blocks in preference to diffing

* Two separate caches

* Distribute cache build time to next slot

* Re-plumb historic-state-cache flag

* Clean up metrics

* Update book

* Update beacon_node/store/src/hdiff.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update beacon_node/store/src/historic_state_cache.rs

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

---------

Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>

* Update database docs

* Update diagram

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Update lockbud to work with bindgen/etc

* Correct pkg name for Debian

* Remove vestigial epochs_per_state_diff

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Markdown lint

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Address Jimmy's review comments

* Simplify ReplayFrom case

* Fix and document genesis_state_root

* Typo

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Merge branch 'unstable' into tree-states-archive

* Compute diff of validators list manually (#6556)

* Split hdiff computation

* Dedicated logic for historical roots and summaries

* Benchmark against real states

* Mutated source?

* Version the hdiff

* Add lighthouse DB config for hierarchy exponents

* Tidy up hierarchy exponents flag

* Apply suggestions from code review

Co-authored-by: Michael Sproul <micsproul@gmail.com>

* Address PR review

* Remove hardcoded paths in benchmarks

* Delete unused function in benches

* lint

---------

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Test hdiff binary format stability (#6585)

* Merge remote-tracking branch 'origin/unstable' into tree-states-archive

* Add deprecation warning for SPRP

* Update xdelta to get rid of duplicate deps

* Document test
2024-11-18 01:51:44 +00:00
Eitan Seri-Levi
654fc6acdc Additional light client metrics (#6545)
* Fix db query and add some additional metrics

* fmt

* Update beacon_node/beacon_chain/src/metrics.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>

* Update beacon_node/beacon_chain/src/metrics.rs

Co-authored-by: Jimmy Chen <jchen.tc@gmail.com>
2024-11-15 07:09:54 +00:00
Jimmy Chen
5f053b0b6d Improving blob propagation post-PeerDAS with Decentralized Blob Building (#6268)
* Get blobs from EL.

Co-authored-by: Michael Sproul <michael@sigmaprime.io>

* Avoid cloning blobs after fetching blobs.

* Address review comments and refactor code.

* Fix lint.

* Move blob computation metric to the right spot.

* Merge branch 'unstable' into das-fetch-blobs

* Merge branch 'unstable' into das-fetch-blobs

# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs
#	beacon_node/beacon_chain/src/block_verification.rs
#	beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs

* Merge branch 'unstable' into das-fetch-blobs

# Conflicts:
#	beacon_node/beacon_chain/src/beacon_chain.rs

* Gradual publication of data columns for supernodes.

* Recompute head after importing block with blobs from the EL.

* Fix lint

* Merge branch 'unstable' into das-fetch-blobs

* Use blocking task instead of async when computing cells.

* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs

* Merge remote-tracking branch 'origin/unstable' into das-fetch-blobs

* Fix semantic conflicts

* Downgrade error log.

* Merge branch 'unstable' into das-fetch-blobs

# Conflicts:
#	beacon_node/beacon_chain/src/data_availability_checker.rs
#	beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs
#	beacon_node/execution_layer/src/engine_api.rs
#	beacon_node/execution_layer/src/engine_api/json_structures.rs
#	beacon_node/network/src/network_beacon_processor/gossip_methods.rs
#	beacon_node/network/src/network_beacon_processor/mod.rs
#	beacon_node/network/src/network_beacon_processor/sync_methods.rs

* Merge branch 'unstable' into das-fetch-blobs

* Publish block without waiting for blob and column proof computation.

* Address review comments and refactor.

* Merge branch 'unstable' into das-fetch-blobs

* Fix test and docs.

* Comment cleanups.

* Merge branch 'unstable' into das-fetch-blobs

* Address review comments and cleanup

* Address review comments and cleanup

* Refactor to de-duplicate gradual publication logic.

* Add more logging.

* Merge remote-tracking branch 'origin/unstable' into das-fetch-blobs

# Conflicts:
#	Cargo.lock

* Fix incorrect comparison on `num_fetched_blobs`.

* Implement gradual blob publication.

* Merge branch 'unstable' into das-fetch-blobs

* Inline `publish_fn`.

* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs

* Gossip verify blobs before publishing

* Avoid queries for 0 blobs and error for duplicates

* Gossip verified engine blob before processing them, and use observe cache to detect duplicates before publishing.

* Merge branch 'das-fetch-blobs' of github.com:jimmygchen/lighthouse into das-fetch-blobs

# Conflicts:
#	beacon_node/network/src/network_beacon_processor/mod.rs

* Merge branch 'unstable' into das-fetch-blobs

* Fix invalid commitment inclusion proofs in blob sidecars created from EL blobs.

* Only publish EL blobs triggered from gossip block, and not RPC block.

* Downgrade gossip blob log to `debug`.

* Merge branch 'unstable' into das-fetch-blobs

* Merge branch 'unstable' into das-fetch-blobs

* Grammar
2024-11-15 03:34:13 +00:00