Commit Graph

5162 Commits

Author SHA1 Message Date
Michael Sproul
f77e3bc0ad Add maxperf build profile (#3608)
## Proposed Changes

Add a new Cargo compilation profile called `maxperf` which enables more aggressive compiler optimisations at the expense of compilation time.

Some rough initial benchmarks show that this can provide up to a 25% reduction to run time for CPU bound tasks like block processing: https://docs.google.com/spreadsheets/d/15jHuZe7lLHhZq9Nw8kc6EL0Qh_N_YAYqkW2NQ_Afmtk/edit

The numbers in that spreadsheet compare the `consensus-context` branch from #3604 to the same branch compiled with the `maxperf` profile using:

```
PROFILE=maxperf make install-lcli
```

## Additional Info

The downsides of the maxperf profile are:

- It increases compile times substantially, which will particularly impact low-spec hardware. Compiling `lcli` is about 3x slower. Compiling Lighthouse is about 5x slower on my 5950X: 17m 38s rather than 3m 28s.

As a result I think we should not enable this everywhere by default.

- **Option 1**: enable by default for our released binaries. This gives the majority of users the fastest version of `lighthouse` possible, at the expense of slowing down our release CI. Source builds will continue to use the default `release` profile unless users opt-in to `maxperf`.
- **Option 2**: enable by default for source builds. This gives users building from source an edge, but makes them pay for it with compilation time. 

I think I would prefer Option 1. I'll try doing some benchmarking to see how long a maxperf build of Lighthouse would take on GitHub actions.

Credit to Nicholas Nethercote for documenting these options in the Rust Performance Book: https://nnethercote.github.io/perf-book/build-configuration.html.
2022-09-29 06:13:33 +00:00
tim gretler
8d325e700b Use #!/usr/bin/env everywhere for local testnets (#3606)
Full local testnet support for people that don't have `/bin/bash`
2022-09-29 06:13:30 +00:00
Age Manning
27bb9ff07d Handle Lodestar's new agent string (#3620)
## Issue Addressed

#3561 

## Proposed Changes

Recognize Lodestars new agent string and appropriately count these peers as lodestar peers.
2022-09-29 01:50:13 +00:00
Age Manning
01b6bf7a2d Improve logging a little (#3619)
Some of the logs in combination with others could be improved. 

It will save some time debugging by improving the wording slightly.
2022-09-29 01:50:12 +00:00
Divma
b1d2510d1b Libp2p v0.48.0 upgrade (#3547)
## Issue Addressed

Upgrades libp2p to v.0.47.0. This is the compilation of
- [x] #3495 
- [x] #3497 
- [x] #3491 
- [x] #3546 
- [x] #3553 

Co-authored-by: Age Manning <Age@AgeManning.com>
2022-09-29 01:50:11 +00:00
Pawan Dhananjay
6779912fe4 Publish subscriptions to all beacon nodes (#3529)
## Issue Addressed

Resolves #3516 

## Proposed Changes

Adds a beacon fallback function for running a beacon node http query on all available fallbacks instead of returning on a first successful result. Uses the new `run_on_all` method for attestation and sync committee subscriptions. 

## Additional Info

Please provide any additional information. For example, future considerations
or information useful for reviewers.
2022-09-28 19:53:35 +00:00
Michael Sproul
abcebf276f Add guide to MEV logs (#3611)
## Proposed Changes

Add some docs on checking the builder configuration, which is a frequently asked question on Discord.

## Additional Info

My text editor also insisted on stripping some trailing newlines, but can put 'em back if we want
2022-09-28 17:45:09 +00:00
Paul Hauner
01e84b71f5 v3.1.2 (#3603)
## Issue Addressed

NA

## Proposed Changes

Bump versions to v3.1.2

## Additional Info

- ~~Blocked on several PRs.~~
- ~~Requires further testing.~~
v3.1.2
2022-09-26 01:17:36 +00:00
Divma
bd873e7162 New rust lints for rustc 1.64.0 (#3602)
## Issue Addressed
fixes lints from the last rust release

## Proposed Changes
Fix the lints, most of the lints by `clippy::question-mark` are false positives in the form of https://github.com/rust-lang/rust-clippy/issues/9518 so it's allowed for now

## Additional Info
2022-09-23 03:52:46 +00:00
Divma
9bd384a573 send attnet unsubscription event on random subnet expiry (#3600)
## Issue Addressed
🐞 in which we don't actually unsubscribe from a random long lived subnet when it expires

## Proposed Changes

Remove code addressing a specific case in which we are subscribed to all subnets and handle the removal of the long lived subnet. I don't think the special case code is particularly important as, if someone is running with that many validators to be subscribed to all subnets, it should use `--subscribe-all-subnets` instead

## Additional Info

Noticed on some test nodes climbing bandwidth usage periodically (around 27hours, the time of subnet expirations) I'm running this code to test this does not happen anymore, but I think it should be good now
2022-09-23 03:52:45 +00:00
Paul Hauner
9246a92d76 Make garbage collection test less failure prone (#3599)
## Issue Addressed

NA

## Proposed Changes

This PR attempts to fix the following spurious CI failure:

```
---- store_tests::garbage_collect_temp_states_from_failed_block stdout ----
thread 'store_tests::garbage_collect_temp_states_from_failed_block' panicked at 'disk store should initialize: DBError { message: "Error { message: \"IO error: lock /tmp/.tmp6DcBQ9/cold_db/LOCK: already held by process\" }" }', beacon_node/beacon_chain/tests/store_tests.rs:59:10
```

I believe that some async task is taking a clone of the store and holding it in some other thread for a short time. This creates a race-condition when we try to open a new instance of the store.

## Additional Info

NA
2022-09-23 03:52:44 +00:00
Pawan Dhananjay
3a3dddc5fb Fix ee integration tests (#3592)
## Issue Addressed

Resolves #3573 

## Proposed Changes

Fix the bytecode for the deposit contract deployment transaction and value for deposit transaction in the execution engine integration tests. Also verify that all published transaction make it to the execution payload and have a valid status.
2022-09-23 03:52:43 +00:00
Paul Hauner
fa6ad1a11a Deduplicate block root computation (#3590)
## Issue Addressed

NA

## Proposed Changes

This PR removes duplicated block root computation.

Computing the `SignedBeaconBlock::canonical_root` has become more expensive since the merge as we need to compute the merke root of each transaction inside an `ExecutionPayload`.

Computing the root for [a mainnet block](https://beaconcha.in/slot/4704236) is taking ~10ms on my i7-8700K CPU @ 3.70GHz (no sha extensions). Given that our median seen-to-imported time for blocks is presently 300-400ms, removing a few duplicated block roots (~30ms) could represent an easy 10% improvement. When we consider that the seen-to-imported times include operations *after* the block has been placed in the early attester cache, we could expect the 30ms to be more significant WRT our seen-to-attestable times.

## Additional Info

NA
2022-09-23 03:52:42 +00:00
Ramana Kumar
76ba0a1aaf Add disable-log-timestamp flag (#3101) (#3586)
## Issues Addressed

Closes https://github.com/sigp/lighthouse/issues/3101

## Proposed Changes

Add global flag to suppress timestamps in the terminal logger.
2022-09-23 03:52:41 +00:00
Paul Hauner
dbe2fa4c1b Add comment about amount field 2022-09-22 16:46:03 +10:00
Paul Hauner
6569a5c93e Remove comments from deposit amount 2022-09-22 16:46:03 +10:00
Paul Hauner
c60b9c6ace Revert "Add password migrate command"
This reverts commit 36477a004f2c8afac4c0b8e8fa8c4473dc74175b.
2022-09-22 16:46:02 +10:00
Paul Hauner
2abb4a7703 Add password migrate command 2022-09-22 16:46:02 +10:00
Paul Hauner
9dfea42513 Add more to docs 2022-09-22 16:46:02 +10:00
Paul Hauner
6af351a075 Update docs 2022-09-22 16:46:02 +10:00
Paul Hauner
4d790027b7 Add dodgy progress to docs 2022-09-22 16:46:02 +10:00
Paul Hauner
9539ca2165 Add "done" message 2022-09-22 16:46:02 +10:00
Paul Hauner
da2a8badc7 Start adding docs 2022-09-22 16:46:02 +10:00
Paul Hauner
07196804e0 Remove naughty return Err 2022-09-22 16:46:01 +10:00
Paul Hauner
4ccb7cbf4d Exit early on keystore upload error 2022-09-22 16:46:01 +10:00
Paul Hauner
b2887f8b55 Ensure password dir exists in val dir builder 2022-09-22 16:46:01 +10:00
Paul Hauner
fdd0311507 Strip newlines from VC token 2022-09-22 16:46:01 +10:00
Paul Hauner
d1aff84f83 Handle a missing command 2022-09-22 16:46:01 +10:00
Paul Hauner
47eaa50c06 Shorten "validator-client" to "vc" 2022-09-22 16:46:01 +10:00
Paul Hauner
2abde0b666 Allow VC to create password files via HTTP API 2022-09-22 16:46:00 +10:00
Paul Hauner
be8463770f Add count to move, fix tests 2022-09-22 16:46:00 +10:00
Paul Hauner
1f8f12541b Add keystore export flag to VC 2022-09-22 16:46:00 +10:00
Paul Hauner
e8da2810ee Revert "Use common version of regex across workspace"
This reverts commit 7f74036465fbf1aec89b056363177c074b7ab978.
2022-09-22 16:46:00 +10:00
Paul Hauner
89b233449c Use common version of regex across workspace 2022-09-22 16:46:00 +10:00
Paul Hauner
e5476cd6b7 Fix tests for windows 2022-09-22 16:46:00 +10:00
Paul Hauner
7f89f1efa7 Add move tests, "builder-proposals" takes bool 2022-09-22 16:46:00 +10:00
Paul Hauner
0c786c0ee2 Add tests for duplicates 2022-09-22 16:45:59 +10:00
Paul Hauner
64decbc65a Handle status from import 2022-09-22 16:45:59 +10:00
Paul Hauner
0c32b5a48b Remove intermediate file from move 2022-09-22 16:45:59 +10:00
Paul Hauner
05bafee5a8 Retry rather than saving file to disk 2022-09-22 16:45:59 +10:00
Paul Hauner
40b1619317 Don't run validator-manager tests in debug 2022-09-22 16:45:59 +10:00
Paul Hauner
328a450b2f Revert abandoned changes 2022-09-22 16:45:59 +10:00
Paul Hauner
1b40dd7a47 Revert changes to fork choice tests 2022-09-22 16:45:59 +10:00
Paul Hauner
bdffff1696 Remove abandoned files 2022-09-22 16:45:58 +10:00
Paul Hauner
65e0fe2fb0 Fix failing tests 2022-09-22 16:45:58 +10:00
Paul Hauner
06fa52ef5a Improve tests 2022-09-22 16:45:58 +10:00
Paul Hauner
0f5092e262 Fix clippy lint 2022-09-22 16:45:58 +10:00
Paul Hauner
bd00a795cb Remove commented-out code 2022-09-22 16:45:58 +10:00
Paul Hauner
43b16b138d Fix clippy lints 2022-09-22 16:45:58 +10:00
Paul Hauner
d20e65a271 Progress with testing 2022-09-22 16:45:58 +10:00