Instrument tracing spans for block processing and import (#7816)

#7815

- removes all existing spans, so some span fields that appear in logs like `service_name` may be lost.
- instruments a few key code paths in the beacon node, starting from **root spans** named below:

* Gossip block and blobs
* `process_gossip_data_column_sidecar`
* `process_gossip_blob`
* `process_gossip_block`
* Rpc block and blobs
* `process_rpc_block`
* `process_rpc_blobs`
* `process_rpc_custody_columns`
* Rpc blocks (range and backfill)
* `process_chain_segment`
* `PendingComponents` lifecycle
* `pending_components`

To test locally:
* Run Grafana and Tempo with https://github.com/sigp/lighthouse-metrics/pull/57
* Run Lighthouse BN with `--telemetry-collector-url http://localhost:4317`

Some captured traces can be found here: https://hackmd.io/@jimmygchen/r1sLOxPPeg

Removing the old spans seem to have reduced the memory usage quite a lot - i think we were using them on long running tasks and too excessively:
<img width="910" height="495" alt="image" src="https://github.com/user-attachments/assets/5208bbe4-53b2-4ead-bc71-0b782c788669" />
This commit is contained in:
Jimmy Chen
2025-08-08 15:32:22 +10:00
committed by GitHub
parent 6dfab22267
commit 40c2fd5ff4
52 changed files with 633 additions and 1164 deletions

View File

@@ -392,6 +392,9 @@ Options:
database.
--target-peers <target-peers>
The target number of peers.
--telemetry-collector-url <URL>
URL of the OpenTelemetry collector to export tracing spans (e.g.,
http://localhost:4317). If not set, tracing export is disabled.
--trusted-peers <TRUSTED_PEERS>
One or more comma-delimited trusted peer ids which always have the
highest score according to the peer scoring system.

View File

@@ -76,6 +76,9 @@ Options:
Path to directory containing eth2_testnet specs. Defaults to a
hard-coded Lighthouse testnet. Only effective if there is no existing
database.
--telemetry-collector-url <URL>
URL of the OpenTelemetry collector to export tracing spans (e.g.,
http://localhost:4317). If not set, tracing export is disabled.
-V, --version
Print version

View File

@@ -134,6 +134,9 @@ Options:
Path to directory containing eth2_testnet specs. Defaults to a
hard-coded Lighthouse testnet. Only effective if there is no existing
database.
--telemetry-collector-url <URL>
URL of the OpenTelemetry collector to export tracing spans (e.g.,
http://localhost:4317). If not set, tracing export is disabled.
--validator-registration-batch-size <INTEGER>
Defines the number of validators per validator/register_validator
request sent to the BN. This value can be reduced to avoid timeouts

View File

@@ -77,6 +77,9 @@ Options:
Path to directory containing eth2_testnet specs. Defaults to a
hard-coded Lighthouse testnet. Only effective if there is no existing
database.
--telemetry-collector-url <URL>
URL of the OpenTelemetry collector to export tracing spans (e.g.,
http://localhost:4317). If not set, tracing export is disabled.
Flags:
--disable-log-timestamp

View File

@@ -93,6 +93,9 @@ Options:
Path to directory containing eth2_testnet specs. Defaults to a
hard-coded Lighthouse testnet. Only effective if there is no existing
database.
--telemetry-collector-url <URL>
URL of the OpenTelemetry collector to export tracing spans (e.g.,
http://localhost:4317). If not set, tracing export is disabled.
Flags:
--disable-deposits

View File

@@ -73,6 +73,9 @@ Options:
Path to directory containing eth2_testnet specs. Defaults to a
hard-coded Lighthouse testnet. Only effective if there is no existing
database.
--telemetry-collector-url <URL>
URL of the OpenTelemetry collector to export tracing spans (e.g.,
http://localhost:4317). If not set, tracing export is disabled.
--validators-file <PATH_TO_JSON_FILE>
The path to a JSON file containing a list of validators to be imported
to the validator client. This file is usually named "validators.json".

View File

@@ -82,6 +82,9 @@ Options:
Path to directory containing eth2_testnet specs. Defaults to a
hard-coded Lighthouse testnet. Only effective if there is no existing
database.
--telemetry-collector-url <URL>
URL of the OpenTelemetry collector to export tracing spans (e.g.,
http://localhost:4317). If not set, tracing export is disabled.
--validators <STRING>
The validators to be moved. Either a list of 0x-prefixed validator
pubkeys or the keyword "all".