gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-06 18:21:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

Devnet6 (#4404)

Browse Source 
* some blob reprocessing work

* remove ForceBlockLookup

* reorder enum match arms in sync manager

* a lot more reprocessing work

* impl logic for triggerng blob lookups along with block lookups

* deal with rpc blobs in groups per block in the da checker. don't cache missing blob ids in the da checker.

* make single block lookup generic

* more work

* add delayed processing logic and combine some requests

* start fixing some compile errors

* fix compilation in main block lookup mod

* much work

* get things compiling

* parent blob lookups

* fix compile

* revert red/stevie changes

* fix up sync manager delay message logic

* add peer usefulness enum

* should remove lookup refactor

* consolidate retry error handling

* improve peer scoring during certain failures in parent lookups

* improve retry code

* drop parent lookup if either req has a peer disconnect during download

* refactor single block processed method

* processing peer refactor

* smol bugfix

* fix some todos

* fix lints

* fix lints

* fix compile in lookup tests

* fix lints

* fix lints

* fix existing block lookup tests

* renamings

* fix after merge

* cargo fmt

* compilation fix in beacon chain tests

* fix

* refactor lookup tests to work with multiple forks and response types

* make tests into macros

* wrap availability check error

* fix compile after merge

* add random blobs

* start fixing up lookup verify error handling

* some bug fixes and the start of deneb only tests

* make tests work for all forks

* track information about peer source

* error refactoring

* improve peer scoring

* fix test compilation

* make sure blobs are sent for processing after stream termination, delete copied tests

* add some tests and fix a bug

* smol bugfixes and moar tests

* add tests and fix some things

* compile after merge

* lots of refactoring

* retry on invalid block/blob

* merge unknown parent messages before current slot lookup

* get tests compiling

* penalize blob peer on invalid blobs

* Check disk on in-memory cache miss

* Update beacon_node/beacon_chain/src/data_availability_checker/overflow_lru_cache.rs

* Update beacon_node/network/src/sync/network_context.rs

Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>

* fix bug in matching blocks and blobs in range sync

* pr feedback

* fix conflicts

* upgrade logs from warn to crit when we receive incorrect response in range

* synced_and_connected_within_tolerance -> should_search_for_block

* remove todo

* add data gas used and update excess data gas to u64

* Fix Broken Overflow Tests

* payload verification with commitments

* fix merge conflicts

* restore payload file

* Restore payload file

* remove todo

* add max blob commitments per block

* c-kzg lib update

* Fix ef tests

* Abstract over minimal/mainnet spec in kzg crate

* Start integrating new KZG

* checkpoint sync without alignment

* checkpoint sync without alignment

* add import

* add import

* query for checkpoint state by slot rather than state root (teku doesn't serve by state root)

* query for checkpoint state by slot rather than state root (teku doesn't serve by state root)

* loosen check

* get state first and query by most recent block root

* Revert "loosen check"

This reverts commit 069d13dd63.

* get state first and query by most recent block root

* merge max blobs change

* simplify delay logic

* rename unknown parent sync message variants

* rename parameter, block_slot -> slot

* add some docs to the lookup module

* use interval instead of sleep

* drop request if blocks and blobs requests both return `None` for `Id`

* clean up `find_single_lookup` logic

* add lookup source enum

* clean up `find_single_lookup` logic

* add docs to find_single_lookup_request

* move LookupSource our of param where unnecessary

* remove unnecessary todo

* query for block by `state.latest_block_header.slot`

* fix lint

* fix merge transition ef tests

* fix test

* fix test

* fix observed  blob sidecars test

* Add some metrics (#33)

* fix protocol limits for blobs by root

* Update Engine API for 1:1 Structure Method

* make beacon chain tests to fix devnet 6 changes

* get ckzg working and fix some tests

* fix remaining tests

* fix lints

* Fix KZG linking issues

* remove unused dep

* lockfile

* test fixes

* remove dbgs

* remove unwrap

* cleanup tx generator

* small fixes

* fixing fixes

* more self reivew

* more self review

* refactor genesis header initialization

* refactor mock el instantiations

* fix compile

* fix network test, make sure they run for each fork

* pr feedback

* fix last test (hopefully)

---------

Co-authored-by: Pawan Dhananjay <pawandhananjay@gmail.com>
Co-authored-by: Mark Mackey <mark@sigmaprime.io>
Co-authored-by: Divma <26765164+divagant-martian@users.noreply.github.com>
Co-authored-by: Michael Sproul <michael@sigmaprime.io>
This commit is contained in:
realbigsean
2023-06-29 15:35:43 -04:00
committed by GitHub
parent 4c9fcf1e83
commit adbb62f7f3
69 changed files with 2114 additions and 1338 deletions
Download Patch File Download Diff File Expand all files Collapse all files

8
beacon_node/execution_layer/src/block_hash.rs
View File

@@ -37,6 +37,7 @@ impl<T: EthSpec> ExecutionLayer<T> {
None
};
let rlp_data_gas_used = payload.data_gas_used().ok();
let rlp_excess_data_gas = payload.excess_data_gas().ok();
// Construct the block header.
@@ -45,6 +46,7 @@ impl<T: EthSpec> ExecutionLayer<T> {
KECCAK_EMPTY_LIST_RLP.as_fixed_bytes().into(),
rlp_transactions_root,
rlp_withdrawals_root,
rlp_data_gas_used.copied(),
rlp_excess_data_gas.copied(),
);
@@ -97,6 +99,9 @@ pub fn rlp_encode_block_header(header: &ExecutionBlockHeader) -> Vec<u8> {
if let Some(withdrawals_root) = &header.withdrawals_root {
rlp_header_stream.append(withdrawals_root);
}
if let Some(data_gas_used) = &header.data_gas_used {
rlp_header_stream.append(data_gas_used);
}
if let Some(excess_data_gas) = &header.excess_data_gas {
rlp_header_stream.append(excess_data_gas);
}
@@ -146,6 +151,7 @@ mod test {
nonce: Hash64::zero(),
base_fee_per_gas: 0x036b_u64.into(),
withdrawals_root: None,
data_gas_used: None,
excess_data_gas: None,
};
let expected_rlp = "f90200a0e0a94a7a3c9617401586b1a27025d2d9671332d22d540e0af72b069170380f2aa01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d4934794ba5e000000000000000000000000000000000000a0ec3c94b18b8a1cff7d60f8d258ec723312932928626b4c9355eb4ab3568ec7f7a050f738580ed699f0469702c7ccc63ed2e51bc034be9479b7bff4e68dee84accfa029b0562f7140574dd0d50dee8a271b22e1a0a7b78fca58f7c60370d8317ba2a9b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000830200000188016345785d8a00008301553482079e42a0000000000000000000000000000000000000000000000000000000000000000088000000000000000082036b";
@@ -175,6 +181,7 @@ mod test {
nonce: Hash64::zero(),
base_fee_per_gas: 0x036b_u64.into(),
withdrawals_root: None,
data_gas_used: None,
excess_data_gas: None,
};
let expected_rlp = "f901fda0927ca537f06c783a3a2635b8805eef1c8c2124f7444ad4a3389898dd832f2dbea01dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d4934794ba5e000000000000000000000000000000000000a0e97859b065bd8dbbb4519c7cb935024de2484c2b7f881181b4360492f0b06b82a050f738580ed699f0469702c7ccc63ed2e51bc034be9479b7bff4e68dee84accfa029b0562f7140574dd0d50dee8a271b22e1a0a7b78fca58f7c60370d8317ba2a9b9010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000800188016345785d8a00008301553482079e42a0000000000000000000000000000000000000000000000000000000000002000088000000000000000082036b";
@@ -205,6 +212,7 @@ mod test {
nonce: Hash64::zero(),
base_fee_per_gas: 0x34187b238_u64.into(),
withdrawals_root: None,
data_gas_used: None,
excess_data_gas: None,
};
let expected_hash =

11
beacon_node/execution_layer/src/engine_api.rs
View File

@@ -7,7 +7,7 @@ use crate::http::{
};
use crate::BlobTxConversionError;
use eth2::types::{SsePayloadAttributes, SsePayloadAttributesV1, SsePayloadAttributesV2};
pub use ethers_core::types::Transaction;
use ethers_core::types::Transaction;
use ethers_core::utils::rlp::{self, Decodable, Rlp};
use http::deposit_methods::RpcError;
pub use json_structures::{JsonWithdrawal, TransitionConfigurationV1};
@@ -190,8 +190,11 @@ pub struct ExecutionBlockWithTransactions<T: EthSpec> {
#[superstruct(only(Capella, Deneb))]
pub withdrawals: Vec<JsonWithdrawal>,
#[superstruct(only(Deneb))]
#[serde(with = "serde_utils::u256_hex_be")]
pub excess_data_gas: Uint256,
#[serde(with = "serde_utils::u64_hex_be")]
pub data_gas_used: u64,
#[superstruct(only(Deneb))]
#[serde(with = "serde_utils::u64_hex_be")]
pub excess_data_gas: u64,
}
impl<T: EthSpec> TryFrom<ExecutionPayload<T>> for ExecutionBlockWithTransactions<T> {
@@ -268,6 +271,7 @@ impl<T: EthSpec> TryFrom<ExecutionPayload<T>> for ExecutionBlockWithTransactions
.into_iter()
.map(|withdrawal| withdrawal.into())
.collect(),
data_gas_used: block.data_gas_used,
excess_data_gas: block.excess_data_gas,
}),
};
@@ -510,6 +514,7 @@ impl<E: EthSpec> ExecutionPayloadBodyV1<E> {
block_hash: header.block_hash,
transactions: self.transactions,
withdrawals,
data_gas_used: header.data_gas_used,
excess_data_gas: header.excess_data_gas,
}))
} else {

91
beacon_node/execution_layer/src/engine_api/http.rs
View File

@@ -11,7 +11,7 @@ use std::collections::HashSet;
use tokio::sync::Mutex;
use std::time::{Duration, Instant};
use types::EthSpec;
use types::{EthSpec, VersionedHash};
pub use deposit_log::{DepositLog, Log};
pub use reqwest::Client;
@@ -807,9 +807,13 @@ impl HttpJsonRpc {
pub async fn new_payload_v3<T: EthSpec>(
&self,
execution_payload: ExecutionPayload<T>,
execution_payload: ExecutionPayloadDeneb<T>,
versioned_hashes: Vec<VersionedHash>,
) -> Result<PayloadStatusV1, Error> {
let params = json!([JsonExecutionPayload::from(execution_payload)]);
let params = json!([
JsonExecutionPayload::V3(execution_payload.into()),
versioned_hashes
]);
let response: JsonPayloadStatusV1 = self
.rpc_request(
@@ -887,26 +891,6 @@ impl HttpJsonRpc {
let params = json!([JsonPayloadIdRequest::from(payload_id)]);
match fork_name {
ForkName::Merge => {
let response: JsonGetPayloadResponseV1<T> = self
.rpc_request(
ENGINE_GET_PAYLOAD_V2,
params,
ENGINE_GET_PAYLOAD_TIMEOUT * self.execution_timeout_multiplier,
)
.await?;
Ok(JsonGetPayloadResponse::V1(response).into())
}
ForkName::Capella => {
let response: JsonGetPayloadResponseV2<T> = self
.rpc_request(
ENGINE_GET_PAYLOAD_V2,
params,
ENGINE_GET_PAYLOAD_TIMEOUT * self.execution_timeout_multiplier,
)
.await?;
Ok(JsonGetPayloadResponse::V2(response).into())
}
ForkName::Deneb => {
let response: JsonGetPayloadResponseV3<T> = self
.rpc_request(
@@ -917,7 +901,7 @@ impl HttpJsonRpc {
.await?;
Ok(JsonGetPayloadResponse::V3(response).into())
}
ForkName::Base | ForkName::Altair => Err(Error::UnsupportedForkVariant(format!(
_ => Err(Error::UnsupportedForkVariant(format!(
"called get_payload_v3 with {}",
fork_name
))),
@@ -1099,16 +1083,30 @@ impl HttpJsonRpc {
pub async fn new_payload<T: EthSpec>(
&self,
execution_payload: ExecutionPayload<T>,
versioned_hashes_opt: Option<Vec<VersionedHash>>,
) -> Result<PayloadStatusV1, Error> {
let engine_capabilities = self.get_engine_capabilities(None).await?;
if engine_capabilities.new_payload_v3 {
self.new_payload_v3(execution_payload).await
} else if engine_capabilities.new_payload_v2 {
self.new_payload_v2(execution_payload).await
} else if engine_capabilities.new_payload_v1 {
self.new_payload_v1(execution_payload).await
} else {
Err(Error::RequiredMethodUnsupported("engine_newPayload"))
match execution_payload {
ExecutionPayload::Merge(_) | ExecutionPayload::Capella(_) => {
if engine_capabilities.new_payload_v2 {
self.new_payload_v2(execution_payload).await
} else if engine_capabilities.new_payload_v1 {
self.new_payload_v1(execution_payload).await
} else {
Err(Error::RequiredMethodUnsupported("engine_newPayload"))
}
}
ExecutionPayload::Deneb(execution_payload_deneb) => {
let Some(versioned_hashes) = versioned_hashes_opt else {
return Err(Error::IncorrectStateVariant);
};
if engine_capabilities.new_payload_v3 {
self.new_payload_v3(execution_payload_deneb, versioned_hashes)
.await
} else {
Err(Error::RequiredMethodUnsupported("engine_newPayloadV3"))
}
}
}
}
@@ -1120,14 +1118,27 @@ impl HttpJsonRpc {
payload_id: PayloadId,
) -> Result<GetPayloadResponse<T>, Error> {
let engine_capabilities = self.get_engine_capabilities(None).await?;
if engine_capabilities.get_payload_v3 {
self.get_payload_v3(fork_name, payload_id).await
} else if engine_capabilities.get_payload_v2 {
self.get_payload_v2(fork_name, payload_id).await
} else if engine_capabilities.new_payload_v1 {
self.get_payload_v1(payload_id).await
} else {
Err(Error::RequiredMethodUnsupported("engine_getPayload"))
match fork_name {
ForkName::Merge | ForkName::Capella => {
if engine_capabilities.get_payload_v2 {
self.get_payload_v2(fork_name, payload_id).await
} else if engine_capabilities.new_payload_v1 {
self.get_payload_v1(payload_id).await
} else {
Err(Error::RequiredMethodUnsupported("engine_getPayload"))
}
}
ForkName::Deneb => {
if engine_capabilities.get_payload_v3 {
self.get_payload_v3(fork_name, payload_id).await
} else {
Err(Error::RequiredMethodUnsupported("engine_getPayloadV3"))
}
}
ForkName::Base | ForkName::Altair => Err(Error::UnsupportedForkVariant(format!(
"called get_payload with {}",
fork_name
))),
}
}

9
beacon_node/execution_layer/src/engine_api/json_structures.rs
View File

@@ -99,8 +99,11 @@ pub struct JsonExecutionPayload<T: EthSpec> {
#[superstruct(only(V2, V3))]
pub withdrawals: VariableList<JsonWithdrawal, T::MaxWithdrawalsPerPayload>,
#[superstruct(only(V3))]
#[serde(with = "serde_utils::u256_hex_be")]
pub excess_data_gas: Uint256,
#[serde(with = "serde_utils::u64_hex_be")]
pub data_gas_used: u64,
#[superstruct(only(V3))]
#[serde(with = "serde_utils::u64_hex_be")]
pub excess_data_gas: u64,
}
impl<T: EthSpec> From<ExecutionPayloadMerge<T>> for JsonExecutionPayloadV1<T> {
@@ -172,6 +175,7 @@ impl<T: EthSpec> From<ExecutionPayloadDeneb<T>> for JsonExecutionPayloadV3<T> {
.map(Into::into)
.collect::<Vec<_>>()
.into(),
data_gas_used: payload.data_gas_used,
excess_data_gas: payload.excess_data_gas,
}
}
@@ -256,6 +260,7 @@ impl<T: EthSpec> From<JsonExecutionPayloadV3<T>> for ExecutionPayloadDeneb<T> {
.map(Into::into)
.collect::<Vec<_>>()
.into(),
data_gas_used: payload.data_gas_used,
excess_data_gas: payload.excess_data_gas,
}
}

194
beacon_node/execution_layer/src/lib.rs
View File

@@ -15,8 +15,7 @@ use engines::{Engine, EngineError};
pub use engines::{EngineState, ForkchoiceState};
use eth2::types::{builder_bid::SignedBuilderBid, ForkVersionedResponse};
use ethers_core::abi::ethereum_types::FromStrRadixErr;
use ethers_core::types::transaction::eip2930::AccessListItem;
use ethers_core::types::{Transaction as EthersTransaction, U64};
use ethers_core::types::Transaction as EthersTransaction;
use fork_choice::ForkchoiceUpdateParameters;
use lru::LruCache;
use payload_status::process_payload_status;
@@ -25,7 +24,6 @@ use sensitive_url::SensitiveUrl;
use serde::{Deserialize, Serialize};
use slog::{crit, debug, error, info, trace, warn, Logger};
use slot_clock::SlotClock;
use ssz::Encode;
use std::collections::HashMap;
use std::fmt;
use std::future::Future;
@@ -43,8 +41,6 @@ use tokio_stream::wrappers::WatchStream;
use tree_hash::TreeHash;
use types::beacon_block_body::KzgCommitments;
use types::blob_sidecar::Blobs;
use types::consts::deneb::BLOB_TX_TYPE;
use types::transaction::{AccessTuple, BlobTransaction, EcdsaSignature, SignedBlobTransaction};
use types::{AbstractExecPayload, BeaconStateError, ExecPayload, VersionedHash};
use types::{
BlindedPayload, BlockType, ChainSpec, Epoch, ExecutionBlockHash, ExecutionPayload,
@@ -1217,6 +1213,7 @@ impl<T: EthSpec> ExecutionLayer<T> {
pub async fn notify_new_payload(
&self,
execution_payload: &ExecutionPayload<T>,
versioned_hashes: Option<Vec<VersionedHash>>,
) -> Result<PayloadStatus, Error> {
let _timer = metrics::start_timer_vec(
&metrics::EXECUTION_LAYER_REQUEST_TIMES,
@@ -1233,7 +1230,11 @@ impl<T: EthSpec> ExecutionLayer<T> {
let result = self
.engine()
.request(|engine| engine.api.new_payload(execution_payload.clone()))
.request(|engine| {
engine
.api
.new_payload(execution_payload.clone(), versioned_hashes)
})
.await;
if let Ok(status) = &result {
@@ -1786,7 +1787,7 @@ impl<T: EthSpec> ExecutionLayer<T> {
VariableList::new(
transactions
.into_iter()
.map(ethers_tx_to_bytes::<T>)
.map(ethers_tx_to_ssz::<T>)
.collect::<Result<Vec<_>, BlobTxConversionError>>()?,
)
.map_err(BlobTxConversionError::SszError)
@@ -1863,6 +1864,7 @@ impl<T: EthSpec> ExecutionLayer<T> {
block_hash: deneb_block.block_hash,
transactions: convert_transactions(deneb_block.transactions)?,
withdrawals,
data_gas_used: deneb_block.data_gas_used,
excess_data_gas: deneb_block.excess_data_gas,
})
}
@@ -2170,159 +2172,35 @@ impl From<serde_json::Error> for BlobTxConversionError {
}
}
/// A utility function to convert a `ethers-rs` `Transaction` into the correct bytes encoding based
/// on transaction type. That means RLP encoding if this is a transaction other than a
/// `BLOB_TX_TYPE` transaction in which case, SSZ encoding will be used.
fn ethers_tx_to_bytes<T: EthSpec>(
transaction: EthersTransaction,
fn random_valid_tx<T: EthSpec>(
) -> Result<Transaction<T::MaxBytesPerTransaction>, BlobTxConversionError> {
let tx_type = transaction
.transaction_type
.ok_or(BlobTxConversionError::NoTransactionType)?
.as_u64();
// Calculate transaction bytes. We don't care about the contents of the transaction.
let transaction: EthersTransaction = serde_json::from_str(
r#"{
"blockHash":"0x1d59ff54b1eb26b013ce3cb5fc9dab3705b415a67127a003c3e61eb445bb8df2",
"blockNumber":"0x5daf3b",
"from":"0xa7d9ddbe1f17865597fbd27ec712455208b6b76d",
"gas":"0xc350",
"gasPrice":"0x4a817c800",
"hash":"0x88df016429689c079f3b2f6ad39fa052532c56795b733da78a91ebe6a713944b",
"input":"0x68656c6c6f21",
"nonce":"0x15",
"to":"0xf02c1c8e6114b1dbe8937a39260b5b0a374432bb",
"transactionIndex":"0x41",
"value":"0xf3dbb76162000",
"v":"0x25",
"r":"0x1b5e176d927f8e9ab405058b2d2457392da3e20f328b16ddabcebc33eaac5fea",
"s":"0x4ba69724e8f69de52f0125ad8b3c5c2cef33019bac3249e2c0a2192766d1721c"
}"#,
)
.unwrap();
ethers_tx_to_ssz::<T>(transaction)
}
let tx = if BLOB_TX_TYPE as u64 == tx_type {
let EthersTransaction {
hash: _,
nonce,
block_hash: _,
block_number: _,
transaction_index: _,
from: _,
to,
value,
gas_price: _,
gas,
input,
v,
r,
s,
transaction_type: _,
access_list,
max_priority_fee_per_gas,
max_fee_per_gas,
chain_id,
other,
} = transaction;
// ******************** BlobTransaction fields ********************
// chainId
let chain_id = chain_id.ok_or(BlobTxConversionError::NoChainId)?;
// nonce
let nonce = if nonce > Uint256::from(u64::MAX) {
return Err(BlobTxConversionError::NonceTooLarge);
} else {
nonce.as_u64()
};
// maxPriorityFeePerGas
let max_priority_fee_per_gas =
max_priority_fee_per_gas.ok_or(BlobTxConversionError::MaxPriorityFeePerGasMissing)?;
// maxFeePerGas
let max_fee_per_gas = max_fee_per_gas.ok_or(BlobTxConversionError::MaxFeePerGasMissing)?;
// gas
let gas = if gas > Uint256::from(u64::MAX) {
return Err(BlobTxConversionError::GasTooHigh);
} else {
gas.as_u64()
};
// data (a.k.a input)
let data = VariableList::new(input.to_vec())?;
// accessList
let access_list = VariableList::new(
access_list
.ok_or(BlobTxConversionError::AccessListMissing)?
.0
.into_iter()
.map(|access_tuple| {
let AccessListItem {
address,
storage_keys,
} = access_tuple;
Ok(AccessTuple {
address,
storage_keys: VariableList::new(storage_keys)?,
})
})
.collect::<Result<Vec<AccessTuple>, BlobTxConversionError>>()?,
)?;
// ******************** BlobTransaction `other` fields ********************
//
// Here we use the `other` field in the `ethers-rs` `Transaction` type because
// `ethers-rs` does not yet support SSZ and therefore the blobs transaction type.
// maxFeePerDataGas
let max_fee_per_data_gas = Uint256::from_str_radix(
other
.get("maxFeePerDataGas")
.ok_or(BlobTxConversionError::MaxFeePerDataGasMissing)?
.as_str()
.ok_or(BlobTxConversionError::MaxFeePerDataGasMissing)?,
16,
)
.map_err(BlobTxConversionError::FromStrRadix)?;
// versionedHashes
let versioned_hashes = other
.get("versionedHashes")
.ok_or(BlobTxConversionError::VersionedHashesMissing)?
.as_array()
.ok_or(BlobTxConversionError::VersionedHashesMissing)?
.iter()
.map(|versioned_hash| {
let hash_bytes = serde_utils::hex::decode(
versioned_hash
.as_str()
.ok_or(BlobTxConversionError::VersionedHashesMissing)?,
)
.map_err(BlobTxConversionError::FromHex)?;
if hash_bytes.len() != Hash256::ssz_fixed_len() {
Err(BlobTxConversionError::InvalidVersionedHashBytesLen)
} else {
Ok(Hash256::from_slice(&hash_bytes))
}
})
.collect::<Result<Vec<VersionedHash>, BlobTxConversionError>>()?;
let message = BlobTransaction {
chain_id,
nonce,
max_priority_fee_per_gas,
max_fee_per_gas,
gas,
to,
value,
data,
access_list,
max_fee_per_data_gas,
versioned_hashes: VariableList::new(versioned_hashes)?,
};
// ******************** EcdsaSignature fields ********************
let y_parity = if v == U64::zero() {
false
} else if v == U64::one() {
true
} else {
return Err(BlobTxConversionError::InvalidYParity);
};
let signature = EcdsaSignature { y_parity, r, s };
// The `BLOB_TX_TYPE` should prepend the SSZ encoded `SignedBlobTransaction`.
let mut signed_tx = SignedBlobTransaction { message, signature }.as_ssz_bytes();
signed_tx.insert(0, BLOB_TX_TYPE);
signed_tx
} else {
transaction.rlp().to_vec()
};
VariableList::new(tx).map_err(Into::into)
fn ethers_tx_to_ssz<T: EthSpec>(
tx: EthersTransaction,
) -> Result<Transaction<T::MaxBytesPerTransaction>, BlobTxConversionError> {
VariableList::new(tx.rlp().to_vec()).map_err(Into::into)
}
fn noop<T: EthSpec>(

288
beacon_node/execution_layer/src/test_utils/execution_block_generator.rs
View File

@@ -6,24 +6,23 @@ use crate::{
},
ExecutionBlock, PayloadAttributes, PayloadId, PayloadStatusV1, PayloadStatusV1Status,
},
BlobsBundleV1, ExecutionBlockWithTransactions,
random_valid_tx, BlobsBundleV1, ExecutionBlockWithTransactions,
};
use kzg::{Kzg, BYTES_PER_BLOB, BYTES_PER_FIELD_ELEMENT, FIELD_ELEMENTS_PER_BLOB};
use rand::RngCore;
use kzg::Kzg;
use rand::thread_rng;
use serde::{Deserialize, Serialize};
use ssz::Encode;
use std::collections::HashMap;
use std::sync::Arc;
use tree_hash::TreeHash;
use tree_hash_derive::TreeHash;
use types::consts::deneb::BLOB_TX_TYPE;
use types::transaction::{BlobTransaction, EcdsaSignature, SignedBlobTransaction};
use types::{
Blob, EthSpec, ExecutionBlockHash, ExecutionPayload, ExecutionPayloadCapella,
ExecutionPayloadDeneb, ExecutionPayloadMerge, ForkName, Hash256, Transaction, Transactions,
Uint256,
BlobSidecar, ChainSpec, EthSpec, ExecutionBlockHash, ExecutionPayload, ExecutionPayloadCapella,
ExecutionPayloadDeneb, ExecutionPayloadHeader, ExecutionPayloadMerge, ForkName, Hash256,
Transactions, Uint256,
};
use super::DEFAULT_TERMINAL_BLOCK;
const GAS_LIMIT: u64 = 16384;
const GAS_USED: u64 = GAS_LIMIT - 1;
@@ -125,12 +124,12 @@ pub struct ExecutionBlockGenerator<T: EthSpec> {
* Post-merge fork triggers
*/
pub shanghai_time: Option<u64>, // withdrawals
pub deneb_time: Option<u64>, // 4844
pub cancun_time: Option<u64>, // deneb
/*
* deneb stuff
*/
pub blobs_bundles: HashMap<PayloadId, BlobsBundleV1<T>>,
pub kzg: Option<Arc<Kzg>>,
pub kzg: Option<Arc<Kzg<T::Kzg>>>,
}
impl<T: EthSpec> ExecutionBlockGenerator<T> {
@@ -139,8 +138,8 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
terminal_block_number: u64,
terminal_block_hash: ExecutionBlockHash,
shanghai_time: Option<u64>,
deneb_time: Option<u64>,
kzg: Option<Kzg>,
cancun_time: Option<u64>,
kzg: Option<Kzg<T::Kzg>>,
) -> Self {
let mut gen = Self {
head_block: <_>::default(),
@@ -154,7 +153,7 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
next_payload_id: 0,
payload_ids: <_>::default(),
shanghai_time,
deneb_time,
cancun_time,
blobs_bundles: <_>::default(),
kzg: kzg.map(Arc::new),
};
@@ -189,7 +188,7 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
}
pub fn get_fork_at_timestamp(&self, timestamp: u64) -> ForkName {
match self.deneb_time {
match self.cancun_time {
Some(fork_time) if timestamp >= fork_time => ForkName::Deneb,
_ => match self.shanghai_time {
Some(fork_time) if timestamp >= fork_time => ForkName::Capella,
@@ -270,10 +269,15 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
finalized_block_hash
));
}
let parent_hash = if block_number == 0 {
ExecutionBlockHash::zero()
let block = if block_number == 0 {
generate_genesis_block(self.terminal_total_difficulty, self.terminal_block_number)?
} else if let Some(block) = self.block_by_number(block_number - 1) {
block.block_hash()
generate_pow_block(
self.terminal_total_difficulty,
self.terminal_block_number,
block_number,
block.block_hash(),
)?
} else {
return Err(format!(
"parent with block number {} not found",
@@ -281,13 +285,6 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
));
};
let block = generate_pow_block(
self.terminal_total_difficulty,
self.terminal_block_number,
block_number,
parent_hash,
)?;
// Insert block into block tree
self.insert_block(Block::PoW(block))?;
@@ -348,10 +345,10 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
Ok(hash)
}
// This does not reject duplicate blocks inserted. This lets us re-use the same execution
// block generator for multiple beacon chains which is useful in testing.
pub fn insert_block(&mut self, block: Block<T>) -> Result<ExecutionBlockHash, String> {
if self.blocks.contains_key(&block.block_hash()) {
return Err(format!("{:?} is already known", block.block_hash()));
} else if block.parent_hash() != ExecutionBlockHash::zero()
if block.parent_hash() != ExecutionBlockHash::zero()
&& !self.blocks.contains_key(&block.parent_hash())
{
return Err(format!("parent block {:?} is unknown", block.parent_hash()));
@@ -410,8 +407,7 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
}
pub fn get_blobs_bundle(&mut self, id: &PayloadId) -> Option<BlobsBundleV1<T>> {
// remove it to free memory
self.blobs_bundles.remove(id)
self.blobs_bundles.get(id).cloned()
}
pub fn new_payload(&mut self, payload: ExecutionPayload<T>) -> PayloadStatusV1 {
@@ -450,6 +446,14 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
forkchoice_state: ForkchoiceState,
payload_attributes: Option<PayloadAttributes>,
) -> Result<JsonForkchoiceUpdatedV1Response, String> {
// This is meant to cover starting post-merge transition at genesis. Useful for
// testing Capella forks and later.
if let Some(genesis_pow_block) = self.block_by_number(0) {
if genesis_pow_block.block_hash() == forkchoice_state.head_block_hash {
self.terminal_block_hash = forkchoice_state.head_block_hash;
}
}
if let Some(payload) = self
.pending_payloads
.remove(&forkchoice_state.head_block_hash)
@@ -518,67 +522,61 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
block_hash: ExecutionBlockHash::zero(),
transactions: vec![].into(),
}),
PayloadAttributes::V2(pa) => {
match self.get_fork_at_timestamp(pa.timestamp) {
ForkName::Merge => ExecutionPayload::Merge(ExecutionPayloadMerge {
parent_hash: forkchoice_state.head_block_hash,
fee_recipient: pa.suggested_fee_recipient,
receipts_root: Hash256::repeat_byte(42),
state_root: Hash256::repeat_byte(43),
logs_bloom: vec![0; 256].into(),
prev_randao: pa.prev_randao,
block_number: parent.block_number() + 1,
gas_limit: GAS_LIMIT,
gas_used: GAS_USED,
timestamp: pa.timestamp,
extra_data: "block gen was here".as_bytes().to_vec().into(),
base_fee_per_gas: Uint256::one(),
block_hash: ExecutionBlockHash::zero(),
transactions: vec![].into(),
}),
ForkName::Capella => {
ExecutionPayload::Capella(ExecutionPayloadCapella {
parent_hash: forkchoice_state.head_block_hash,
fee_recipient: pa.suggested_fee_recipient,
receipts_root: Hash256::repeat_byte(42),
state_root: Hash256::repeat_byte(43),
logs_bloom: vec![0; 256].into(),
prev_randao: pa.prev_randao,
block_number: parent.block_number() + 1,
gas_limit: GAS_LIMIT,
gas_used: GAS_USED,
timestamp: pa.timestamp,
extra_data: "block gen was here".as_bytes().to_vec().into(),
base_fee_per_gas: Uint256::one(),
block_hash: ExecutionBlockHash::zero(),
transactions: vec![].into(),
withdrawals: pa.withdrawals.clone().into(),
})
}
ForkName::Deneb => {
ExecutionPayload::Deneb(ExecutionPayloadDeneb {
parent_hash: forkchoice_state.head_block_hash,
fee_recipient: pa.suggested_fee_recipient,
receipts_root: Hash256::repeat_byte(42),
state_root: Hash256::repeat_byte(43),
logs_bloom: vec![0; 256].into(),
prev_randao: pa.prev_randao,
block_number: parent.block_number() + 1,
gas_limit: GAS_LIMIT,
gas_used: GAS_USED,
timestamp: pa.timestamp,
extra_data: "block gen was here".as_bytes().to_vec().into(),
base_fee_per_gas: Uint256::one(),
block_hash: ExecutionBlockHash::zero(),
transactions: vec![].into(),
withdrawals: pa.withdrawals.clone().into(),
// FIXME(deneb) maybe this should be set to something?
excess_data_gas: Uint256::one(),
})
}
_ => unreachable!(),
}
}
PayloadAttributes::V2(pa) => match self.get_fork_at_timestamp(pa.timestamp) {
ForkName::Merge => ExecutionPayload::Merge(ExecutionPayloadMerge {
parent_hash: forkchoice_state.head_block_hash,
fee_recipient: pa.suggested_fee_recipient,
receipts_root: Hash256::repeat_byte(42),
state_root: Hash256::repeat_byte(43),
logs_bloom: vec![0; 256].into(),
prev_randao: pa.prev_randao,
block_number: parent.block_number() + 1,
gas_limit: GAS_LIMIT,
gas_used: GAS_USED,
timestamp: pa.timestamp,
extra_data: "block gen was here".as_bytes().to_vec().into(),
base_fee_per_gas: Uint256::one(),
block_hash: ExecutionBlockHash::zero(),
transactions: vec![].into(),
}),
ForkName::Capella => ExecutionPayload::Capella(ExecutionPayloadCapella {
parent_hash: forkchoice_state.head_block_hash,
fee_recipient: pa.suggested_fee_recipient,
receipts_root: Hash256::repeat_byte(42),
state_root: Hash256::repeat_byte(43),
logs_bloom: vec![0; 256].into(),
prev_randao: pa.prev_randao,
block_number: parent.block_number() + 1,
gas_limit: GAS_LIMIT,
gas_used: GAS_USED,
timestamp: pa.timestamp,
extra_data: "block gen was here".as_bytes().to_vec().into(),
base_fee_per_gas: Uint256::one(),
block_hash: ExecutionBlockHash::zero(),
transactions: vec![].into(),
withdrawals: pa.withdrawals.clone().into(),
}),
ForkName::Deneb => ExecutionPayload::Deneb(ExecutionPayloadDeneb {
parent_hash: forkchoice_state.head_block_hash,
fee_recipient: pa.suggested_fee_recipient,
receipts_root: Hash256::repeat_byte(42),
state_root: Hash256::repeat_byte(43),
logs_bloom: vec![0; 256].into(),
prev_randao: pa.prev_randao,
block_number: parent.block_number() + 1,
gas_limit: GAS_LIMIT,
gas_used: GAS_USED,
timestamp: pa.timestamp,
extra_data: "block gen was here".as_bytes().to_vec().into(),
base_fee_per_gas: Uint256::one(),
block_hash: ExecutionBlockHash::zero(),
transactions: vec![].into(),
withdrawals: pa.withdrawals.clone().into(),
data_gas_used: 0,
excess_data_gas: 0,
}),
_ => unreachable!(),
},
};
match execution_payload.fork_name() {
@@ -631,61 +629,22 @@ impl<T: EthSpec> ExecutionBlockGenerator<T> {
pub fn generate_random_blobs<T: EthSpec>(
n_blobs: usize,
kzg: &Kzg,
kzg: &Kzg<T::Kzg>,
) -> Result<(BlobsBundleV1<T>, Transactions<T>), String> {
let mut bundle = BlobsBundleV1::<T>::default();
let mut transactions = vec![];
for blob_index in 0..n_blobs {
// fill a vector with random bytes
let mut blob_bytes = [0u8; BYTES_PER_BLOB];
rand::thread_rng().fill_bytes(&mut blob_bytes);
// Ensure that the blob is canonical by ensuring that
// each field element contained in the blob is < BLS_MODULUS
for i in 0..FIELD_ELEMENTS_PER_BLOB {
blob_bytes[i * BYTES_PER_FIELD_ELEMENT + BYTES_PER_FIELD_ELEMENT - 1] = 0;
}
let random_valid_sidecar = BlobSidecar::<T>::random_valid(&mut thread_rng(), kzg)?;
let blob = Blob::<T>::new(Vec::from(blob_bytes))
.map_err(|e| format!("error constructing random blob: {:?}", e))?;
let BlobSidecar {
blob,
kzg_commitment,
kzg_proof,
..
} = random_valid_sidecar;
let commitment = kzg
.blob_to_kzg_commitment(blob_bytes.into())
.map_err(|e| format!("error computing kzg commitment: {:?}", e))?;
let proof = kzg
.compute_blob_kzg_proof(blob_bytes.into(), commitment)
.map_err(|e| format!("error computing kzg proof: {:?}", e))?;
let versioned_hash = commitment.calculate_versioned_hash();
let blob_transaction = BlobTransaction {
chain_id: Default::default(),
nonce: 0,
max_priority_fee_per_gas: Default::default(),
max_fee_per_gas: Default::default(),
gas: 100000,
to: None,
value: Default::default(),
data: Default::default(),
access_list: Default::default(),
max_fee_per_data_gas: Default::default(),
versioned_hashes: vec![versioned_hash].into(),
};
let bad_signature = EcdsaSignature {
y_parity: false,
r: Uint256::from(0),
s: Uint256::from(0),
};
let signed_blob_transaction = SignedBlobTransaction {
message: blob_transaction,
signature: bad_signature,
};
// calculate transaction bytes
let tx_bytes = [BLOB_TX_TYPE]
.into_iter()
.chain(signed_blob_transaction.as_ssz_bytes().into_iter())
.collect::<Vec<_>>();
let tx = Transaction::<T::MaxBytesPerTransaction>::from(tx_bytes);
let tx = random_valid_tx::<T>()
.map_err(|e| format!("error creating valid tx SSZ bytes: {:?}", e))?;
transactions.push(tx);
bundle
@@ -694,11 +653,11 @@ pub fn generate_random_blobs<T: EthSpec>(
.map_err(|_| format!("blobs are full, blob index: {:?}", blob_index))?;
bundle
.commitments
.push(commitment)
.push(kzg_commitment)
.map_err(|_| format!("blobs are full, blob index: {:?}", blob_index))?;
bundle
.proofs
.push(proof)
.push(kzg_proof)
.map_err(|_| format!("blobs are full, blob index: {:?}", blob_index))?;
}
@@ -709,6 +668,51 @@ fn payload_id_from_u64(n: u64) -> PayloadId {
n.to_le_bytes()
}
pub fn generate_genesis_header<T: EthSpec>(
spec: &ChainSpec,
post_transition_merge: bool,
) -> Option<ExecutionPayloadHeader<T>> {
let genesis_fork = spec.fork_name_at_slot::<T>(spec.genesis_slot);
let genesis_block_hash =
generate_genesis_block(spec.terminal_total_difficulty, DEFAULT_TERMINAL_BLOCK)
.ok()
.map(|block| block.block_hash);
match genesis_fork {
ForkName::Base | ForkName::Altair => None,
ForkName::Merge => {
if post_transition_merge {
let mut header = ExecutionPayloadHeader::Merge(<_>::default());
*header.block_hash_mut() = genesis_block_hash.unwrap_or_default();
Some(header)
} else {
Some(ExecutionPayloadHeader::<T>::Merge(<_>::default()))
}
}
ForkName::Capella => {
let mut header = ExecutionPayloadHeader::Capella(<_>::default());
*header.block_hash_mut() = genesis_block_hash.unwrap_or_default();
Some(header)
}
ForkName::Deneb => {
let mut header = ExecutionPayloadHeader::Capella(<_>::default());
*header.block_hash_mut() = genesis_block_hash.unwrap_or_default();
Some(header)
}
}
}
pub fn generate_genesis_block(
terminal_total_difficulty: Uint256,
terminal_block_number: u64,
) -> Result<PoWBlock, String> {
generate_pow_block(
terminal_total_difficulty,
terminal_block_number,
0,
ExecutionBlockHash::zero(),
)
}
pub fn generate_pow_block(
terminal_total_difficulty: Uint256,
terminal_block_number: u64,

8
beacon_node/execution_layer/src/test_utils/mock_execution_layer.rs
View File

@@ -43,12 +43,12 @@ impl<T: EthSpec> MockExecutionLayer<T> {
executor: TaskExecutor,
terminal_block: u64,
shanghai_time: Option<u64>,
deneb_time: Option<u64>,
cancun_time: Option<u64>,
builder_threshold: Option<u128>,
jwt_key: Option<JwtKey>,
spec: ChainSpec,
builder_url: Option<SensitiveUrl>,
kzg: Option<Kzg>,
kzg: Option<Kzg<T::Kzg>>,
) -> Self {
let handle = executor.handle().unwrap();
@@ -60,7 +60,7 @@ impl<T: EthSpec> MockExecutionLayer<T> {
terminal_block,
spec.terminal_block_hash,
shanghai_time,
deneb_time,
cancun_time,
kzg,
);
@@ -204,7 +204,7 @@ impl<T: EthSpec> MockExecutionLayer<T> {
Some(payload.clone())
);
let status = self.el.notify_new_payload(&payload).await.unwrap();
let status = self.el.notify_new_payload(&payload, None).await.unwrap();
assert_eq!(status, PayloadStatus::Valid);
// Use junk values for slot/head-root to ensure there is no payload supplied.

19
beacon_node/execution_layer/src/test_utils/mod.rs
View File

@@ -25,7 +25,8 @@ use warp::{http::StatusCode, Filter, Rejection};
use crate::EngineCapabilities;
pub use execution_block_generator::{
generate_pow_block, generate_random_blobs, Block, ExecutionBlockGenerator,
generate_genesis_header, generate_pow_block, generate_random_blobs, Block,
ExecutionBlockGenerator,
};
pub use hook::Hook;
pub use mock_builder::{Context as MockBuilderContext, MockBuilder, Operation, TestingBuilder};
@@ -65,7 +66,7 @@ pub struct MockExecutionConfig {
pub terminal_block: u64,
pub terminal_block_hash: ExecutionBlockHash,
pub shanghai_time: Option<u64>,
pub deneb_time: Option<u64>,
pub cancun_time: Option<u64>,
}
impl Default for MockExecutionConfig {
@@ -77,7 +78,7 @@ impl Default for MockExecutionConfig {
terminal_block_hash: ExecutionBlockHash::zero(),
server_config: Config::default(),
shanghai_time: None,
deneb_time: None,
cancun_time: None,
}
}
}
@@ -106,7 +107,7 @@ impl<T: EthSpec> MockServer<T> {
pub fn new_with_config(
handle: &runtime::Handle,
config: MockExecutionConfig,
kzg: Option<Kzg>,
kzg: Option<Kzg<T::Kzg>>,
) -> Self {
let MockExecutionConfig {
jwt_key,
@@ -115,7 +116,7 @@ impl<T: EthSpec> MockServer<T> {
terminal_block_hash,
server_config,
shanghai_time,
deneb_time,
cancun_time,
} = config;
let last_echo_request = Arc::new(RwLock::new(None));
let preloaded_responses = Arc::new(Mutex::new(vec![]));
@@ -124,7 +125,7 @@ impl<T: EthSpec> MockServer<T> {
terminal_block,
terminal_block_hash,
shanghai_time,
deneb_time,
cancun_time,
kzg,
);
@@ -186,8 +187,8 @@ impl<T: EthSpec> MockServer<T> {
terminal_block: u64,
terminal_block_hash: ExecutionBlockHash,
shanghai_time: Option<u64>,
deneb_time: Option<u64>,
kzg: Option<Kzg>,
cancun_time: Option<u64>,
kzg: Option<Kzg<T::Kzg>>,
) -> Self {
Self::new_with_config(
handle,
@@ -198,7 +199,7 @@ impl<T: EthSpec> MockServer<T> {
terminal_block,
terminal_block_hash,
shanghai_time,
deneb_time,
cancun_time,
},
kzg,
)