gnosis/lighthouse
1
0
Fork 0
mirror of https://github.com/sigp/lighthouse.git synced 2026-03-09 03:31:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

Prevent port re-use in HTTP API tests (#4745)

Browse Source 
## Issue Addressed

CI is plagued by `AddrAlreadyInUse` failures, which are caused by race conditions in allocating free ports.

This PR removes all usages of the `unused_port` crate for Lighthouse's HTTP API, in favour of passing `:0` as the listen address. As a result, the listen address isn't known ahead of time and must be read from the listening socket after it binds. This requires tying some self-referential knots, which is a little disruptive, but hopefully doesn't clash too much with Deneb 🤞

There are still a few usages of `unused_tcp4_port` left in cases where we start external processes, like the `watch` Postgres DB, Anvil, Geth, Nethermind, etc. Removing these usages is non-trivial because it's hard to read the port back from an external process after starting it with `--port 0`. We might be able to do something on Linux where we read from `/proc/`, but I'll leave that for future work.
This commit is contained in:
Michael Sproul
2023-09-20 01:19:03 +00:00
parent d386a07b0c
commit 4b6cb3db2c
12 changed files with 172 additions and 184 deletions
Download Patch File Download Diff File Expand all files Collapse all files

61
beacon_node/execution_layer/src/lib.rs
View File

@@ -5,6 +5,7 @@
//! deposit-contract functionality that the `beacon_node/eth1` crate already provides.
use crate::payload_cache::PayloadCache;
use arc_swap::ArcSwapOption;
use auth::{strip_prefix, Auth, JwtKey};
use builder_client::BuilderHttpClient;
pub use engine_api::EngineCapabilities;
@@ -209,7 +210,7 @@ pub enum FailedCondition {
struct Inner<E: EthSpec> {
engine: Arc<Engine>,
builder: Option<BuilderHttpClient>,
builder: ArcSwapOption<BuilderHttpClient>,
execution_engine_forkchoice_lock: Mutex<()>,
suggested_fee_recipient: Option<Address>,
proposer_preparation_data: Mutex<HashMap<u64, ProposerPreparationDataEntry>>,
@@ -324,25 +325,9 @@ impl<T: EthSpec> ExecutionLayer<T> {
Engine::new(api, executor.clone(), &log)
};
let builder = builder_url
.map(|url| {
let builder_client = BuilderHttpClient::new(url.clone(), builder_user_agent)
.map_err(Error::Builder)?;
info!(
log,
"Using external block builder";
"builder_url" => ?url,
"builder_profit_threshold" => builder_profit_threshold,
"local_user_agent" => builder_client.get_user_agent(),
);
Ok::<_, Error>(builder_client)
})
.transpose()?;
let inner = Inner {
engine: Arc::new(engine),
builder,
builder: ArcSwapOption::empty(),
execution_engine_forkchoice_lock: <_>::default(),
suggested_fee_recipient,
proposer_preparation_data: Mutex::new(HashMap::new()),
@@ -356,19 +341,45 @@ impl<T: EthSpec> ExecutionLayer<T> {
last_new_payload_errored: RwLock::new(false),
};
Ok(Self {
let el = Self {
inner: Arc::new(inner),
})
}
}
};
if let Some(builder_url) = builder_url {
el.set_builder_url(builder_url, builder_user_agent)?;
}
Ok(el)
}
impl<T: EthSpec> ExecutionLayer<T> {
fn engine(&self) -> &Arc<Engine> {
&self.inner.engine
}
pub fn builder(&self) -> &Option<BuilderHttpClient> {
&self.inner.builder
pub fn builder(&self) -> Option<Arc<BuilderHttpClient>> {
self.inner.builder.load_full()
}
/// Set the builder URL after initialization.
///
/// This is useful for breaking circular dependencies between mock ELs and mock builders in
/// tests.
pub fn set_builder_url(
&self,
builder_url: SensitiveUrl,
builder_user_agent: Option<String>,
) -> Result<(), Error> {
let builder_client = BuilderHttpClient::new(builder_url.clone(), builder_user_agent)
.map_err(Error::Builder)?;
info!(
self.log(),
"Using external block builder";
"builder_url" => ?builder_url,
"builder_profit_threshold" => self.inner.builder_profit_threshold.as_u128(),
"local_user_agent" => builder_client.get_user_agent(),
);
self.inner.builder.swap(Some(Arc::new(builder_client)));
Ok(())
}
/// Cache a full payload, keyed on the `tree_hash_root` of the payload