Prevent port re-use in HTTP API tests (#4745)

## Issue Addressed

CI is plagued by `AddrAlreadyInUse` failures, which are caused by race conditions in allocating free ports.

This PR removes all usages of the `unused_port` crate for Lighthouse's HTTP API, in favour of passing `:0` as the listen address. As a result, the listen address isn't known ahead of time and must be read from the listening socket after it binds. This requires tying some self-referential knots, which is a little disruptive, but hopefully doesn't clash too much with Deneb 🤞

There are still a few usages of `unused_tcp4_port` left in cases where we start external processes, like the `watch` Postgres DB, Anvil, Geth, Nethermind, etc. Removing these usages is non-trivial because it's hard to read the port back from an external process after starting it with `--port 0`. We might be able to do something on Linux where we read from `/proc/`, but I'll leave that for future work.
This commit is contained in:
Michael Sproul
2023-09-20 01:19:03 +00:00
parent d386a07b0c
commit 4b6cb3db2c
12 changed files with 172 additions and 184 deletions

View File

@@ -3635,12 +3635,13 @@ pub fn serve<T: BeaconChainTypes>(
// send the response back to our original HTTP request
// task via a channel.
let builder_future = async move {
let builder = chain
let arc_builder = chain
.execution_layer
.as_ref()
.ok_or(BeaconChainError::ExecutionLayerMissing)
.map_err(warp_utils::reject::beacon_chain_error)?
.builder()
.builder();
let builder = arc_builder
.as_ref()
.ok_or(BeaconChainError::BuilderMissing)
.map_err(warp_utils::reject::beacon_chain_error)?;

View File

@@ -129,17 +129,9 @@ pub async fn create_api_server<T: BeaconChainTypes>(
test_runtime: &TestRuntime,
log: Logger,
) -> ApiServer<T::EthSpec, impl Future<Output = ()>> {
// Get a random unused port.
let port = unused_port::unused_tcp4_port().unwrap();
create_api_server_on_port(chain, test_runtime, log, port).await
}
// Use port 0 to allocate a new unused port.
let port = 0;
pub async fn create_api_server_on_port<T: BeaconChainTypes>(
chain: Arc<BeaconChain<T>>,
test_runtime: &TestRuntime,
log: Logger,
port: u16,
) -> ApiServer<T::EthSpec, impl Future<Output = ()>> {
let (network_senders, network_receivers) = NetworkSenders::new();
// Default metadata