Skip to content

test(e2e): give test nodes 10 s for DHT requests, in step with ant-client - #277

Merged
Nic-dorman merged 1 commit into
mainfrom
nic/v2-1869-ant-node-e2e-harness-gives-every-dht-request-only-2-s
Nov 5, 2026
Merged

Nic-dorman merged 1 commit into
mainfrom
nic/v2-1869-ant-node-e2e-harness-gives-every-dht-request-only-2-s

Conversation

@Nic-dorman

Copy link
Copy Markdown
Member

tests/e2e/testnet.rs set TEST_CORE_CONNECTION_TIMEOUT_SECS = 2. It came from 857398c, which kept DHT leave waits short during teardown and, in the same commit, made shutdown concurrent.

But saorsa-core uses a node's connection_timeout for two things: the connection timeout, and the request timeout of every DHT operation (network.rs: request_timeout: config.connection_timeout). So the e2e nodes gave each peer 2 s to answer any DHT lookup, against 25 s in production.

ant-client's harness had the same trap at 5 s, and it caused its E2E macOS flake (V2-1852, ant-client #248). On loaded runners, witnessed close-group lookups dropped slow but live peers, and uploads failed with "found 6 peers, need 7". Both harnesses now use 10 s under the same constant name, with the same explanation, and each says to keep the two in step. shutdown() here already stops nodes concurrently, so the longer value doesn't add up across nodes.

Linear issue

Closes V2-1869

Risk tier

  • T0 — docs / tooling / CI / pure UX-output. Repo CI only.
  • T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
  • T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
  • T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Compatibility

  • Wire: none
  • Storage: none
  • API: none. E2E test harness only (tests/e2e/testnet.rs).

Semver impact

  • breaking
  • feature
  • fix

Test evidence

This repo, dev2 (Linux, 4 cores, debug, -D warnings). The e2e target was run with --features test-utils -- --test-threads=1, as CI runs it, both binaries built from 369a2e2:

Timeout Result Time
2 s (main) 120 passed, 0 failed, 3 ignored 988.7 s
10 s (this PR) 120 passed, 0 failed, 3 ignored 987.4 s

The tests that stop nodes partway through (replication, chunk) are no slower.

Choosing 10 s (ant-client's flaky test, 14 nodes, CPU capped with systemd-run -p CPUQuota):

  • At 1.5 cores and at 1 core, 5 s failed every run, and 10, 15, 20 and 25 s passed every run (3/3, or 2/2 for the higher values at 1 core).
  • 10 s was the lowest value tried that held.
  • Run time didn't depend on the value.

New dependency

none

ADR

n/a

Mitigation / rollback

Revert. If a test that waits out a dead peer turns out too slow, lower the constant, and lower ant-client's to match.

🤖 Generated with Claude Code

…ient

TEST_CORE_CONNECTION_TIMEOUT_SECS was 2, set in 857398c to keep DHT
leave waits short during teardown. saorsa-core also uses a node's
connection_timeout as the request timeout for every DHT operation
(network.rs: request_timeout: config.connection_timeout), so the e2e
nodes gave each peer 2 s to answer a lookup, against 25 s in production.

ant-client's harness had the same trap at 5 s, and it caused its E2E
macOS flake: witnessed close-group lookups dropped slow but live peers
on loaded runners ("found 6 peers, need 7", V2-1852). On dev2, with
ant-client's flaky test capped to 1 or 1.5 CPU cores, 5 s failed every
run and 10 s passed every run.

Both harnesses now use 10 s under the same constant name, and each
says to keep the two in step. shutdown() already stops nodes
concurrently, so the longer value doesn't add up across nodes.

Closes V2-1869

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@MaidSafe-Hermes MaidSafe-Hermes left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

APPROVE — Reviewed WithAutonomi/ant-node#277 at 6c1df6e3882a029e8c996a29eef2690385b0a839 (base 369a2e28bcbfbaa0127034be0f8f11b10fbfb00b). The change is limited to the E2E test harness timeout; it does not alter production configuration. The timeout increase is reasonable for slow but live peers, and shutdown waits run concurrently. No material correctness or compatibility defect found.

Coverage: 2/2 seats reviewed — sol-1 (gpt-6.1-sol xhigh) and glm-1 (z-ai/glm-5.3); both seats agree (two model families). E2E results cited in the PR description are author-reported, not independently reproduced.

The failing Security Audit for inherited RUSTSEC-2026-0387 is a merge gate, not a blocker to this approval.

@Nic-dorman
Nic-dorman merged commit b0df77a into main Nov 5, 2026
14 of 15 checks passed
@Nic-dorman
Nic-dorman deleted the nic/v2-1869-ant-node-e2e-harness-gives-every-dht-request-only-2-s branch November 5, 2026 12:11
grumbach added a commit to grumbach/ant-node that referenced this pull request Nov 6, 2026
Brings in main up to 360d472: OS-assigned ports for e2e test nodes (WithAutonomi#281)
and devnet nodes (WithAutonomi#282), release-key and signature checks (WithAutonomi#276), the
third-party notices and audit-regression CI gates (WithAutonomi#274, WithAutonomi#275), the 10 s
DHT request timeout for test nodes (WithAutonomi#277), and the 0.24.0 release (WithAutonomi#283).

One conflict, in tests/e2e/harness.rs: main removed the network bring-up
retry on a fresh port range together with NETWORK_START_ATTEMPTS and its
`warn` import, and this branch had added two helpers next to them
(POSSESSION_PENALTY_TRUST_RATIO and parseable_single_node_proof). The
resolution keeps both helpers and takes main's removal.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants