Repository navigation
Description
Found by code review and reproduced in a lab; never seen in production.
Summary
NewXRayCore runs xray version with exec.Command(...).Run(): no context, no timeout, no WaitDelay (backend/xray/core.go:94-99, called at :63, at dev 9ccd2638c7fe). If that process never exits, Start never returns. Because StartBackend holds the controller's c.mu for the whole build (controller/controller.go:281-283), and every REST request takes c.mu in middleware (controller/rest/middleware.go:20 ApiKey(), :70 NewRequest()), GetBaseInfo, GetSystemStats and Stop hang too, for as long as the probe hangs.
Reproduction (real node, stand-in binary)
Real node built from dev 9ccd2638c7fe, SERVICE_PROTOCOL=rest, on macOS arm64. XRAY_EXECUTABLE_PATH points to a shell script that stands in for a hung binary: on version it runs exec sleep 600; anything else goes to the real Xray 26.3.27. Then POST /start, and 2 s later GET /info, GET /stats/system and PUT /stop:
| call | result on dev 9ccd2638 |
|---|---|
Start |
no answer within the client's 60 s |
GetBaseInfo, GetSystemStats, Stop sent during Start |
no answer within 45 s |
| the same three, sent after the Start client gave up | no answer within 5 s each |
SIGTERM to the node |
the process did not exit; about 60 s later I killed the stand-in's sleep and the node exited within 0.1 s |
With the real Xray 26.3.27 as the executable, the same node starts in about 2 s (started=true, core_version=26.3.27).
Impact
- While the probe hangs, the node answers nothing. The panel's Start call goes through node_bridge_py with the node's
default_timeout(10 s unless an admin changed it; panel devf92acaf51cecapp/operation/node.py:427-439sets no timeout for Xray, bridgec5632b65a10arest.py:174falls back todefault_timeout), so the panel sees a timeout after 10 s; the node itself stays stuck after that. Found by reading the panel and bridge, not run against a real panel. - The process does not stop on
SIGTERMwhile the probe hangs, because shutdown callsDisconnect, which also needsc.mu. - When the probe process exits,
Startreturns and the lock is released, so no manual restart is needed (found by reading the code; in the lab the pendingSIGTERMmade the node exit at that moment, so recovery on dev was not observed).
Only a binary that does not exit on version (or a wrapper that leaves a child holding stdout) triggers this. The real xray version (26.3.27) took 0.01 s in five runs here.
Affected versions
dev 9ccd2638c7fe and main v0.5.7 739eab94a934 (the probe code is the same on both).
Proposed fix
Run the probe under exec.CommandContext with a 30 s timeout (the same bound the MTProto backend uses for telemt --version, telemtCommandTimeout at backend/mtproto/process.go:29 on dev 9ccd2638) and set cmd.WaitDelay. PR #132.
Duplicate search
On 2026-11-04 a search of PasarGuard/node issues and PRs for version timeout and health restart found #50, #21, #117, #116 and #113, which are about restarts and start lifetimes, not this probe. The search was not repeated for this edit.
Activity
[-]xray: the version probe has no timeout, so a hung xray binary blocks core creation until it exits[/-][+]xray: the version probe has no timeout, so a hung xray binary blocks Start and every other API call[/+]