Skip to content

fix(test): kill the exact mount process in crash tests and check it is gone - #73

Merged
mattmattox merged 4 commits into
mainfrom
claude/test-real-crash-kills
Nov 12, 2026
Merged

mattmattox merged 4 commits into
mainfrom
claude/test-real-crash-kills

Conversation

@mattmattox

@mattmattox mattmattox commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

The VM "Restart Keeps Peer Journal" test has never crashed node A. On the Alpine VMs the gcompat loader runs the glibc binary, so the process name and argv[0] belong to the loader (ld-musl-x86_64.). pkill, pidof and /proc/*/exe never match it. The test's umount -l then stopped A cleanly. The clean-unmount wait used pidof too, so it never waited.

Every crash path in the suites:

  • VM: mount_pids matches the exact path setup starts the binary with. The restart test is the only VM test that kills the mount process; it was the only one silently not crashing. kill_vm now waits until QEMU is gone.
  • Docker: kill_node waits until the container stops; three tests that called docker kill directly use it.
  • Go: MountedFS.Kill waits for the process to exit; KillNode's fixed 500 ms sleep is gone.

Each fails the test if the process is still running 10 s after SIGKILL.

Results

🤖 Generated with Claude Code

…s gone

The VM restart test ran `pkill -9 goclusterfs-mount`, which never
matched: the kernel cuts process names to 15 characters
("goclusterfs-mou"). The test's lazy unmount then stopped node A
cleanly, so it tested a clean shutdown instead of a crash, and failed at
random when the remount ran before the old process had exited.

The VM tests now find goclusterfs-mount by its executable path and kill
those PIDs, and every crash path fails unless the process is gone within
10 s: the VM restart test, the VM harness's QEMU kill, Docker's
kill_node (now used by all Docker crash tests), and the Go Kill helper,
which now waits for the mount process to exit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mattmattox and others added 3 commits November 12, 2026 05:51
Matching /proc/PID/exe found nothing on the Alpine VMs, so the restart
test had no process to kill, and the clean-unmount wait returned while
the mount was still running, which made fsck -repair refuse. Match the
path setup_goclusterfs starts the mount with, the start of
/proc/PID/cmdline, instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…loader runs it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmattox
mattmattox merged commit af976ec into main Nov 12, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant