Skip to content

test(history): accept page-cache tears and ESTALE opens under churn, keep the image off tmpfs - #174

Merged
mattmattox merged 4 commits into
mainfrom
test/history-stale-open
Nov 16, 2026
Merged

mattmattox merged 4 commits into
mainfrom
test/history-stale-open

Conversation

@mattmattox

Copy link
Copy Markdown
Contributor

Teaches the history checker the documented outcomes it was flagging, so the remaining violations are real.

  • Page-cache tears: a torn read is allowed when a write on the reader's own node overlapped it and every piece is a value the read may see, including one boundary word whose bytes mix both neighbors. Through one mount, buffered I/O tore inside a 4 KB sector 352 times in 6.4M reads; O_DIRECT, which bypasses the page cache, never did.
  • ESTALE on open: allowed only when the name lost its file during the call and once more within the cache window before it. Counted separately in the summary, as on NFS.
  • GCFS_IMAGE_DIR: keeps the image off tmpfs. tmpfs tears O_DIRECT inside sectors by itself (95k–158k tears in 10 s, against 0 on ext4 or ZFS).

History checker, 3 nodes for 3 minutes, 1 GB image:

Image on Calls Accepted ESTALE opens Violations
ZFS 68,094 5 1
tmpfs 182,808 2 1

The ZFS violation is the known cross-node 4 KB tear: a read on one node during a write on another. It stays a violation; the planned cluster read lock fixes it. The tmpfs one comes from tmpfs tearing device writes.

🤖 Generated with Claude Code

mattmattox and others added 4 commits November 16, 2026 15:59
…le twice

An open that reaches a removed file gets ESTALE, and the kernel retries
it once; under same-name churn the retry can find a removed file too, as
on NFS. The checker now records ESTALE from open as its own result,
counts it in the summary, and allows it only when the name lost its file
at least twice during the call. OPERATIONS.md documents the case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…node

A buffered read that overlaps a write on the same node can split at any
8-byte boundary: the kernel copies an up-to-date page to the reader
while the write copies into it. Through one GoClusterFS mount, buffered
I/O tore inside a 4 KB sector 352 times in 6.4M reads, and O_DIRECT,
which bypasses the page cache, never did. The checker now allows such
tears at 8-byte boundaries when a write on the reader's node overlapped,
and GCFS_IMAGE_DIR lets it keep the image off tmpfs, which tears writes
inside a sector by itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ntry from before the open

A checker run with the image on ZFS still flagged 8 torn reads and 2
ESTALE opens. Each torn read had one 8-byte word at a boundary whose
bytes mixed the values on either side: the kernel's copy can split
inside a word. Each ESTALE followed a removal shortly before the open
and another during it: the first left the kernel's cached entry stale,
the second removed the file the retry found.

The checker now ignores such a mixed boundary word, and allows ESTALE
on open when the name lost its file during the call and once more
within the cache window before it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@mattmattox
mattmattox merged commit 48d1e51 into main Nov 16, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant