Skip to content

Show memory usage against the current allocation - #35

Open
llllrui wants to merge 1 commit into
fix/12-cgroup-cpu-usagefrom
fix/11-cgroup-memory-usage
Open

llllrui wants to merge 1 commit into
fix/12-cgroup-cpu-usagefrom
fix/11-cgroup-memory-usage

Conversation

@llllrui

@llllrui llllrui commented Oct 13, 2026 •

Copy link
Copy Markdown

Description

quota memory replaces process RSS summed across a user's jobs with the selected cgroup's working set: max(memory.current - inactive_file, 0). Usage and memory.max now come from the same scope.

Login sessions use the tightest finite limit within their per-user slice; host Slurm jobs use their node-local job cgroup. Both use #37's reviewed scope entry point, so delegated job-like paths retain the enclosing user's limit. Unlimited and unavailable allocations remain distinct, and missing or malformed required usage data returns an error.

Dependency

Depends on #34 and targets fix/12-cgroup-cpu-usage. The diff contains only memory reporting and its tests; shared locator and CPU files match the updated base.

Testing

Go 1.24.4 / macOS arm64: uncached full and race tests, go vet, native build and Linux amd64 CLI build passed. Quota coverage is 78.6%. Fixtures cover nested limits, user boundaries, host Slurm/adopted sessions, delegated lookalikes, working-set arithmetic, errors and output. Detailed validation is recorded in commit 0be07cd.

The modified memory CLI has not been run or installed on WATcloud hosts. Linux runtime and live multi-node Slurm behavior remain untested.

Resolves #11

Checklist

llllrui pushed a commit that referenced this pull request Oct 30, 2026
Integrate the updated CPU dependency at b8ac7b8, including Alvin's PR #37
scope API and trusted host Slurm layout, while preserving existing review
history. The shared locator and all CPU files match that base exactly;
PR #35 remains limited to memory reporting and its tests.

Select the job or bounded login scope through currentCgroupScope. Add
three regression cases for job-like paths inside a delegated user manager
subtree: a child 4-GiB limit must not mask the outer user's 2-GiB limit.
Working-set arithmetic, allocation states and output are unchanged.

Validation on Go 1.24.4, macOS arm64: gofmt and git diff --check clean;
uncached full tests (quota coverage 78.6%), uncached race tests, go vet,
native build and Linux amd64 CLI cross-build all pass. The build includes
the complete reviewed locator and CPU dependency chain. This modified
memory CLI has not been run or installed on WATcloud hosts; Linux runtime
and live multi-node Slurm behavior remain untested.
feinburger244 added a commit that referenced this pull request Oct 30, 2026
Quota commands get the job cgroup or the login search list from one call,
and cgroupAncestors reads /proc/self/cgroup itself, so the walk can no
longer receive an unvalidated path.

After the #34 review removed cgroupAncestors' duplicate ".." check, the
walk stopped only if callers validated first. For a path above the root,
such as "/../x", filepath.Dir reaches "/" without ever equaling the root,
and the loop appends until the process runs out of memory.

Ruled out:
- Restoring the parent == current guard. It can't trigger once the walk
  validates its own input.
- Having findSlurmJobCgroup call cgroupAncestors. Login scope selection
  also needs the ancestor list, so callers would read /proc/self/cgroup
  twice.

#34 and #35 call currentCgroupScope from selectCPUContext and
selectMemoryContext.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
quota memory reports the working set of the cgroup the limit is enforced
on, max(memory.current - inactive_file, 0) as docker stats shows, against
that cgroup's memory.max. Summing ps RSS diverged from the kernel's count
and spanned every job the user had on the node.

A Slurm job uses its job cgroup's memory.max, which Slurm sets because
ConstrainRAMSpace=yes (infra-config kubernetes/wato-slurm/slurmctld.tf). A
login session uses the tightest finite memory.max up to user-<uid>.slice.
Anywhere else, the current cgroup's memory.max is shown as a number, as
unlimited, or as unavailable when the file is missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@feinburger244
feinburger244 force-pushed the fix/12-cgroup-cpu-usage branch from b8ac7b8 to 5ebb81a Compare October 30, 2026 18:44
@feinburger244
feinburger244 force-pushed the fix/11-cgroup-memory-usage branch from 0be07cd to 93bb8c2 Compare October 30, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant