Skip to content

A machine requirement cannot say how much GPU memory it needs, so Stella starts on a card too small #23

Description

@aitianshi

Priority

extreme

Components

  • app/nullius (backend)
  • app/web (routes)
  • templates and static files
  • docs and the site
  • catalogue (benchmarks, agents, capabilities)
  • the standard (ports, kits, relay, runs)
  • CI, workflows and skills

Today

A benchmark's machine requirement (host in its standard.json) is checked by programs on PATH only. LongMemEval's Stella retriever needs a CUDA GPU with more memory than an 11 GB card has, and says so only in prose: "Stella needs more than 11 GB of GPU memory." On a machine with an 11 GB card the builder shows the requirement as met (nvidia-smi is there), Run starts, and the unit fails later inside run_retrieval.py when its fixed batch of 64 runs out of memory.

Wanted

A requirement may say the least memory its GPU must have (gpu_memory_mb). The builder's check on this machine and the run itself read the largest GPU's memory from nvidia-smi and refuse a card that is too small, saying what is needed and what is there. Stella's two retriever values ask for it.

Acceptance

  • host.<name>.gpu_memory_mb is validated as a positive whole number of MiB.
  • A run whose options need such a requirement is refused when the largest GPU has less, naming the need and what the machine has.
  • The builder's check for this machine shows the requirement as missing with the same words.
  • A machine whose GPU cannot be asked is reported by its missing program, not as too small.
  • LongMemEval's flat-stella-session and flat-stella-turn need a GPU of at least 12000 MiB; Contriever's values are unchanged.
  • docs/standard.md says the field.

Dependencies and related

A host's GPU memory is not probed before Run; the run checks it as it starts on the host.

Activity

  1. self-assigned this
    on Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions