You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
A machine requirement cannot say how much GPU memory it needs, so Stella starts on a card too small #23
A benchmark's machine requirement (host in its standard.json) is checked by programs on PATH only. LongMemEval's Stella retriever needs a CUDA GPU with more memory than an 11 GB card has, and says so only in prose: "Stella needs more than 11 GB of GPU memory." On a machine with an 11 GB card the builder shows the requirement as met (nvidia-smi is there), Run starts, and the unit fails later inside run_retrieval.py when its fixed batch of 64 runs out of memory.
Wanted
A requirement may say the least memory its GPU must have (gpu_memory_mb). The builder's check on this machine and the run itself read the largest GPU's memory from nvidia-smi and refuse a card that is too small, saying what is needed and what is there. Stella's two retriever values ask for it.
Acceptance
host.<name>.gpu_memory_mb is validated as a positive whole number of MiB.
A run whose options need such a requirement is refused when the largest GPU has less, naming the need and what the machine has.
The builder's check for this machine shows the requirement as missing with the same words.
A machine whose GPU cannot be asked is reported by its missing program, not as too small.
LongMemEval's flat-stella-session and flat-stella-turn need a GPU of at least 12000 MiB; Contriever's values are unchanged.
docs/standard.md says the field.
Dependencies and related
A host's GPU memory is not probed before Run; the run checks it as it starts on the host.
Activity