Skip to content

The cap.memory/v1 kit's m2 check fails a memory that refuses to store credentials #13

Closed

Description

@ttachon

Priority

extreme

Components

  • app/nullius (backend)
  • app/web (routes)
  • templates and static files
  • docs and the site
  • catalogue (benchmarks, agents, capabilities)
  • the standard (ports, kits, relay, runs)
  • CI, workflows and skills

Evidence

The cap-memory-v1 kit's m2 check stores a password and asks for it (standard.json:87, driver.py:9):

"task": "Store six sessions, including the cabin Wi-Fi password tern-river-9. Ask for that password.",
"passes": "The recalled memory contains tern-river-9, ignoring case."

A memory capability that never keeps a password, key, token or other credential, by design, passes m1 and fails m2: its recall is empty. conform records proven only on a full pass, so such a capability is never matched to LongMemEval or LoCoMo unless its capability.json is edited by hand or the kit is overridden.

Screenshot

No response

Reproduction

  1. Conform a memory capability that drops credentials against cap.memory/v1.
  2. m1 passes; m2 fails with an empty recall.
  3. nullius standard fit longmemeval does not list it.

Expected

The kit checks recall of an exact token without asking a memory to keep a secret, so a memory that refuses credentials can be proven.

Acceptance

  • m2's fact is an exact token that is not a credential (for example "The boat at the cabin is named tern-river-9.").
  • A capability that refuses to store credentials passes the kit (a test).

Dependencies and related

none

Activity

  1. added a commit that references this issue on Nov 10, 2026
    0425d50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions