The PyPI install puts the tool's site-packages on a driver's PYTHONPATH, which breaks LongMemEval #11

Priority

extreme

Components

• app/nullius (backend)
• app/web (routes)
• templates and static files
• docs and the site
• catalogue (benchmarks, agents, capabilities)
• the standard (ports, kits, relay, runs)
• CI, workflows and skills

Evidence

Installed with uv tool install nullius-eval==0.1.0rc2, a LongMemEval run stops before its first unit:

nullius.standard.jsonl.Failed: units: the program ended without replying (SyntaxError: invalid syntax)

The driver's log (<job>/.work/driver-*.log), home shortened to ~:

  File "~/.local/share/nullius/standard/upstreams/longmemeval/9e0b455f4ef0e2ab8f2e582289761153549043fc/src/evaluation/evaluate_qa.py", line 6, in <module>
  File "~/.local/share/nullius/standard/drivers/longmemeval/8f6ca5cafb32/.venv/lib/python3.9/site-packages/openai/__init__.py", line 8, in <module>
  ...
  File "~/.local/share/uv/tools/nullius-eval/lib/python3.12/site-packages/click/utils.py", line 298
    match message:
          ^
SyntaxError: invalid syntax

The driver runs in its own Python 3.9 venv, but it imports click from the tool's Python 3.12 environment.

Cause: run.py:75 sets APP to the folder that holds the nullius package ("app/: what a kit's driver imports"), and run.py:450 passes it to the driver as PYTHONPATH (prove.py:78 does the same). From a source checkout that folder is app/, which holds only Nullius. From the wheel it is the tool's whole site-packages, so every package there comes before the driver venv's own on sys.path.

Screenshot

No response

Reproduction

1. uv tool install nullius-eval==0.1.0rc2
2. nullius standard run longmemeval keyword-memory --dataset longmemeval_oracle --limit 1 --provider custom --model <model> --judge-model <model>
3. It fails at units; the driver log under the job's .work/ shows the traceback above. No model call is made.

Running the same command from a v0.1.0rc2 checkout (PYTHONPATH=app with the tool's Python) works.

Expected

A driver imports its own venv's packages and only the nullius package from the tool, so LongMemEval runs from the PyPI install as it does from a checkout.

Acceptance

• From uv tool install nullius-eval, nullius standard run longmemeval keyword-memory --limit 1 gets past units.
• A driver's PYTHONPATH gives it the nullius package and no other package of the tool's environment, both from the wheel and from a checkout (a test).

Dependencies and related

none