SWE-bench Verified: agents start outside the instance's conda environment and cannot import the project #24

Priority

extreme

Components

• app/nullius (backend)
• app/web (routes)
• templates and static files
• docs and the site
• catalogue (benchmarks, agents, capabilities)
• the standard (ports, kits, relay, runs)
• CI, workflows and skills

Evidence

On the Nullius standard an agent works in the SWE-bench instance's own container, started there with the image's PATH. The image's PATH finds the base conda environment; the project and its dependencies are in the testbed environment, which the image's own /root/.bashrc and the harness's eval script activate first. In an instance's image:

plain:    /opt/miniconda3/bin/python  ImportError: SymPy now depends on mpmath as an external library.
BASH_ENV=/root/.bashrc: /opt/miniconda3/envs/testbed/bin/python  /testbed/sympy/__init__.py  3.9.20

In a run of mini-swe-agent on sympy__sympy-24213 (docker backend), the agent's log shows it:

ModuleNotFoundError: No module named 'sympy'
cd /testbed && pip show sympy 2>&1 | head; which python; ...
/opt/miniconda3/bin/python
ModuleNotFoundError: No module named 'mpmath'
...
source /opt/miniconda3/etc/profile.d/conda.sh && conda activate testbed && ...

The agent spends its steps finding the environment the benchmark's own agents are given (mini-swe-agent's swebench.yaml sets BASH_ENV: /root/.bashrc for that reason). An agent that does not find it cannot run the project's code or tests at all. Every agent on SWE-bench Verified is affected the same way.

Screenshot

No response

Reproduction

1. nullius standard run swe-bench-verified mini-swe-agent --model <any> --backend docker --limit 1.
2. Read the trial's agent/agent.log: the first python the agent runs is /opt/miniconda3/bin/python and cannot import the project.

Expected

The agent starts in the instance's container with the instance's conda environment activated, as the image and the harness activate it.

Acceptance

• A unit's setup may carry env, the variables the task's own harness works with, given to the agent started in the unit's container; PATH goes before the image's own.
• A unit's env is validated: plain names, string values, absolute PATH folders, never a model endpoint or credential; what the run gives the agent wins over it.
• SWE-bench Verified's driver gives what conda activate testbed sets.
• An agent's first python in an instance is the testbed environment's.
• docs/standard.md says the field.

Dependencies and related

none