Repository navigation
Open
Labels
enhancementNew feature or requestNew feature or request
Description
Priority
extreme
Components
- app/nullius (backend)
- app/web (routes)
- templates and static files
- docs and the site
- catalogue (benchmarks, agents, capabilities)
- the standard (ports, kits, relay, runs)
- CI, workflows and skills
Today
τ³-bench's banking_knowledge offers 11 of the upstream's search methods. Nine are recorded as not offered in upstream-settings.json, for two reasons, both checked against the upstream at v1.0.1:
- The four reranker methods (
bm25_reranker,bm25_reranker_grep,openai_embeddings_reranker,openai_embeddings_reranker_grep). The pointwise reranker asks a chat model through the OpenAI client (knowledge/postprocessors/pointwise_llm_reranker.py:59, modelgpt-5.2), which readsOPENAI_BASE_URL. In a run that variable is the embeddings route, whose one model istext-embedding-3-large, or it is not set at all. - The five qwen methods (
qwen_embeddings,..._grep,..._reranker,..._reranker_grep,alltools-qwen). Their embedder calls OpenRouter at an address written in the code (knowledge/embedders/openrouter_embedder.py:118,base_url="https://openrouter.ai/api/v1") withOPENROUTER_API_KEY, past the relay, which never hands a driver a key.
Wanted
The nine methods offered, with every call still through the run's relay.
- A relay route that sends a request by what it asks for: an embeddings request as the embedding model, a chat or responses request as the reranker's model. The driver's one
OPENAI_BASE_URLthen serves both, and a method without embeddings (bm25_reranker) gets the route for its reranker alone. - For qwen, a route to OpenRouter on the run's relay, and the embedder pointed at it. The address is written in the upstream's code, so this needs either a change upstream (reading it from the environment) or one line of ours setting it in the driver; the second is code beyond glue and is the owner's yes to give.
Acceptance
- A relay route can pin one model for embeddings requests and another for chat requests, each recorded as the model sent.
- The four reranker methods are values of
retrieval, each with what it needs said (needs). - The owner has said how the qwen embedder is pointed at the relay.
- The five qwen methods are values of
retrieval, or stay recorded as not offered with the reason. - A run of each offered method on a few tasks makes every model call through the relay (its record shows the reranker's and the embedder's calls).
Dependencies and related
none
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request
Activity