You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Follow-up items from #133 (simplify inference routing) that were deferred from the initial implementation in #146.
Items
1. Default route: managed NVIDIA inference when no local model is present
Currently, inference.local requires explicit configuration via nemoclaw cluster inference set --provider <name> --model <id>. When no cluster inference is configured, requests to inference.local return a 503 with a hint to run the set command.
The original issue envisioned that inference.local would default to managed NVIDIA inference when no local model is deployed (e.g., on Brev/CPU). This would improve the out-of-box experience — a freshly deployed cluster would route inference to NVIDIA endpoints automatically if an nvidia provider exists.
2. Drop a skill/instructions in the sandbox telling agents about inference.local
Agents running inside sandboxes currently have no way to discover inference.local unless told externally. We should inject instructions (e.g., a file or environment hint) into sandboxes so agents know they can use https://inference.local/v1/chat/completions for inference without credentials.
Note: inference.local only works over HTTPS (the proxy intercepts CONNECT requests).
Activity
pimlock commentedon Mar 6, 2026
Another thing - some kind of validation of the provider/inference when it's being configured.