A coding agent for firmware that proves its fixes, on the fewest tokens.
Website ·
Proof ·
Run it ·
Evaluation guide ·
Releases
For judges: claims → code → proof · Bifrost integration · Testing · Pitch deck · Website
BoardLoop is a coding agent for embedded firmware that gets the job done on the fewest tokens. It writes and fixes ESP32/Arduino code, proves the fix on a compiler and a simulated circuit, keeps working when an AI provider goes down, never spends past a per-user budget, keeps confidential code on a local model, and asks before every risky action (writing files, flashing a board).
Built for the Geek Room Global AI Hackathon, theme "Build a Coding Agent That Gets the Job Done on the Fewest Tokens", on Bifrost by Maxim AI as the LLM gateway.
Coding agents burn tokens: every fix round re-sends the whole file, the whole compiler log and the whole chat history, and the model is asked even when the compiler already says what's wrong. They stop when one provider has an outage, they have no spending limit per person, they send private code to the cloud, and many write files or run tools without asking.
| Theme requirement | What BoardLoop does | Where |
|---|---|---|
| Fewest tokens | 1. Deterministic fixes first: errors the compiler fully explains (e.g. a missing ;) are fixed by code, 0 tokens. 2. Fix cache, "reason once": a fix that led to a verified build is stored under an error fingerprint and replayed next time with 0 model calls. 3. Compact prompts: only the first compiler error (later ones are usually caused by it) and ±4 lines around it; for behaviour bugs only the decision lines (comparisons). No chat history. |
coder.py |
| Correctness | Nothing counts as done until mcu_verify passes: real compile + behaviour tests on a simulated circuit (sensors, pins, serial, OLED). Fix budgets (3 compile, 5 behaviour) stop endless loops. |
loop.py |
| Survives provider outages | Calls go through Bifrost with a model chain (fallbacks), so Bifrost switches provider server-side. If the gateway itself is down, the client walks the chain and finally calls local Ollama directly. |
gateway.py |
| Per-user spending limits | Before every call the worst-case cost is checked against the user's budget; after it, the real cost from the provider's usage is added to a per-user ledger. A Bifrost virtual key (--virtual-key) enforces the same budget on the gateway. Over budget = the agent stops, it doesn't crash. |
gateway.py Budget |
| Confidential repos on local models | --confidential (or a CONFIDENTIAL file in the project) restricts the chain to ollama/*. If no local model is configured the agent refuses instead of falling back to the cloud. |
Gateway.chain() |
| Approval before risky operations | Every file write shows a diff and waits for y. Flashing a board asks first. With no terminal attached it answers no. Shell commands are fixed argv templates with timeouts, never a shell. |
coder.ask, procs.py |
flowchart LR
V[mcu_verify<br/>compile + simulated tests] -->|verified| D[done · report tokens/$]
V -->|fix needed| R{rule fix?}
R -->|yes, 0 tokens| W
R -->|no| C{fix cache hit?}
C -->|yes, 0 tokens| W
C -->|no| P[compact prompt<br/>1 error + few lines]
P --> B[Bifrost gateway<br/>budget · fallbacks · virtual key]
B --> M1[cloud model] & M2[fallback model] & M3[Ollama local<br/>confidential only]
B --> W[diff → approve? → write]
W --> V
D -->|--port| F[approve? → flash board]
Same task, same planted bugs (a missing ; and a reversed comparison in a thermostat), run by
the naive agent (--baseline: whole file + full tool output + growing history, no rules, no
cache) and by BoardLoop. Numbers are printed by the agent itself from the provider's usage
field; reproduce with the commands under Run it.
Live run through Bifrost (local gateway, bifrost/config.json), model chain
anthropic/claude-haiku-4-5 → groq/openai/gpt-oss-120b, 21 Nov 2026. No Anthropic key was
configured, so every call shows Bifrost's automatic fallback to Groq in action:
| Run | Result | LLM calls | 0-token fixes (rule/cache) | Tokens (in+out) | Cost (USD) |
|---|---|---|---|---|---|
| Baseline (naive agent) | verified | 1 | 0/0 | 1,951 (1,630+321) | $0.000437 |
| BoardLoop, first time | verified | 1 | 1/0 | 504 (345+159) | $0.000147 |
| BoardLoop, same bugs again | verified | 0 | 0/2 | 0 | $0.000000 |
BoardLoop, user bob with a $0.00001 budget |
budget_exceeded | 0 | 1/0 | 0 | $0.000000 |
Bifrost's own request log and dashboard for these runs (red = Anthropic refused, green = Groq answered via fallback):
| LLM Logs | Dashboard |
|---|---|
![]() |
![]() |
| Fallback + 504 tokens | Same bugs again: 0 tokens | Naive agent: 1,951 tokens |
|---|---|---|
![]() |
![]() |
![]() |
| Budget stop | Confidential: Ollama only via Bifrost |
|---|---|
![]() |
![]() |
Terminal output of each run, unedited (first run, same bugs again, naive baseline, budget stop):
These runs used the repo's offline arduino-cli/wokwi-cli stand-ins (--fake-tools), because
the ESP32 core and a Wokwi token weren't available on the recording machine; every token, cost
and provider number above is real and comes from Groq via Bifrost.
That is 74% fewer tokens and 66% lower cost on the first run, and 100% fewer on every
repeat. The missing ; never reaches the model (fixed from the compiler message), and only the
comparison lines are sent for the behaviour bug. Small local models (llama3.2 3B) get the
compile fix for free too, but are not strong enough to fix the logic bug on their own: use them
for confidential work and as the last fallback, with a stronger model first in the chain.
The CI test test_compact_prompt_uses_far_fewer_tokens_than_baseline
fails if BoardLoop ever uses more than a third of the baseline's tokens, and
test_fixes_both_bugs_and_replays_from_cache fails if a repeated bug costs a single token. Full suite: 197 tests pass (python -m pytest -q).
A DHT22 temperature sensor, an SSD1306 OLED and an LED on an ESP32, simulated on Wokwi. The sketch
has two planted bugs: a missing ; and a reversed comparison. Every frame below comes from one
unedited run of the same fix loop (mcu_verify), driven from the Hermes Agent chat (left) with
the circuit on Wokwi (right):
The same circuit running live in the wokwi.com editor (DHT22 at 35 °C, OLED alert, LED on), and the OLED screenshots BoardLoop captured during its test run:
| Live on wokwi.com | OLED screenshots from the tests |
|---|---|
![]() |
![]() |
git clone https://github.com/LSUDOKO/BoardLoop && cd BoardLoop
pip install "pyyaml>=6,<7"
# 1. Bifrost gateway (Maxim AI). Providers are in bifrost/config.json; keys come from your env.
docker run -d --name bifrost -p 8080:8080 --add-host=host.docker.internal:host-gateway \
-v "$PWD/bifrost:/app/data" -e OPENAI_API_KEY -e ANTHROPIC_API_KEY maximhq/bifrost:latest
# (or: npx -y @maximhq/bifrost). Open http://localhost:8080 to create a virtual key with a budget.
# 2. A local model for confidential work and last-resort fallback
ollama pull llama3.2
# 3. Fix the buggy demo project (--fake-tools = offline compiler/simulator, no toolchain needed)
python run_agent.py demo --from-template demo-thermostat-buggy --fake-tools \
--model anthropic/claude-haiku-4-5 --model openai/gpt-4o-mini --model ollama/llama3.2 \
--user alice --budget-usd 0.05
# Same bugs again: replayed from the fix cache, 0 tokens
python run_agent.py demo2 --from-template demo-thermostat-buggy --fake-tools --user alice
# The naive agent, for the comparison table
python run_agent.py demo3 --from-template demo-thermostat-buggy --fake-tools --baseline
# Confidential: cloud models are dropped from the chain, code never leaves the machine
python run_agent.py demo4 --from-template demo-thermostat-buggy --fake-tools --confidential \
--model openai/gpt-4o-mini --model ollama/llama3.2
# Without Bifrost, straight to a provider's OpenAI endpoint (e.g. Groq)
export GROQ_API_KEY=... # never commit it
python run_agent.py demo6 --from-template demo-thermostat-buggy --fake-tools --direct \
--gateway https://api.groq.com/openai/v1 --api-key-env GROQ_API_KEY --model groq/openai/gpt-oss-120b
# Budget: a tiny limit stops the agent before the call is made
python run_agent.py demo7 --from-template demo-thermostat-buggy --fake-tools --user bob --budget-usd 0.00001
# Provider outage: point at a dead gateway, the agent keeps going on local Ollama
python run_agent.py demo5 --from-template demo-thermostat-buggy --fake-tools --gateway http://localhost:9/v1Each write stops at [approve] write 1 line(s) to demo.ino? [y/N]. Drop --fake-tools to use
your real arduino-cli and wokwi-cli (BOARDLOOP_ARDUINO_CLI, BOARDLOOP_WOKWI_CLI,
WOKWI_CLI_TOKEN); add --port /dev/ttyUSB0 to flash a real board after it verifies (asks first).
| Option | Meaning |
|---|---|
--gateway URL |
Bifrost OpenAI-compatible endpoint (default http://localhost:8080/v1) |
--model provider/model |
Model chain, first is primary, repeat for fallbacks |
--virtual-key |
Bifrost virtual key: server-side budget and rate limits |
--user, --budget-usd |
Whose budget this run spends, and its limit |
--confidential |
Local ollama/* models only |
--direct, --api-key-env |
Call a provider's own OpenAI-compatible endpoint instead of Bifrost |
--baseline |
Naive agent for comparison |
--yes |
Approve writes without asking (CI only; printed on every write) |
- One endpoint, many providers: the agent only knows
http://localhost:8080/v1; providers and keys live inbifrost/config.json, never in the agent. - Automatic fallbacks: every request carries
fallbacks(the rest of the chain), so a 5xx or timeout on the primary is retried on the next provider inside Bifrost. - Governance: a Bifrost virtual key (
x-bf-vkheader) adds a per-key budget on the gateway; a budget refusal from Bifrost stops the agent cleanly (budget_exceeded). - Observability: every call shows up in Bifrost's request log with provider, tokens and cost.
- Start it with the repo's provider list:
docker run ... -v "$PWD/bifrost:/app/data" maximhq/bifrost(bifrost/config.jsondeclares Ollama, OpenAI, Anthropic and Groq; keys are read from env). - In the Bifrost UI (
http://localhost:8080) → Governance → Virtual keys, create one key per developer with a monthly budget and allowed models. Pass it with--virtual-key. - Run the agent with a chain, e.g.
--model anthropic/claude-haiku-4-5 --model groq/openai/gpt-oss-120b --model ollama/llama3.2. Bifrost tries them in order on every request; the agent prints which provider answered, and Bifrost's request log shows tokens and cost per call.
| Bifrost feature | Used for |
|---|---|
OpenAI-compatible /v1/chat/completions |
the only endpoint the agent knows |
fallbacks in the request body |
provider outage → next provider, same request |
Virtual keys (x-bf-vk) + budgets |
per-developer spend limit on the gateway |
Provider config with env.* keys |
secrets stay out of the agent and the repo |
extra_fields.provider / resolved_model_used |
the agent bills the model that really answered and logs every fallback |
GET /v1/models at startup |
health check: which models the gateway serves; unreachable gateway → client-side failover |
Ollama provider (ollama_key_config) |
confidential projects run on a local model through Bifrost, so they are logged and governed like cloud calls |
| Request logs | per-call tokens/cost/latency for the team |
The model is asked only for what code can't decide: which source line to change, and to what. It answers with JSON line edits that are validated (line in range, no no-op edits) before the user sees a diff. Everything else, compiling, testing, routing, budgets, approvals and the cache, is deterministic code.
The agent drives BoardLoop's tool layer: compile with arduino-cli, behaviour tests on the
Wokwi simulator, wiring checks, and flashing over serial. The same tools are
also packaged as a Hermes Agent plugin (plugin.yaml),
so any agent that speaks tools can use them.
| Tool | What it does | Approval |
|---|---|---|
mcu_doctor |
Checks arduino-cli, cores, libraries, wokwi-cli, the token (never shown), pyserial, the workspace and Wokwi minutes; gives the exact fix for each gap | |
mcu_project_new |
Creates a project from a template: esp32-blank (esp32/s3/c3), esp32-thermostat, uno-blink, demo-thermostat-buggy |
|
mcu_project_info |
Parts list, wiring table (dht1:SDA → esp:4), tests and last result, so code matches the circuit |
|
mcu_libraries |
Search, install (and record in the project) or list Arduino libraries | |
mcu_compile |
Compiles; ≤10 parsed errors with source context and fix hints; flash/RAM use | |
mcu_simulate |
One Wokwi run: expect/fail text, serial tail, part screenshot | |
mcu_test |
Behaviour tests on simulated hardware, each in an isolated run folder; only=[...] reruns failures. target="real" runs them on a connected board: flash, reset, check the serial lines |
real only |
mcu_verify |
The loop step: compile → test → one status with an attempt budget | |
mcu_wiring_check |
Catches input-only pins used as outputs, LEDs without resistors, buttons on strapping pins, parts without ground, plus wokwi-cli lint |
|
mcu_export |
Exports the sketch, diagram and libraries for the wokwi.com editor, to watch the circuit live | |
mcu_ports |
Lists connected boards | |
mcu_upload |
Flashes a real board; the prompt shows board, port, size and whether tests passed | yes |
mcu_serial_read |
Reads a real board's serial output, optionally checking expect_serial |
|
mcu_serial_write |
Sends a line to the board | yes |
Also: the /mcu command (status, projects, history, doctor), a bundled skill that teaches
the agent the workflow and how to write tests (skills/boardloop/SKILL.md),
a Wokwi minutes meter, and a Hermes Desktop half (desktop/plugin.js):
- a Workbench pane with the minutes meter, a project picker, the last verify, the latest
simulated and real-board results with OLED screenshots and failure serial tails, and history.
It reads
dashboard/plugin_api.pyand refreshes when a test run ends; - a screenshot card that shows each OLED screenshot in chat with its test name, PASS/FAIL and reason.
Both are SDK-only (validate: desktop surface passes) and are tested with real React in jsdom
(tests/desktop/); the backend was checked live in the Hermes dashboard server.
- arduino-cli, the cores and the libraries (install guide):
arduino-cli config init arduino-cli config add board_manager.additional_urls https://espressif.github.io/arduino-esp32/package_esp32_index.json arduino-cli core update-index arduino-cli core install esp32:esp32 arduino:avr arduino-cli lib install "Adafruit SSD1306" "Adafruit GFX Library" "DHT sensor library" "Adafruit Unified Sensor"
- wokwi-cli (getting started) and a free CI token from https://wokwi.com/dashboard/ci.
- The plugin:
hermes plugins install boardloop --enable # from the catalog, once listed hermes plugins install LSUDOKO/BoardLoop --enable # or straight from GitHub
- Put the token in Settings → Plugins → BoardLoop → Wokwi CI token (stored as a Hermes secret).
- In Hermes:
/mcu doctor. Every missing piece is listed with its install command.
- "Create an ESP32 project called demo1 from the thermostat template and verify it."
- "The thermostat-buggy project is broken. Make it pass its tests." (create it first with the
demo-thermostat-buggytemplate) - "Make a plant-watering alarm: soil moisture below 30% beeps a buzzer. Write the tests too."
- "Button on GPIO 27 toggles the relay; a long press resets it. Test both."
- "Flash it to my board on /dev/ttyUSB0 and check it prints BOOT OK."
- "Run the tests on my real board on COM5." (
mcu_test target="real": you approve the flash first)
name: thermostat
board: esp32 # esp32 | esp32s3 | esp32c3 | uno | nano | mega, or set fqbn
sketch: thermostat.ino
libraries: ["Adafruit SSD1306", "Adafruit GFX Library", "DHT sensor library", "Adafruit Unified Sensor"]
boot_text: "BOOT OK" # every test waits for this first
fail_text: "ERROR:" # any line containing it fails the run
tests:
- name: hot room turns the LED on
sensors: { dht1: { temperature: 35, humidity: 50 } } # patched into a copy of diagram.json
run_ms: 3000
expect_serial: ["LED=ON"] # in order
expect_pins: [ { part: led1, pin: A, value: 1 } ] # checked at the end
screenshot: oled1
- name: holding the button keeps the LED on
controls: [ { part: btn1, control: pressed, value: 1 } ] # live Wokwi control after boot
run_ms: 1500
expect_pins: [ { part: led1, pin: A, value: 1 } ]Each test becomes a Wokwi automation scenario (scenario.py). Every part id is
checked against diagram.json, so a test can't refer to a part that isn't in the circuit
(projects.py). The skill tells the agent to write a normal case, an opposite case
and an edge case for every behaviour. Teachers can hand out the tests and use them for grading.
mcu_test target="real" port=... uses the same boardloop.yaml. It recompiles if needed and
flashes the board (after your approval). Then it resets the board, captures its serial output and
checks each test's boot_text and expect_serial. Tests that set simulated sensor values or
press simulated buttons can't be reproduced unattended on hardware, so they are reported as
skipped with what to do instead, never as passes. Pin expectations are listed as unchecked.
ESP32 DevKit, ESP32-S3, ESP32-C3, Arduino Uno, Nano and Mega, and any part Wokwi simulates.
The examples use the DHT22, SSD1306 OLED, LEDs, resistors and pushbuttons. BoardLoop works with
any diagram.json; sensor values are part attributes and live inputs are Wokwi controls.
| Setting | Default | |
|---|---|---|
workspace_dir |
<HERMES_HOME>/plugin-data/boardloop/projects |
where projects live |
wokwi_token |
– | secret, env WOKWI_CLI_TOKEN |
wokwi_minutes_limit |
50 | your plan's limit (free plan: 50 per 30 days) |
sim_timeout_ms |
6000 | default for mcu_simulate, max 20000 |
compile_timeout_s |
300 | the first ESP32 build is slow |
compile_budget / test_budget |
3 / 5 | fix attempts before needs_human |
parallel_tests |
3 | Wokwi simulations run at once (1–5) |
default_board, arduino_cli_path, wokwi_cli_path |
What BoardLoop runs and sends
- Local programs: runs
arduino-cli(compile, library install, board list, upload) andwokwi-cli(simulation, lint) as subprocesses with a fixed argv, no shell, a timeout, and a whole-process-tree kill on timeout (procs.py). You install both tools yourself; BoardLoop never downloads them. - Cloud simulation:
wokwi-clisends your compiled firmware and circuit diagram to Wokwi's simulation service using your Wokwi CI token. Simulated time counts against your Wokwi plan; BoardLoop shows an estimate (/mcu) and warns at 80%. Source.inofiles are not uploaded by BoardLoop: each run folder holds onlywokwi.toml, the firmware,diagram.jsonand the scenario (testrunner.py). - Network: arduino-cli downloads cores and libraries from their usual sources when you ask it to. BoardLoop itself sends no telemetry.
- Real hardware: with a board connected, BoardLoop can list serial ports, flash firmware and
read or write serial data. Flashing and serial writes always ask for your approval first
(
approvals.py); under cron or the gateway, Hermes's approval policy refuses them. - Files: projects, build outputs, run folders, screenshots and a small SQLite history live in
Hermes's plugin data folder (
<HERMES_HOME>/plugin-data/boardloop/) or your chosen projects folder. Run folders older than 24 hours are deleted automatically. - Secrets: the Wokwi token is read only through Hermes's secret scope, passed only in
wokwi-cli's environment, and redacted from every tool result (
config.py). arduino-cli and wokwi-cli get a minimal allow-listed environment, never Hermes's own, so provider API keys never reach them (procs.py).
Safety measures in the code, each with a test: path traversal and symlink escapes blocked; strict
patterns for project names, ports and library names; yaml.safe_load only; text read from serial
ports is labelled as data, not instructions; a guard test fails if a new
upload/write/flash/erase tool is added without approval (tests/test_approvals.py).
See SECURITY.md.
The free plan gives 50 simulated minutes per rolling 30 days. Measured on the thermostat: each
test runs about 6 simulated seconds (boot, the DHT22's 2 s warm-up, a 3 s settle), and the CLI
exits as soon as a scenario's last step passes. So a full 3-test verify uses about 18 simulated
seconds, roughly 160 full verifies a month on the free plan. BoardLoop's meter records each
test's timeout budget (9 s) because Wokwi doesn't report the end time of a passing run, so
/mcu errs high (27 s per verify). While fixing, mcu_test only=[...] reruns just the failing
tests.
Wall-clock time is a different matter: ESP32 simulation runs several times slower than real time in Wokwi's cloud, so one ESP32 test takes roughly 40–90 s and a full verify a few minutes. The first ESP32 compile of a project also takes 1–2 minutes; later ones are faster.
esp32-debug diagnoses crashes and rigforge designs the machine; BoardLoop makes the code actually
work on the board. gadget uses ESP32s as voice terminals for Hermes. BoardLoop programs boards
with new code that the agent writes and tests.
- Simulation isn't reality: timing, power, noise and some peripherals differ. Use real-board
mode (
mcu_upload+mcu_serial_read) for the final check. - Wokwi is a cloud service: it needs a network connection and a token, and heavy use needs a paid plan.
- Scenario automation is marked "alpha" by Wokwi. BoardLoop pins its formats to recorded real runs
(
tests/fixtures/wokwi/NOTES.md). - Sensors such as the DHT22 have no live Wokwi control, so their values are set per test before the run, not changed during it.
- The real-board tools (
mcu_upload,mcu_serial_*,mcu_test target="real") are tested with fakes and a fake serial port, and their approval gates are checked through Hermes itself. They have not yet been run against physical hardware for this release. Everything else is verified end to end on the real toolchain (arduino-cli 1.5.1, ESP32 core 3.3.12, wokwi-cli 0.29.1;tests/test_real.py). - An ESP32 verify still takes a few minutes of wall time: the first compile plus cloud simulation.
Tests run in parallel (
parallel_tests, default 3); lower it if your Wokwi plan limits concurrency.
python -m venv .venv && .venv/bin/pip install -r requirements-dev.txt
.venv/bin/pytest # ~180 tests, ~15 s, no network (fake tools)
BOARDLOOP_REAL=1 .venv/bin/pytest -m real # real arduino-cli + Wokwi (uses minutes)
hermes plugins doctor . --ci && hermes plugins validate . --install-deps
python scripts/check_repo.py # no binaries / tokens tracked
<hermes-venv>/bin/python scripts/hermes_e2e.py ~/.hermes/hermes-agent # through Hermes's own dispatcherThe fakes in tests/fakes/ reproduce the real tools' exit codes and messages,
recorded from real runs, and model the example circuits so behaviour tests are meaningful.
Architecture: docs/ARCHITECTURE.md. How the project maps to the
evaluation guide: docs/EVALUATION.md.
boardloop.pages.dev is built from site/ (Vite, React,
Framer Motion) and deployed to Cloudflare Pages with npm run deploy. Its session replay uses the
transcript of a real Hermes run (site/src/session.json) and real Wokwi
frames. The logo, banner and social images are generated by
assets/brand/build.py from the open-licensed Outfit typeface.
MIT © 2026 Arpit Singh

















