You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Showcase / question: a board-proven offline language runtime on ESP32-C3, and whether this points to a more extreme form of language/runtime co-design #875
I wanted to share a small but unusual language-runtime project that may be relevant to the kind of language/runtime co-design boundary this repo is already exploring.
We built a public demo line called Engram and deployed it on a commodity ESP32-C3.
Current public numbers:
Host-side benchmark capability
LogiQA = 0.392523
IFEval = 0.780037
Published board proof
LogiQA 642 = 249 / 642 = 0.3878504672897196
host_full_match = 642 / 642
runtime artifact size = 1,380,771 bytes
Important scope note:
This is not presented as unrestricted open-input native LLM generation on MCU.
The board-side path is closer to a flash-resident, table-driven runtime with:
packed token weights
hashed lookup structures
fixed compiled probe batches
streaming fold / checksum style execution over precompiled structures
So this is not a standard lightweight dense inference runtime on a small device. It is closer to a task-specialized language runtime whose behavior has been crystallized into a compact executable form under very severe physical constraints.
Why I’m posting here is that gemma.cpp seems to sit at an interesting point between research-friendly implementation, low-level execution, and simplified language inference systems.
What I’d be curious about is whether systems like this should be thought of as:
outside the normal lightweight inference-runtime family
an extreme endpoint of language/runtime co-design
or an early sign that some language-task capability may eventually be deployed in more specialized executable forms than even a minimalist dense runtime
If this direction is relevant to your team, I’d be glad to compare notes.
Hi, I don't have any expertise on embedded HW nor comments on your approach - beyond amazement that it is possible to do anything LLM-like with a single-core 160 MHz CPU :)
Hi, I don't have any expertise on embedded HW nor comments on your approach - beyond amazement that it is possible to do anything LLM-like with a single-core 160 MHz CPU :)
Actually, I'm not very familiar with embedded systems. This product was a happy accident from my personal AGI research, and I'm still actively working on fundamental AGI infrastructure.
@jan-wassenberg
Since we are both exploring the absolute boundaries of language runtimes—your team optimizing for peak dense inference, and me exploring extreme physical constraints—I’d love to see if there’s any synergy.
If the Gemma ecosystem ever plans to explore these highly constrained edge environments or specialized executable formats, I’d be thrilled to collaborate or contribute.
Let me know if you'd be open to a quick virtual coffee sometime to share notes! You can reach me at kh.deng@foxmail.com. Either way, huge fan of your work on Highway and Gemma.
Activity
jan-wassenberg commentedon Mar 20, 2026
Hi, I don't have any expertise on embedded HW nor comments on your approach - beyond amazement that it is possible to do anything LLM-like with a single-core 160 MHz CPU :)
Juemian-Deng commentedon Mar 20, 2026
Actually, I'm not very familiar with embedded systems. This product was a happy accident from my personal AGI research, and I'm still actively working on fundamental AGI infrastructure.
Juemian-Deng commentedon Mar 20, 2026
@jan-wassenberg
Since we are both exploring the absolute boundaries of language runtimes—your team optimizing for peak dense inference, and me exploring extreme physical constraints—I’d love to see if there’s any synergy.
If the Gemma ecosystem ever plans to explore these highly constrained edge environments or specialized executable formats, I’d be thrilled to collaborate or contribute.
Let me know if you'd be open to a quick virtual coffee sometime to share notes! You can reach me at kh.deng@foxmail.com. Either way, huge fan of your work on Highway and Gemma.
jan-wassenberg commentedon Mar 26, 2026
Thanks for the warm words. So far we don't have any plans for highly constrained environments :)