You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[User Experience Failure] Impossible to Build and Run on a Resource-Constrained Google Edge Device (Chromebook) #688
I am a (hobby) programmer with 40 years of experience...blah. it's actually 45 years. My first machine was the TRS80. By 1999 I was compiling Allegro Games on 500 MHz. Logically, I am that dinosaur that would gravitate towards gemma.cpp
I tend to isolate and don't socialize much online. New to GitHub.
My goal was to follow the official documentation to compile and run a Gemma 3 model on a standard, Google-made edge device: an Acer 715 Chromebook with the Linux Development Environment. This is the exact use case for which gemma.cpp is marketed.
The official workflow for an on-device build is fundamentally broken for a resource-constrained environment. I have spent multiple days navigating a series of undocumented dependencies, build failures, and broken documentation. The process failed at every critical stage.
Build Process Failure: The standard build process fails with an Out-of-Memory (OOM) error on a standard Chromebook. The compiler is killed by the OS.
Build Process Workaround Failure: Forcing a single-threaded build (-j 1) also fails with an OOM error, proving that even single source files have memory requirements that exceed the capacity of a standard edge device.
Partial Build Catastrophe: The only way to compile the C++ executable is to disable tests (-DGEMMA_ENABLE_TESTS=OFF). However, this silently and catastrophically fails to build the required Python libraries (compression.python), making the necessary convert_from_safetensors.py script non-functional with an ImportError. This is a critical, undocumented failure of the build system.
Documentation Failure (Missing Files): The official documentation directs users to download pre-converted .sbs/.sfp model files from Kaggle. These files do not exist for the smaller 1B/4B models. The documentation is a lie.
Dependency Hell: The Python conversion script has numerous undocumented dependencies (numpy, torch, sentencepiece, etc.). The requirements.txt file is woefully incomplete, forcing users into a painful, multi-step ModuleNotFoundError loop.
Bloatware Requirement: The conversion process requires installing the entire, multi-gigabyte PyTorch library (including useless NVIDIA drivers) on a small edge device for what should be a simple file format conversion. This is a complete betrayal of the project's goal of being lightweight and efficient.
As it stands, gemma.cpp is not a usable tool for its target audience on its target hardware. It is impossible to correctly compile the full project and its dependencies on a standard Chromebook. The documentation is incorrect, and the workflow is hostile to the user.
For this Google project to fulfill its mission, it must provide one of two things:
A build process that can actually be completed on a resource-constrained device.
The pre-converted .sbs/.sfp model files for all model sizes, as the documentation incorrectly promises.
Cool, I also made my first baby steps on TRS-80 and Allegro rings a bell :) Thanks for sharing your (painful) experience here.
single source files have memory requirements that exceed the capacity of a standard edge device.
Yes, that makes sense. The matmul file requires quite some work by the compiler.
I would not advise compiling on the device itself, but rather cross-compiling on a developer machine.
(Personally, I am most interested in the HEDT/workstation use case.)
Documentation Failure (Missing Files): The official documentation directs users to download pre-converted .sbs/.sfp model files from Kaggle. These files do not exist for the smaller 1B/4B models.
I successfully downloaded the official, pre-converted 270m-it-sfp.sbs file from the link provided in this thread. I then ran it with the gemma executable that was compiled on this Chromebook (using the -DGEMMA_ENABLE_TESTS=OFF flag, which was the only way to overcome the out-of-memory errors during compilation).
Here is the result of the first prompt:
The program immediately fails with a Segmentation fault (core dumped).
This confirms, at least for me, that the official gemma.cpp toolchain, when built on its target edge hardware following the necessary workarounds, is unstable at runtime. The on-device build path, as it currently stands, is not a viable solution for this hardware.
Is there a fundamental disconnect between the project's public marketing and its internal development focus?
The official Google for Developers blog, in its announcement for the Gemma 3 270m, explicitly recommends and links to gemma.cpp as a viable tool for on-device inference. This sets a clear user expectation that the project is optimized for resource-constrained hardware like Chromebooks. My decision to use gemma.cpp was based on this logic, to stay within the Google ecosystem since my plan involves fine-tuning on Colab; Google all the way. I would like to tell the World, lo! I did this ALL with Google products.
In contrast, you have stated that your personal focus is on the HEDT/workstation use case.
While I made many mistakes on my own learning journey, this strategic misalignment is the root cause of the on-device user's struggle. A focus on high-end hardware has inevitably led to a development process (heavy build requirements, complex Python dependencies) that is fundamentally at odds with the needs of the very users the project is being marketed to.
Harrumph. For posterity.
On the build process failure when attempting to compile gemma.cpp on a resource-constrained edge device (Acer 715 Chromebook, Debian Linux container).
This was over two days.
Day one:
The build process consistently fails at 56% progress with an Out-of-Memory (OOM) error. The OS terminates the c++ compiler process (Killed signal) when it attempts to compile a specific test file, making a full on-device build impossible on this hardware.
The failure occurs on the same file every time, even when the build is restricted to a single thread (-j 1) to minimize resource consumption.
Failing File: _deps/highway-build/CMakeFiles/masked_arithmetic_test.dir/hwy/tests/masked_arithmetic_test.cc.o
This demonstrates that the memory requirement for compiling this single test file from the Google Highway dependency exceeds the available RAM on this target hardware.
Compiler Output:
The failure log is identical in both the default multi-threaded build and the single-threaded build:
This memory bottleneck makes on-device build impossible. It forces the user to adopt the -DGEMMA_ENABLE_TESTS=OFF flag as a workaround. I fell asleep and woke with joy. The file was compiled.
Day two: The model ordeal. I was shocked when I saw what was being installed on this machine...I digress
The requirements.txt issue is probobly my error?
The command that directly caused the cascade of ModuleNotFoundError errors was:
pip install -r python/requirements.txt
I then added the --break-system-packages flag to this command to force it to run.
I executed this command from the ~/gemma.cpp directory.
Today , I discovered, that specific requirements.txt file only contains absl-py.
It does not contain numpy, sentencepiece, safetensors, or torch.
Therefore, this command directly led to the series of ModuleNotFoundError errors because it failed to install the necessary libraries for the convert_from_safetensors.py script to run.
Activity
jan-wassenberg commentedon Sep 3, 2025
Cool, I also made my first baby steps on TRS-80 and Allegro rings a bell :) Thanks for sharing your (painful) experience here.
Yes, that makes sense. The matmul file requires quite some work by the compiler.
I would not advise compiling on the device itself, but rather cross-compiling on a developer machine.
(Personally, I am most interested in the HEDT/workstation use case.)
huh, that's a surprise. I see them here: https://www.kaggle.com/models/google/gemma-3/gemmaCpp/3.0-1b-it-sfp (also 3.0-4b-it-sfp)
Have you signed in to Kaggle and consented to the usage terms?
Sorry about that. Now that you have gathered a list, can you share it with us to add to the .txt?
TaylorShards commentedon Sep 12, 2025
@n-wassenberg,
My results on the on-device build path.
I successfully downloaded the official, pre-converted 270m-it-sfp.sbs file from the link provided in this thread. I then ran it with the gemma executable that was compiled on this Chromebook (using the -DGEMMA_ENABLE_TESTS=OFF flag, which was the only way to overcome the out-of-memory errors during compilation).
Here is the result of the first prompt:
The program immediately fails with a Segmentation fault (core dumped).
This confirms, at least for me, that the official gemma.cpp toolchain, when built on its target edge hardware following the necessary workarounds, is unstable at runtime. The on-device build path, as it currently stands, is not a viable solution for this hardware.
Is there a fundamental disconnect between the project's public marketing and its internal development focus?
The official Google for Developers blog, in its announcement for the Gemma 3 270m, explicitly recommends and links to gemma.cpp as a viable tool for on-device inference. This sets a clear user expectation that the project is optimized for resource-constrained hardware like Chromebooks. My decision to use gemma.cpp was based on this logic, to stay within the Google ecosystem since my plan involves fine-tuning on Colab; Google all the way. I would like to tell the World, lo! I did this ALL with Google products.
In contrast, you have stated that your personal focus is on the HEDT/workstation use case.
While I made many mistakes on my own learning journey, this strategic misalignment is the root cause of the on-device user's struggle. A focus on high-end hardware has inevitably led to a development process (heavy build requirements, complex Python dependencies) that is fundamentally at odds with the needs of the very users the project is being marketed to.
Harrumph. For posterity.
On the build process failure when attempting to compile gemma.cpp on a resource-constrained edge device (Acer 715 Chromebook, Debian Linux container).
This was over two days.
Day one:
The build process consistently fails at 56% progress with an Out-of-Memory (OOM) error. The OS terminates the c++ compiler process (Killed signal) when it attempts to compile a specific test file, making a full on-device build impossible on this hardware.
The failure occurs on the same file every time, even when the build is restricted to a single thread (-j 1) to minimize resource consumption.
Failing File: _deps/highway-build/CMakeFiles/masked_arithmetic_test.dir/hwy/tests/masked_arithmetic_test.cc.o
This demonstrates that the memory requirement for compiling this single test file from the Google Highway dependency exceeds the available RAM on this target hardware.
Compiler Output:
The failure log is identical in both the default multi-threaded build and the single-threaded build:
code
Code
download
content_copy
expand_less
[ 56%] Building CXX object _deps/highway-build/CMakeFiles/masked_arithmetic_test.dir/hwy/tests/masked_arithmetic_test.cc.o
c++: fatal error: Killed signal terminated program cc1plus
compilation terminated.
gmake[2]: *** [...] Error 1
gmake[1]: *** [...] Error 2
gmake: *** [Makefile:156: all] Error 2
This memory bottleneck makes on-device build impossible. It forces the user to adopt the -DGEMMA_ENABLE_TESTS=OFF flag as a workaround. I fell asleep and woke with joy. The file was compiled.
Day two: The model ordeal. I was shocked when I saw what was being installed on this machine...I digress
The requirements.txt issue is probobly my error?
The command that directly caused the cascade of ModuleNotFoundError errors was:
pip install -r python/requirements.txt
I then added the --break-system-packages flag to this command to force it to run.
I executed this command from the ~/gemma.cpp directory.
Today , I discovered, that specific requirements.txt file only contains absl-py.
It does not contain numpy, sentencepiece, safetensors, or torch.
Therefore, this command directly led to the series of ModuleNotFoundError errors because it failed to install the necessary libraries for the convert_from_safetensors.py script to run.