You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I ask a question, the model starts generating gibberish text continuously and doesn’t stop at all. I’ve tried rebuilding the project multiple times, but the issue persists right from the first prompt.
System Details:
macOS (Apple Silicon - M4 chip)
CMake version: 3.22.0
Compiler: Apple Clang (Xcode CLT)
gemma.cpp repo from GitHub, built via standard cmake and make
Could this issue be related to the model itself, or is it likely caused by something in the build or runtime setup?
Thanks for reporting. Tests are currently passing for me with Gemma 3 4B on x86, but there might be a difference with M4.
Let's first rule out partial/corrupted weights downloads - sha1sum of the weights should be c8af5c86600f9f389ff157fab76b497e5badf3d9 and for the tokenizer 8a8e05deb279bd5bdca094fb7ececabe732a9534.
Next, are you building from the dev branch? Does an asan build (add -fsanitize=address to CXXFLAGS) report any issues?
sha1sum is the same as yours
4b-it-sfp.sbs => c8af5c86600f9f389ff157fab76b497e5badf3d9
tokenizer.spm => 8a8e05deb279bd5bdca094fb7ececabe732a9534
are you building from the dev branch?
yes
Does an asan build (add -fsanitize=address to CXXFLAGS) report any issues?
I'm very new to this AI environment — in fact, I'm just trying to learn and don’t yet fully understand many of the keywords, as my expertise lies in a different software domain.
I didn’t completely understand what needed to be done, but after doing some research, I followed the steps below. I ran into an error, so I’d appreciate it if you could confirm whether these steps were what you intended — or if not, could you please explain the correct steps in more detail?
I cleared my build/ folder, keeping only the model weight and tokenizer files inside it.
I went back to the root of the gemma.cpp project and ran this command: cmake -B build -DCMAKE_CXX_FLAGS="-fsanitize=address"
After that completed, I ran: cmake --build build
Then I moved into the build/ directory and executed the model with: ./gemma \ --tokenizer tokenizer.spm \ --weights 4b-it-sfp.sbs \ --weight_type sfp \ --model 4b-it
The model loaded and gave a prompt (Hello), but it threw an error.
I’ve copied the full logs and error into a text file — let me know if that would help you debug.
Thank you, that is helpful. The asserts point to the problem: Highway had a mismatch between the feature check and usage of bf16 dot products. I expect to land a fix for that tomorrow.
Glad to hear, thanks for following up :)
The warnings are to be expected, OS X does not provide the APIs that would be required for pinning. We can update the code to skip these warnings on OS X.
Activity
jan-wassenberg commentedon Jun 19, 2025
Thanks for reporting. Tests are currently passing for me with Gemma 3 4B on x86, but there might be a difference with M4.
Let's first rule out partial/corrupted weights downloads - sha1sum of the weights should be c8af5c86600f9f389ff157fab76b497e5badf3d9 and for the tokenizer 8a8e05deb279bd5bdca094fb7ececabe732a9534.
Next, are you building from the dev branch? Does an asan build (add -fsanitize=address to CXXFLAGS) report any issues?
SudarshanShetty1796 commentedon Jun 19, 2025
sha1sum is the same as yours
4b-it-sfp.sbs => c8af5c86600f9f389ff157fab76b497e5badf3d9
tokenizer.spm => 8a8e05deb279bd5bdca094fb7ececabe732a9534
yes
I'm very new to this AI environment — in fact, I'm just trying to learn and don’t yet fully understand many of the keywords, as my expertise lies in a different software domain.
I didn’t completely understand what needed to be done, but after doing some research, I followed the steps below. I ran into an error, so I’d appreciate it if you could confirm whether these steps were what you intended — or if not, could you please explain the correct steps in more detail?
cmake -B build -DCMAKE_CXX_FLAGS="-fsanitize=address"cmake --build build./gemma \ --tokenizer tokenizer.spm \ --weights 4b-it-sfp.sbs \ --weight_type sfp \ --model 4b-itI’ve copied the full logs and error into a text file — let me know if that would help you debug.
Thanks in advance!
Error.txt
jan-wassenberg commentedon Jun 19, 2025
Thank you, that is helpful. The asserts point to the problem: Highway had a mismatch between the feature check and usage of bf16 dot products. I expect to land a fix for that tomorrow.
SudarshanShetty1796 commentedon Jun 21, 2025
Hello,
Is this fixed? Can I try?
jan-wassenberg commentedon Jun 23, 2025
The Highway fix has landed and #630 should go in soon as well.
Please let us know if that fixes it for you.
SudarshanShetty1796 commentedon Jun 24, 2025
I tried it, and it's working fine now! Thank you.
I'm able to chat with the model.
However, when I run the ./gemma command, it prints the following error. Is this something to be concerned about, or can it be ignored?
jan-wassenberg commentedon Jul 3, 2025
Glad to hear, thanks for following up :)
The warnings are to be expected, OS X does not provide the APIs that would be required for pinning. We can update the code to skip these warnings on OS X.
Avoid affinity related warnings on Apple. Refs #625
Avoid affinity related warnings on Apple. Refs #625
Avoid affinity related warnings on Apple. Refs #625
Avoid affinity related warnings on Apple. Refs #625