You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Support TranslateGemma 4B text-only inference (Gemma 3 4B LM-only, no VIT) #888
TranslateGemma 4B (google/translategemma-4b-it) is Google's translation-specific model built on Gemma 3 4B architecture. It is a text-only model (no vision encoder needed for inference), but gemma.cpp currently only supports the VLM variant of Gemma 3 4B.
Problem
When running TranslateGemma 4B with gemma.cpp:
SBS conversion: convert_from_safetensors.py assumes PaliGemma VLM format — requires vision tower tensors that TranslateGemma doesn't have
Loading: ConfigGemma3_4B() returns VLM config with vit_config.image_size=896, causing Tensor enc_norm_bias is required but not found in file error
No LM-only dispatch: ConfigGemma3_4B_LM() exists in code but is never used as the primary config
What We Did (Workarounds)
We successfully ran TranslateGemma on gemma.cpp with these changes:
1. Convert script modifications
Skip vision_tower.* and multi_modal_projector.* tensors during loading
Fix vocab_size (262144 instead of PaliGemma's 257152+64 trim)
Add QK norm tensors (query_norm, key_norm) to layer config as BF16
Zero out vit_config in SBS metadata before writing
2. C++ changes needed
configs.cc: Dispatch GEMMA3_4B to ConfigGemma3_4B_LM() when no VIT tensors present
tensor_info.cc: Guard VIT tensor registration with if (config.vit_config.image_size > 0)
Hi, thanks for reaching out. These changes sound very reasonable and we're happy to help enable support for TranslateGemma.
It sounds like you have the changes already - would you like to send them as a pull request to the dev branch?
@jan-wassenberg This will also be super useful for my personal research so I went ahead and created a PR. Please let me know if you have any feedback. Thanks.
@jan-wassenberg friendly ping on this — PR #918 has been open since 5/19 with CLA green and no conflicts. Happy to address any feedback whenever you have a chance to take a look. Thanks!
Activity
jan-wassenberg commentedon Apr 7, 2026
Hi, thanks for reaching out. These changes sound very reasonable and we're happy to help enable support for TranslateGemma.
It sounds like you have the changes already - would you like to send them as a pull request to the dev branch?
plawanrath commentedon May 19, 2026
@jan-wassenberg This will also be super useful for my personal research so I went ahead and created a PR. Please let me know if you have any feedback. Thanks.
plawanrath commentedon May 25, 2026
@jan-wassenberg friendly ping on this — PR #918 has been open since 5/19 with CLA green and no conflicts. Happy to address any feedback whenever you have a chance to take a look. Thanks!
jan-wassenberg commentedon May 26, 2026
Sorry to reply late. I've been OOO and am now back. Thanks for sending the PR! Happy to land this.
Add Gemma 3 LM-only model variants (issue google#888)