[Idea]: Image encoder/multimodal support #29

Activity

1. Vasyl-Trefilov commented on Nov 12, 2026

   

   Vasyl-Trefilov

   on Nov 12, 2026

   Member

   More actions

   Yes, I have not implemented it yet, but this is maybe the next things that I will do, so I will tag you when it's done, maybe 1-2 days
2. david-boles commented on Nov 12, 2026

   

   david-boles

   on Nov 12, 2026

   Author

   More actions

   Ha! Fantastic; can't wait to try it. Thanks for your work!

   ❤️React with ❤️1Reacted by Vasya
3. Vasyl-Trefilov commented on Nov 13, 2026

   

   Vasyl-Trefilov

   on Nov 13, 2026

   Member

   More actions

   @david-boles Hi, it's implemented, here is few examples of usage

   target/release/rustingstrata chat \
           --model  /mnt/ollama/models/.studio_links/9a9ae26c56/qwen3.5-2b.gguf \
           --mmproj /mnt/ollama/models/.studio_links/9a9ae26c56/qwen3.5-2b-mmproj.gguf \
           --image /tmp/red.png --gpu
   cuda device 0: NVIDIA GeForce RTX 3060, sm_86
   plan: weights+state+scratch 2272 MB (KV 48 MB for 4096 tokens, prefill 321 MB), expert slots 0 MB, resident experts 0/0
   chat template: Xml
   > What color is the image? One word.
    red
   

   or maybe Gemma

   printf 'What color is the image? One word.\n' | \
         target/release/rustingstrata chat \
           --model /home/vasylt/models/gemma4-26b/gemma-4-26B-A4B-it-UD-Q5_K_M.gguf \
           --image /tmp/red.png --gpu -n 8
   cuda device 0: NVIDIA GeForce RTX 3060, sm_86
   > Red
   

   But there are still some problems, there are no GPU vision, so it takes like 1-3min for bigger models like Qwen3.6:35b-a3b. So I will improve the speed for sure, but for now it's already usable, but just takes way for long
4. Vasyl-Trefilov commented on Nov 13, 2026

   

   Vasyl-Trefilov

   on Nov 13, 2026

   Member

   More actions

   Forgot to say, it also support OpenAI API