ik-llama-cpp

Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

llama.cpp based distributed MoE inference platform for mixed architecture clusters

Turboquant Q4/Q3 with IQK FA

A build log for one workstation: a 284B mixture-of-experts model across two GPUs and 256 GB of RAM, measured.

Turboquant Q4/Q3 with IQK FA

K2 + SantanaQAT: 3-bit trellis quantization measurement study — paper (20 pp), 48-file evidence archive, SC→RCO allocation pipeline, reference artifacts

A fork of ik_llama.cpp that enables AIs to perform web searches

无独显笔记本跑 30B 代码模型：ik_llama.cpp + MoE 量化，纯 CPU 13.67 tok/s，OpenAI 兼容、断网可用、零 API 费用 | CPU-only local LLM on Windows, no GPU needed

A one-command inference server and benchmark harness for running large language models that don't fit in your GPU's VRAM. Optimized for RTX PRO 6000.