v2.2.1

What's Changed

• README: separate the primary target from secondary support by @xycld in #9
• sm75: VAE encoder, runtime LoRAs, quantized tiers and a q8_fp16 attention kernel on Turing by @CloverIris in #13
• docs: contributing guide, PR template, CODEOWNERS; build trees in build/ by @xycld in #14
• Version 2.2.1 by @xycld in #15

Full Changelog: v2.2.0...v2.2.1

v2.2.0

What's Changed

• moss-infer-vulkan: the Vulkan fallback program, with Linux and Windows packages by @CloverIris in #8
• moss-infer-hip: the native AMD (ROCm/HIP) program by @xycld in #11
• moss-infer-vulkan: the engine's request checks, --help/--version, --vram-budget by @xycld in #10
• Version 2.2.0 by @xycld in #12

Full Changelog: v2.1.2...v2.2.0

v2.1.2

What's Changed

• README: speed and GPU memory charts on an RTX 5090 by @xycld in #4
• README charts: scatter plots instead of lines between configurations by @xycld in #5
• Add sm70 support by @samyycX in #6
• Version 2.1.2 by @xycld in #7

New Contributors

• @samyycX made their first contribution in #6

Full Changelog: v2.1.1...v2.1.2

v2.1.1

What's Changed

• sm_75 (Turing) support: fp16 GEMMs, INT8 GEMMs, exact attention, VAE by @CloverIris in #1
• sm_75 follow-ups: fp32 GEMM output in the workspace, attention reads bf16, FA2 causal alignment by @xycld in #2
• Version 2.1.1 by @xycld in #3

New Contributors

• @CloverIris made their first contribution in #1
• @xycld made their first contribution in #2

Full Changelog: v2.1.0...v2.1.1

v2.1.0

MossInfer 2.1.0 adds features on top of 2.0 and keeps its interfaces: ABI 1, server protocol 2. Builds and applications written for 2.0 run unchanged.

New

• Edit adapters (reference images). Edit adapters such as chinoll/Moss-Image-Edit load with --lora and edit up to four reference images (--reference). On a bf16 DiT the adapter is merged on the GPU at load, about 1 s. On INT8, INT4 and NVFP4 it runs as a side branch. moss-infer quantize --lora bakes it into an INT8 file. Runs at an 8 GiB VRAM budget.
• GGUF DiT files. Community GGUF DiTs (Q8_0, Q6_K, Q5_K, Q5_1, Q5_0, Q4_K, Q4_1, Q4_0, BF16, F16, F32) load as dit_file. They are converted once into INT8 ConvRot, or into bf16 for tier reference, and kept in cache_dir. Their speed and VRAM are those of the format they run as.
• Component API. Run the pipeline step by step from your own sampler loop: text encoding, run setup, single denoise steps with host or engine guidance, latent batches, VAE encode/decode. It is in the C library and in moss-server (over shared memory). With the same inputs the steps reproduce mi_generate bit for bit. See docs/c-api.md and docs/server-protocol.md.
• Check before running. mi_check(), the server's check method and moss-infer check plan a request without loading weights. They report whether it fits, the plan and an estimate of the time.
• Runtime LoRA placement. --lora-placement default|merge|side_branch. bf16 merges now run on the GPU.

Faster

• INT8 high: 3.2 % less time per step at 1024² and 4.8 % less at 2048² on an RTX 5090, with bit-identical images. INT8 + 8-bit attention now takes 0.186 s per step at 1024², 2.5× the bf16 reference. It is 1.13× faster per step than ComfyUI's INT8 at 1024² and 1.08× faster at 2048².
• LoRA side branch on INT8: a rank-16 LoRA costs about 5 % per step, down from 8.7 %.

Hardened

• Windows file handling. User paths, cache entries and output files go through handles that refuse pipes, devices and links leading outside their folder. Files are published by rename.
• Model files replaced while the engine runs load anew at the next generate, or fail cleanly with the file named.

Compatibility

• mi_generate output is bit-identical to 2.0.0, and its speed is unchanged or faster. Measured on an RTX 5090 over 16 paired processes per configuration.
• New capability bits: MI_CAP_COMPONENTS, MI_CAP_REFERENCE_IMAGES, MI_CAP_CHECK. New server capability: components.

v2.0.0

Full Changelog: v1.0.0...v2.0.0

v1.0.0

MossInfer 1.0.0: the first stable release of the native C++/CUDA inference engine for MOSS-Image. It needs no PyTorch or Python at run time.

Highlights

• Memory planning: the engine plans against the GPU memory that is actually free, with three modes: min_memory, balanced and max_speed. When the model doesn't fit, it streams DiT blocks; a 1024² image runs in about 2.3 GB of VRAM.
• Reference tier: exact bf16, matching PyTorch. Per DiT step it is 1.33–1.39× faster than PyTorch (measured on an RTX 5090). On a 32 GB card a full 1024² image is 1.45× faster than the official pipeline running fully resident; on 16 GB cards, where the official pipeline has to offload, the engine is 2–3.5× faster.
• Quantized models: INT8 ConvRot and INT4 ConvRot DiT files, INT8 and W4A8 text encoders (ComfyUI / community formats), and 8-bit attention.
• Runtime LoRAs: kohya, PEFT / diffusers, LyCORIS LoHa / LoKr and full deltas, applied per generation.
• Generation: 9 samplers and 7 schedulers, img2img, inpainting, a hires fix, distill bundles, latent previews, and PNG metadata.
• Interfaces: a C API (libmossinfer, ABI 1) and the moss-server JSON-lines sidecar.
• Packages: Windows x64 and Linux x86-64, each with CUDA 12 and CUDA 13 builds. Each comes as a full package (bundled cuBLASLt) and a small -system-cuda package that uses an installed CUDA.
• Command line only in 1.0: control LoRAs (colour / depth maps) and the lossy accelerators (CFG truncation, FBCache step cache). They come to the C API and the server in 1.1.

Notes

• No quality certificates ship in 1.0. auto therefore runs the exact Reference tier for the bf16 model. Choose high or turbo explicitly for the quantized speed tiers; those runs are reported as uncertified. RTX 50 certificates are in progress.
• GPU support: NVIDIA compute capability 8.x and 12.x (RTX 30, RTX 40, RTX 50, A100, L4/L40). GPU runs are verified on Linux. Windows builds are compiled and CPU-tested in CI.

Verification of this build

• CI: builds, packages and CPU tests pass on all four flavours.
• GPU test suite: 410/410 on an RTX 5090 (sm_120). The bit-exact subset, 11/11, passes on an RTX 4090 Laptop (sm_89).
• With the new features off: output is byte-identical to the previous main in 5 configurations.

See docs/integration.md for integrating the engine and SHA256SUMS to verify the downloads.

Full Changelog: https://github.com/xycld/owoinfer/commits/v1.0.0