diff --git a/README.md b/README.md index 4703e0b..7054561 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ A native C++/CUDA inference engine for the MOSS-Image text-to-image model (5B Di Models: MOSS-Image Base and SFT, in the official folders and common community formats; other models are refused ([supported models](#supported-models)). -Version 2.1.2. Supported: Linux x86-64 and Windows x64 (MSVC), each with a CUDA 12 and a CUDA 13 build, as a full package and a `-system-cuda` package. Turing (compute capability 7.5, e.g. RTX 2080) is supported by builds configured with `-DMOSSINFER_SM75=ON` (the `windows-msvc-cuda13-sm75` preset packages it): fp16 tensor-core GEMMs replace the bf16 ones, INT8 and INT4 ConvRot files run the INT8 GEMMs, the exact attention and the VAE run sm_75 kernels, all three quality tiers work, and an unset attention resolves to `exact`. See [docs/sm75.md](docs/sm75.md) for what runs, the measured speed on an RTX 2080 Ti and the remaining gaps. Volta (Tesla V100, sm_70) is supported by the CUDA 12 `linux-cuda12-sm70` preset; see [docs/sm70.md](docs/sm70.md) for its kernels, validation and performance. Other GPUs (AMD, Intel, older NVIDIA) can run txt2img at the reference tier through `moss-infer-vulkan`, a separate and slower Vulkan program in its own package ([docs/vulkan.md](docs/vulkan.md)); on Linux, `moss-infer-hip`, a separate ROCm 7 program, runs it on AMD GPUs: built for gfx1100, gfx1200 and gfx1201 (Radeon RX 7900 and RX 9000), validated on gfx1200 (RX 9060 XT) ([docs/hip.md](docs/hip.md)). The full support matrix is in [docs/integration.md](docs/integration.md#support-status). +Version 2.2.0. Supported: Linux x86-64 and Windows x64 (MSVC), each with a CUDA 12 and a CUDA 13 build, as a full package and a `-system-cuda` package. Turing (compute capability 7.5, e.g. RTX 2080) is supported by builds configured with `-DMOSSINFER_SM75=ON` (the `windows-msvc-cuda13-sm75` preset packages it): fp16 tensor-core GEMMs replace the bf16 ones, INT8 and INT4 ConvRot files run the INT8 GEMMs, the exact attention and the VAE run sm_75 kernels, all three quality tiers work, and an unset attention resolves to `exact`. See [docs/sm75.md](docs/sm75.md) for what runs, the measured speed on an RTX 2080 Ti and the remaining gaps. Volta (Tesla V100, sm_70) is supported by the CUDA 12 `linux-cuda12-sm70` preset; see [docs/sm70.md](docs/sm70.md) for its kernels, validation and performance. Other GPUs (AMD, Intel, older NVIDIA) can run txt2img at the reference tier through `moss-infer-vulkan`, a separate and slower Vulkan program in its own package ([docs/vulkan.md](docs/vulkan.md)); on Linux, `moss-infer-hip`, a separate ROCm 7 program, runs it on AMD GPUs: built for gfx1100, gfx1200 and gfx1201 (Radeon RX 7900 and RX 9000), validated on gfx1200 (RX 9060 XT) ([docs/hip.md](docs/hip.md)). The full support matrix is in [docs/integration.md](docs/integration.md#support-status). The caller picks the quality tier: `reference` (bf16, exact), `high` (the loaded DiT's GEMMs with 8-bit attention) or `turbo` (a 4-bit DiT). An INT8 DiT with `high` is the everyday choice: 2.5 times the bf16 speed on an RTX 5090, in 40 % less VRAM ([performance](docs/integration.md#performance)). Upgrading from 1.x: [migration](docs/integration.md#migrating-from-1x). diff --git a/docs/integration.md b/docs/integration.md index 92ad491..d0db201 100644 --- a/docs/integration.md +++ b/docs/integration.md @@ -8,7 +8,7 @@ The reference documents are: This page does not repeat them. It explains how the pieces fit together, and which choices the launcher has to make. -Version 2.1.2 (ABI 1, server protocol 2). See [Versioning](#versioning) for what stays compatible, and [Migrating from 1.x](#migrating-from-1x) for what changed. +Version 2.2.0 (ABI 1, server protocol 2). See [Versioning](#versioning) for what stays compatible, and [Migrating from 1.x](#migrating-from-1x) for what changed. MossInfer is a low-level engine: it runs what the caller asks for, refuses what the hardware or the files cannot run with the reason, and reports what ran. It does not choose for the caller. The launcher picks the tier, the memory mode and the budgets; [Performance](#performance) says what each costs and gives. @@ -205,7 +205,7 @@ cmake --build --preset linux-cuda13 -j8 CUDA_VISIBLE_DEVICES="" ctest --preset linux-cuda13 -L '^cpu$' cmake --install build-linux-cuda13 # -> build-linux-cuda13/install cmake --install build-linux-cuda13 --component Unspecified --prefix build-linux-cuda13/install-system-cuda -(cd build-linux-cuda13 && cpack --config CPackConfig.cmake) # -> mossinfer-2.1.2-linux-cu13-x86_64.{tar.gz,zip} +(cd build-linux-cuda13 && cpack --config CPackConfig.cmake) # -> mossinfer-2.2.0-linux-cu13-x86_64.{tar.gz,zip} ``` - **glibc:** the archive needs the glibc of the machine that built it, or a newer one. Build on the oldest distribution you want to support. CI uses Ubuntu 22.04, which gives glibc 2.35. A build from a rolling distribution such as Arch will not start on Ubuntu LTS. @@ -555,7 +555,7 @@ Attach it to bug reports, together with the server's stderr log and `hello`'s `p ## Versioning - **Version numbers.** - - `mi_version()` returns the release version (`2.1.2`). + - `mi_version()` returns the release version (`2.2.0`). - `mi_abi_version()` and `MI_ABI_VERSION` give the binary interface version (1). - The server's `hello` reports its `protocol_version` (2). - **Additive changes keep ABI 1:** new functions, new fields at the end of structs, new enum values, new optional server parameters and events. diff --git a/docs/server-protocol.md b/docs/server-protocol.md index 3dc60b1..285abe3 100644 --- a/docs/server-protocol.md +++ b/docs/server-protocol.md @@ -8,7 +8,7 @@ ``` moss-server # speak the protocol on stdin/stdout moss-server --parent-pid # the same, and exit when process (the launcher) exits -moss-server --version # "moss-server 2.1.2 (protocol 2, ABI 1)" +moss-server --version # "moss-server 2.2.0 (protocol 2, ABI 1)" ``` ### `--parent-pid` diff --git a/docs/sm75.md b/docs/sm75.md index 1f7f413..a7fdf2f 100644 --- a/docs/sm75.md +++ b/docs/sm75.md @@ -17,7 +17,7 @@ cmake --preset windows-msvc-cuda13-sm75 cmake --build --preset windows-msvc-cuda13-sm75 ctest --preset windows-msvc-cuda13-sm75 -L '^cpu$' cpack --config build-windows-msvc-cuda13-sm75/CPackConfig.cmake -B build-windows-msvc-cuda13-sm75 -# -> build-windows-msvc-cuda13-sm75/mossinfer-2.1.2-windows-cu13-x86_64-sm75.zip +# -> build-windows-msvc-cuda13-sm75/mossinfer-2.2.0-windows-cu13-x86_64-sm75.zip ``` Any other preset takes the option too (`-DMOSSINFER_SM75=ON`), e.g. a Linux build from a custom preset; the ordinary diff --git a/docs/vulkan.md b/docs/vulkan.md index ca754a8..497fb15 100644 --- a/docs/vulkan.md +++ b/docs/vulkan.md @@ -57,7 +57,8 @@ driver's watchdog on a slow GPU. ## Accuracy -Against the official PyTorch pipeline's golden data at 1024x1024 (`tests/golden/g_fridge_1024`), on the `coop` path: +Against the official PyTorch pipeline's golden data at 1024x1024 (`tests/golden/g_fridge_1024`), on the `coop` path. All +figures on this page come from an RTX 4090 Laptop GPU (every path); CI runs the kernel checks on lavapipe (`plain`). | Stage | Result | |---|---| diff --git a/include/mossinfer/mossinfer.h b/include/mossinfer/mossinfer.h index d4f14ac..e4bbf03 100644 --- a/include/mossinfer/mossinfer.h +++ b/include/mossinfer/mossinfer.h @@ -74,8 +74,8 @@ extern "C" { /* ------------------------------------------------------------------------------------------------ version */ #define MI_VERSION_MAJOR 2 -#define MI_VERSION_MINOR 1 -#define MI_VERSION_PATCH 2 +#define MI_VERSION_MINOR 2 +#define MI_VERSION_PATCH 0 /* Incremented on every incompatible change of the binary interface (see the conventions above). */ #define MI_ABI_VERSION 1