A lightweight C++ implementation of the Brain Floating Point (bfloat16) format.
bfloat16 is a 16-bit floating point format developed by Google Brain for use in machine learning applications. It preserves the dynamic range of 32-bit floating point (using the same 8-bit exponent) while reducing precision by storing only 7 bits of mantissa (compared to 23 bits in float32).
This implementation represents bfloat16 as:
- 1 sign bit
- 8 exponent bits (bias 127, same as float32)
- 7 mantissa bits
- C99 / C++17 compatible C API (
extern "C", usable from both languages) - Portable 16-bit storage (
bfloat16_t.bits) on every platform - Conversions between bfloat16, float32 and float64 with round-to-nearest-even
- Basic arithmetic (
add,subtract,multiply,divide,negate,abs, fusedfma, mixed-precisiondot) with IEEE 754 NaN / Inf / signed-zero handling - Full comparison set (
eq,ne,lt,le,gt,ge) plus classification (isnan,isinf,isfinite,iszero,issubnormal,signbit) - Exponent utilities (
get_exponent,set_exponent,ldexp,exp2) - Lightweight C++ operator overloads (
+ - * /, comparisons) - CMake build with CTest suite (134 C++ assertions, C99 API test, Python exact-fma and ml_dtypes differential tests, plus opt-in exhaustive tests)
include/bfloat16.h public API (C + C++ overloads)
src/bfloat16_convo.cpp float/double conversions
src/bfloat_arithmetics.cpp arithmetic (widen-compute-narrow)
src/bfloat_comp.cpp comparisons and classification
src/bfloat16_bias.cpp exponent utilities
bfloat_test.cpp C++ test suite (also serves as a usage example)
test_c.c C99 API test
test_fma_exact.py exact-rational fma differential test
test_ml_dtypes.py sampled ml_dtypes differential test
CMakeLists.txt build (static lib + tests + install rules)
cmake -S . -B build
cmake --build build
ctest --test-dir build --output-on-failureCTest runs the C++ suite, bfloat_test_c (a C99 translation unit compiled
with -std=c99 -Wall -Wextra), an exact-rational fma differential test, and,
when Python's ml_dtypes is installed, a sampled arithmetic differential
test against that independent implementation.
The full exhaustive suite checks all 2^32 float inputs and all 2^32 bf16
operand pairs for add/subtract/multiply/divide against GCC's independent
__bf16 arithmetic. It is opt-in because it takes a few minutes:
cmake -S . -B build -DBFLOAT16_EXHAUSTIVE_TESTS=ON
cmake --build build
ctest --test-dir build -L exhaustive --output-on-failureOr without CMake:
g++ -Iinclude -Wall -Wextra -std=c++17 bfloat_test.cpp src/*.cpp -o bfloat_test -lm
./bfloat_test # quick suite
./bfloat_test --exhaustive # exhaustive suiteTo use the library in your own project, add include/ to your include path
and compile the four src/*.cpp files (or link the installed libbfloat16.a).
- Exhaustive
float_to_bfloat16against GCC__bf16over all 2^32 float bit patterns. - Exhaustive add, subtract, multiply, and divide against GCC
__bf16over all 2^32 bf16 operand pairs for each operation. - Fused multiply-add against an exact rational reference, including every
addend for targeted tie cases such as
fma(15, 23, 2^-60). - A deterministic sample cross-checked against Python
ml_dtypes. - The public header compiled and linked as strict C99.
#include "bfloat16.h"
bfloat16_t a = float_to_bfloat16(1.5f);
bfloat16_t b = float_to_bfloat16(2.0f);
float sum = bfloat16_to_float(bfloat16_add(a, b)); /* 3.5f */#include "bfloat16.h"
bfloat16_t a = float_to_bfloat16(1.5f);
bfloat16_t b = float_to_bfloat16(2.0f);
bfloat16_t c = a + b; // C++ overloads
bool less = a < b;- Rounding: every narrowing conversion uses round-to-nearest-even.
double -> bfloat16rounds directly from the 64-bit pattern (no intermediatefloatstep), so there is no double-rounding error. - Arithmetic: add, subtract, multiply, and divide widen exactly to
doubleand then round once to bfloat16. They are correctly rounded: the 53-bit intermediate is far wider than the 2p+2 = 18 bits required to make double rounding harmless for a p = 8 target.fmaalso roundsa*b+ccorrectly, using round-to-odd on the double result to preserve the residual when the exact value lies on a bf16 midpoint. Inf / NaN / signed-zero semantics are inherited from IEEE 754 arithmetic; overflow produces infinity, as in IEEE round-to-nearest. dot: each bf16 pair is multiplied exactly indouble, then the products are accumulated left-to-right (i = 0..n-1) in a singledoubleaccumulator. Every addition rounds todouble; terms are not reordered or reduced pairwise.-ffast-math: do not use-ffast-mathor-ffinite-math-onlywhen compiling this library. They let the compiler assume NaN and infinity cannot occur, which breaks the documented IEEE 754 behavior.- Platform caveats: FTZ/DAZ modes flush bf16 subnormals to zero in the widening arithmetic and break the documented subnormal behavior. On 32-bit x86 without SSE2, x87 extended-precision evaluation can reintroduce double rounding; x86-64 and ARM64 are unaffected.
- NaN handling: conversions preserve NaN payload top bits and force the quiet bit, so a NaN can never collapse into zero or infinity by truncation.
set_exponentvsldexp:set_exponentis raw bit manipulation (keeps sign + mantissa, leaves NaN/Inf/zero unchanged, truncates on the way into the denormal range);ldexp/exp2are true correctly-rounded arithmetic.- Native types: storage is always the portable struct, even where the
compiler offers
__bf16.BFLOAT16_HAS_NATIVEreports its presence for informational purposes only.
Thanks to tansy for the first pull request and for pointing out the header-only claim, which prompted a cleanup of the build and documentation.
This project is licensed under the MIT License - see the LICENSE file for details.
- Google Brain bfloat16 specification
- TensorFlow bfloat16 implementation
- NVIDIA's bfloat16 implementation
Contributions are welcome! Please feel free to submit a Pull Request.