From d951320c0872f32fe0ccf3fd12ebe2ee8a21f54c Mon Sep 17 00:00:00 2001 From: tannevaled Date: Sun, 8 Nov 2026 09:57:56 +0100 Subject: [PATCH] docs: float64 Exp on 16 cores is slowed by allocation, not the kernel The README listed float64 Exp below 256 Ki as 0.82x NumPy (16 cores). On one core it is 1.2-1.3x faster at every size. With GOMAXPROCS=16 the same serial code got slower than with 1 (89 vs 73 us at 16 Ki), as does Add (14 -> 23-26 us) while AddInto does not move: the per-result allocation costs more with more Ps. In a Workspace on 16 cores, Exp is 1.3-1.5x NumPy up to 16 Ki and 7-17x from 256 Ki (new BenchmarkExpWS). The README now says so; perf.md gets the table, and its note on the other architectures now reflects that the kernels live in go-simd/floats since v0.37.0, NEON included (unbenchmarked). Co-Authored-By: Claude Opus 5.5 --- README.md | 16 +++++++++------- benchmarks/bench_test.go | 17 +++++++++++++++++ docs/perf.md | 34 ++++++++++++++++++++++++++++++++-- 3 files changed, 58 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index fe9c058..c3da110 100644 --- a/README.md +++ b/README.md @@ -127,8 +127,9 @@ is wrong on subnormals (full tables in **[docs/perf.md](docs/perf.md)**): - **Faster:** whole-array reductions (`Sum`/`Mean`/`Max` at 4 Mi: 2.3–4.5×), - row reductions (`SumAxis(1)` 2.2×, `MaxAxis(1)` 1.7×), `Exp` from 256 Ki - elements (2.4–5.6×), the `*Into` forms of `Add`/`Mul`/`Sqrt` (faster or at + row reductions (`SumAxis(1)` 2.2×, `MaxAxis(1)` 1.7×), float64 `Exp` at + every size (on one core 1.2–1.3×; in a `Workspace` on 16 cores 1.3–1.5× + up to 16 Ki and 7–17× from 256 Ki), the `*Into` forms of `Add`/`Mul`/`Sqrt` (faster or at parity at every size), and `MatMul` against single-threaded OpenBLAS (3.7–5.9× from 512²); in a `Workspace`, `x + y` and `sqrt(x*y + x)` match or beat NumPy at every size measured; float64 `Log` on one core with AVX2 @@ -136,11 +137,12 @@ is wrong on subnormals - **At parity:** `SumAxis(0)`, `Inner` against 16-thread OpenBLAS. - **Slower:** `MatMul` against 16-thread OpenBLAS at 512² and up (0.42× at 512², 0.51× at 1024², measured in one run with OpenBLAS; at 256² ours is - 1.44× faster); `Dot` and mat·vec against NumPy's 16-thread BLAS (0.69× and 0.52× since v0.6.0's helper pool, measured in one run; 0.16× before); `Exp` - below 256 Ki elements (0.82×); the *allocating* forms of elementwise ops up - to 256 Ki elements, `Concatenate`/`Stack` and slice copies outside a - `Workspace` (0.2–0.5×: Go's allocator and garbage collector, which - `Workspace` largely removes). + 1.44× faster); `Dot` and mat·vec against NumPy's 16-thread BLAS (0.69× and 0.52× since v0.6.0's helper pool, measured in one run; 0.16× before); the + *allocating* forms of elementwise ops up to 256 Ki elements (allocating + float64 `Exp` with 16 Ps is at parity there, 0.9–1.0×, although its kernel + is faster), `Concatenate`/`Stack` and slice copies outside a `Workspace` + (0.2–0.5×: Go's allocator and garbage collector, whose cost grows with + GOMAXPROCS and which `Workspace` largely removes). On Apple silicon the GEMM reached parity with tuned BLAS at 1024² (OpenBLAS in an arm64 VM, single-threaded vecLib on an M4 Max), and every product beats the diff --git a/benchmarks/bench_test.go b/benchmarks/bench_test.go index a29ec7f..f0b4271 100644 --- a/benchmarks/bench_test.go +++ b/benchmarks/bench_test.go @@ -79,6 +79,23 @@ func BenchmarkExp(b *testing.B) { } } +// BenchmarkExpWS is BenchmarkExp in a Workspace reset every iteration: the +// kernel without the allocation of each result, whose cost grows with +// GOMAXPROCS (each garbage-collector cycle involves every P). +func BenchmarkExpWS(b *testing.B) { + for _, n := range elemSizes { + ws := nd.NewWorkspace() + x := ws.Use(vec(n)) + b.Run(fmt.Sprintf("n=%d", n), func(b *testing.B) { + b.SetBytes(int64(n * 8)) + for i := 0; i < b.N; i++ { + _ = x.Exp() + ws.Reset() + } + }) + } +} + func BenchmarkSqrt(b *testing.B) { for _, n := range elemSizes { x := vec(n) diff --git a/docs/perf.md b/docs/perf.md index 6673ee7..a327e8b 100644 --- a/docs/perf.md +++ b/docs/perf.md @@ -1634,8 +1634,38 @@ the constant block's layout is tested. −47% and −49% against v0.35.0 (`abbench`, 8 interleaved rounds, p = 0.000), both allocating, on `BenchmarkDTypeExpLog`'s data. -The float32 kernels for the other architectures are still to write; there -`Exp` and `Log` run the Go transcriptions. +Since v0.37.0 these kernels live in go-simd/floats (`floats.Exp`, `Log`, +`Float32Exp`, `Float32Log`, go-simd/floats#29), with NEON versions of the +float32 pair for arm64; ndarray calls them (`internal/kernels/explog.go`). +The NEON kernels are proven correctly rounded over all 2³² inputs on +darwin/arm64 but were not benchmarked: every arm64 host available was at +load 48–100. ppc64le, s390x, riscv64 and loong64 still run the Go +transcriptions. + +#### float64 `Exp` on 16 cores: the allocation, not the kernel (2026-11-08) + +The README listed float64 `Exp` below 256 Ki as slower than NumPy (0.82×), +measured with 16 cores. On one core it is faster at every size. With +GOMAXPROCS = 16 the same serial code (below `ParThreshold`, 32 Ki, nothing +is split) got slower than with 1: 89 µs against 73 at 16 Ki. The same holds +for `Add`, which goes from 14 to 23–26 µs, while `AddInto`, which allocates +nothing, costs 4.3 µs at both. Each result's allocation costs more with more +Ps: every garbage-collector cycle starts and stops work on all of them. In a +`Workspace` (`BenchmarkExpWS`) the difference disappears: + +| float64 `Exp`, Zen 3, load ≤ 5 | 1 Ki | 16 Ki | 256 Ki | 4 Mi | +|---|---:|---:|---:|---:| +| NumPy 2.5.4, `np.exp(a)` (one thread) | 6.0 µs | 85.5 µs | 1.32 ms | 25.0 ms | +| ours, 1 P, allocating | 4.9 µs | 73 µs | 1.11 ms | 19.3 ms | +| ours, 16 P, allocating | 5.8–6.6 µs | 83–95 µs | 0.63–0.78 ms | 4.8–6.7 ms | +| ours, 16 P, `Workspace` | **4.1 µs** | **64 µs** | **0.18 ms** | **1.47 ms** | + +`BenchmarkExp` and `BenchmarkExpWS` (`(i%97)·0.5`, the README's data), two +rounds each at `-cpu 1` and `-cpu 16` on cores 0–15; NumPy medians of nine +on one pinned core. In a `Workspace` on 16 cores, float64 `Exp` is 1.3–1.5× +NumPy up to 16 Ki and 7–17× from 256 Ki. Allocating, it is at parity up to +16 Ki, and at 4 Mi the allocation of each 32 MB result takes about 3.3 ms of +the 4.8. ## SIMD coverage