Skip to content

perf: docs/perf.md compares multicore Go with single-threaded NumPy 2.2.4; re-measure on one core against the pinned NumPy #93

Closed

Description

@tannevaled

docs/perf.md does not measure what the stack's rule requires: Go is measured multicore against single-threaded NumPy, against NumPy 2.2.4 rather than the pinned NumPy. Under the one-core rule (go-scikit/roadmap COVERAGE.md, "Rules for every slice"; docs/performance-campaign.md section 1), its ratios cannot be used as a verdict.

Witness (clone at a1745f5)

  • docs/perf.md:4: "against NumPy 2.2.4". Elsewhere in the stack, go-simd/floats and go-statistics/stats measure against NumPy 2.5.4, and COVERAGE.md names numpy 2.5.3.
  • docs/perf.md:9-18: "NumPy's elementwise and reduction ufuncs … are single-threaded SIMD C. So a pure-Go library wins them by combining two levers … 1. Multicore — fan the contiguous loop across GOMAXPROCS goroutines".
  • docs/perf.md:27-29: matmul is compared with "multi-threaded OpenBLAS".
  • docs/perf.md:34: the machine is "debian arm64 (Apple-silicon Tart VM), 4 vCPU".
  • The report is produced by shell and Python scripts (benchmarks/gen_report.sh, benchmarks/merge_report.py, benchmarks/bench_numpy.py).

To do

  • Re-measure every row at -cpu 1 / GOMAXPROCS=1 against the pinned NumPy, with the reference at one thread (OMP_NUM_THREADS, OPENBLAS_NUM_THREADS, MKL_NUM_THREADS, VECLIB_MAXIMUM_THREADS = 1). Use interleaved rounds, the 1-minute load below 8 and recorded, and the same statistic on both sides. The multicore column can stay as information only.
  • Measure the random draws for the first time. stats' docs/perf.md attributes 25 of its slower rows to this generator.
  • File every row that is still slower as a defect.
  • Planned home: wave W3 of the campaign, on the shared refbench harness (go-scikit/roadmap tracking issue).

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions