You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Perf #478] KernelRidge RBF predict: eliminate ~7.7x inference gap with batched kernel evaluation #488
Canonical v2 Diabetes: Flow predict 4.861 ms vs sklearn 0.628 ms. Unlike several other losing estimators, both training and inference are independently slow, so prediction needs its own fix.
Profile test-to-train RBF kernel construction, squared-distance computation, exponentials, alpha dot products, batching, vectorization and temporary matrices. Audit whether Flow materializes more data than required and whether the final kernel-vector product can use optimized dense kernels.
Concrete source-level finding for the RBF predict gap: KERNEL_TYPE_RBF is 2, but kernel_ridge_predict selects its specialized RBF branch with model.kernel_type == 1, so public RBF prediction falls into the generic path. That generic path allocates one query-row buffer per query and one training-row buffer per query/train pair, copies each row, then immediately reads it for kernel_compute.
There is a clean first optimization that does not change current numerical semantics: eliminate matrix_row materialization and read matrix_at(X,i,f) / matrix_at(X_train,j,f) directly while preserving the existing f32 squared-distance accumulation, f64 exp conversion, and f32 dual accumulation order. For M queries, N training rows, F features this removes exactly M(N+1) temporary row allocations/frees and FM(N+1) copied f32 values while keeping the same MN RBF evaluations.
For N=353, F=10 and an 89-query batch that is 31,506 temporary vectors and 315,060 copied floats = 1,260,240 bytes of explicit row-copy payload before allocator overhead.
I archived the proof + Flow witness in godofecht/OpenProjects as ml/zero-allocation-kernel-ridge-rbf-predict.{md,flow}. Recommended A/B sequence: current copied-row path -> direct-index f32 path (must preserve predictions), then separately test the corrected == 2 f64 specialization because that second change also changes arithmetic precision.
Activity
godofecht commentedon Aug 26, 2026
Concrete source-level finding for the RBF predict gap:
KERNEL_TYPE_RBFis2, butkernel_ridge_predictselects its specialized RBF branch withmodel.kernel_type == 1, so public RBF prediction falls into the generic path. That generic path allocates one query-row buffer per query and one training-row buffer per query/train pair, copies each row, then immediately reads it forkernel_compute.There is a clean first optimization that does not change current numerical semantics: eliminate
matrix_rowmaterialization and readmatrix_at(X,i,f)/matrix_at(X_train,j,f)directly while preserving the existing f32 squared-distance accumulation, f64expconversion, and f32 dual accumulation order. For M queries, N training rows, F features this removes exactlyM(N+1)temporary row allocations/frees andFM(N+1)copied f32 values while keeping the same MN RBF evaluations.For N=353, F=10 and an 89-query batch that is 31,506 temporary vectors and 315,060 copied floats = 1,260,240 bytes of explicit row-copy payload before allocator overhead.
I archived the proof + Flow witness in
godofecht/OpenProjectsasml/zero-allocation-kernel-ridge-rbf-predict.{md,flow}. Recommended A/B sequence: current copied-row path -> direct-index f32 path (must preserve predictions), then separately test the corrected== 2f64 specialization because that second change also changes arithmetic precision.