You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Perf #478] KernelRidge RBF fit: eliminate the ~9.7x training gap #487
Canonical v2 Diabetes is the largest current hole: Flow fit 57.95 ms vs sklearn 5.998 ms. Score parity is verified, so this is a pure implementation/performance problem.
Profile RBF Gram construction, symmetry use, distance evaluation, solver choice, factorization, copies/layout and allocation. Determine whether sklearn is using a materially better LAPACK/BLAS path or whether Flow is performing redundant O(n^2)/O(n^3) work. Compare kernel and solve phases independently.
Acceptance: separate Gram/build/solve profiles; count kernel evaluations and matrix copies; use symmetry where legal; benchmark n=100..10k where feasible; retain verified R2 parity; fit >=0.95x first, target >=1.05x.
Found a concrete first-order cause of the KernelRidge RBF fit gap. KR_KERNEL_RBF / KERNEL_TYPE_RBF are both 2, and the canonical benchmark passes KERNEL_TYPE_RBF, but the specialized f64 branches in both kernel_ridge_fit and kernel_ridge_predict currently test kernel_type == 1. Canonical RBF therefore falls through to the generic compute_kernel_matrix path.
On the canonical Diabetes split (n_train_samples = 353, 10 features), that generic builder evaluates all 124,609 ordered kernel pairs and creates 124,962 temporary row allocations (plus the same frees), then copies the f32 Gram matrix into f64 K_reg. Restoring the RBF branch and exploiting Gram symmetry reduces kernel evaluations to 62,128 strict-upper-triangle pairs, eliminates 624,810 feature-distance terms, eliminates the temporary row allocations, and removes the temporary f32 Gram/copy. RBF diagonal entries are exactly 1, so they can be set directly before adding alpha.
Full derivation + executable Flow witness: godofecht/OpenProjects/ml/kernel-ridge-rbf-fast-path.md and .flow. The next benchmark should repair dispatch first, then profile T_gram vs T_chol before considering a LAPACK/solver rewrite.
Activity
godofecht commentedon Aug 24, 2026
Found a concrete first-order cause of the KernelRidge RBF fit gap.
KR_KERNEL_RBF/KERNEL_TYPE_RBFare both2, and the canonical benchmark passesKERNEL_TYPE_RBF, but the specialized f64 branches in bothkernel_ridge_fitandkernel_ridge_predictcurrently testkernel_type == 1. Canonical RBF therefore falls through to the genericcompute_kernel_matrixpath.On the canonical Diabetes split (
n_train_samples = 353, 10 features), that generic builder evaluates all 124,609 ordered kernel pairs and creates 124,962 temporary row allocations (plus the same frees), then copies the f32 Gram matrix into f64K_reg. Restoring the RBF branch and exploiting Gram symmetry reduces kernel evaluations to 62,128 strict-upper-triangle pairs, eliminates 624,810 feature-distance terms, eliminates the temporary row allocations, and removes the temporary f32 Gram/copy. RBF diagonal entries are exactly 1, so they can be set directly before adding alpha.Full derivation + executable Flow witness:
godofecht/OpenProjects/ml/kernel-ridge-rbf-fast-path.mdand.flow. The next benchmark should repair dispatch first, then profileT_gramvsT_cholbefore considering a LAPACK/solver rewrite.