Skip to content

[Perf #478] KernelRidge RBF fit: eliminate the ~9.7x training gap #487

Open

Description

@godofecht

Parent: #478

Canonical v2 Diabetes is the largest current hole: Flow fit 57.95 ms vs sklearn 5.998 ms. Score parity is verified, so this is a pure implementation/performance problem.

Profile RBF Gram construction, symmetry use, distance evaluation, solver choice, factorization, copies/layout and allocation. Determine whether sklearn is using a materially better LAPACK/BLAS path or whether Flow is performing redundant O(n^2)/O(n^3) work. Compare kernel and solve phases independently.

Acceptance: separate Gram/build/solve profiles; count kernel evaluations and matrix copies; use symmetry where legal; benchmark n=100..10k where feasible; retain verified R2 parity; fit >=0.95x first, target >=1.05x.

Activity

  1. godofecht commented on Aug 24, 2026

    @godofecht
    OwnerAuthor

    Found a concrete first-order cause of the KernelRidge RBF fit gap. KR_KERNEL_RBF / KERNEL_TYPE_RBF are both 2, and the canonical benchmark passes KERNEL_TYPE_RBF, but the specialized f64 branches in both kernel_ridge_fit and kernel_ridge_predict currently test kernel_type == 1. Canonical RBF therefore falls through to the generic compute_kernel_matrix path.

    On the canonical Diabetes split (n_train_samples = 353, 10 features), that generic builder evaluates all 124,609 ordered kernel pairs and creates 124,962 temporary row allocations (plus the same frees), then copies the f32 Gram matrix into f64 K_reg. Restoring the RBF branch and exploiting Gram symmetry reduces kernel evaluations to 62,128 strict-upper-triangle pairs, eliminates 624,810 feature-distance terms, eliminates the temporary row allocations, and removes the temporary f32 Gram/copy. RBF diagonal entries are exactly 1, so they can be set directly before adding alpha.

    Full derivation + executable Flow witness: godofecht/OpenProjects/ml/kernel-ridge-rbf-fast-path.md and .flow. The next benchmark should repair dispatch first, then profile T_gram vs T_chol before considering a LAPACK/solver rewrite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions