Skip to content

[Perf #478] KernelRidge RBF predict: eliminate ~7.7x inference gap with batched kernel evaluation #488

Open

Description

@godofecht

Parent: #478

Canonical v2 Diabetes: Flow predict 4.861 ms vs sklearn 0.628 ms. Unlike several other losing estimators, both training and inference are independently slow, so prediction needs its own fix.

Profile test-to-train RBF kernel construction, squared-distance computation, exponentials, alpha dot products, batching, vectorization and temporary matrices. Audit whether Flow materializes more data than required and whether the final kernel-vector product can use optimized dense kernels.

Acceptance: prediction-only profile; batch/sample/support-size curves; zero avoidable per-element allocation; parity remains verified; canonical predict >=0.95x first, target >=1.05x.

Activity

  1. godofecht commented on Aug 26, 2026

    @godofecht
    OwnerAuthor

    Concrete source-level finding for the RBF predict gap: KERNEL_TYPE_RBF is 2, but kernel_ridge_predict selects its specialized RBF branch with model.kernel_type == 1, so public RBF prediction falls into the generic path. That generic path allocates one query-row buffer per query and one training-row buffer per query/train pair, copies each row, then immediately reads it for kernel_compute.

    There is a clean first optimization that does not change current numerical semantics: eliminate matrix_row materialization and read matrix_at(X,i,f) / matrix_at(X_train,j,f) directly while preserving the existing f32 squared-distance accumulation, f64 exp conversion, and f32 dual accumulation order. For M queries, N training rows, F features this removes exactly M(N+1) temporary row allocations/frees and FM(N+1) copied f32 values while keeping the same MN RBF evaluations.

    For N=353, F=10 and an 89-query batch that is 31,506 temporary vectors and 315,060 copied floats = 1,260,240 bytes of explicit row-copy payload before allocator overhead.

    I archived the proof + Flow witness in godofecht/OpenProjects as ml/zero-allocation-kernel-ridge-rbf-predict.{md,flow}. Recommended A/B sequence: current copied-row path -> direct-index f32 path (must preserve predictions), then separately test the corrected == 2 f64 specialization because that second change also changes arithmetic precision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions