Skip to content

[Bench #478] Scale every canonical operation across samples, features, classes and model size #490

Open

Description

@godofecht

Parent: #478

The 19-row canonical suite is excellent for reproducibility but one fixed shape can confuse interpreter/setup overhead with algorithmic throughput. Every canonical operation needs a scaling surface.

Build deterministic scaled fixtures covering relevant axes: n_samples, n_features, n_classes/clusters, n_estimators/depth, support vectors/kernel matrix size and batch size. Use logarithmic sizes from tiny through realistically large, with explicit memory-feasibility caps for O(n^2)/O(n^3) estimators.

Acceptance: each canonical operation has at least three meaningful sizes; slopes/complexity regimes are machine-readable; Pages can show crossover points; a Flow win at tiny size is not considered complete if it becomes a material loss at larger supported sizes; all scaled losses automatically link into #478.

Activity

  1. godofecht commented on Aug 26, 2026

    @godofecht
    OwnerAuthor

    Worked out a concrete benchmark-grid rule for #490 and archived the derivation + Flow witness in godofecht/OpenProjects.

    For fixed endpoints n_min < n_max and L benchmark points, if r_i = n_{i+1}/n_i, then prod r_i = n_max/n_min. Therefore R = max r_i obeys

    R >= (n_max / n_min)^(1/(L-1)),

    with equality iff all adjacent ratios are equal. So geometric spacing is minimax-optimal for the worst unresolved multiplicative scale interval.

    This gives a clean default for the scaled suite: geometric base points over each feasible axis, then refine only observed status transitions at geometric midpoints. If the initial transition bracket has multiplicative width rho, after k refinements the width is rho^(1/2^k).

    Example: 7 points over a 64x range give worst-case factor-2 crossover localization immediately. A 5-point exact fixture [81, 324, 1296, 5184, 20736] attains the 4x minimax lower bound; an uneven grid over the same endpoints has an 8x hole.

    The note also records the limitation: speedup need not be monotone, so same-status intervals can still hide double crossovers; slope/curvature changes should be allowed to trigger extra refinement.

    Files:

    • ml/minimax-geometric-scaling-surfaces.md
    • ml/minimax-geometric-scaling-surfaces.flow

    No hosted CI/workflow was touched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions