You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Parity #478] Track dtype, memory layout, preprocessing and conversion work explicitly #502
Flow/sklearn timings must expose representation differences rather than accidentally rewarding one side for cheaper inputs. Record source dtype, estimator internal dtype, row/column-major layout, contiguity/alignment, preprocessing/scaling, label encoding and every timed/untimed conversion or copy.
Where sklearn internally promotes or copies, distinguish API-realistic end-to-end timing from kernel-only equivalent-representation timing. Where Flow uses f32 and sklearn f64, make that a visible semantic/performance dimension with numerical-error evidence.
Acceptance: canonical and scaled artifacts contain representation/preprocessing metadata; hidden untimed conversion bytes = 0; equivalent-representation microbenchmarks exist for disputed gaps; Pages can distinguish end-to-end API wins from dtype/layout advantages.
I worked through the representation-parity part of this issue and archived the full derivation + Flow experiment in OpenProjects as ml/representation-normalized-benchmark-certificates.md and .flow.
The key result is a non-identifiability theorem: if two implementations enter their timed kernels with different representations, end-to-end timing alone cannot identify which numerical kernel is faster. Example: K_A=12, K_B=10, but conversion costs C_A=0, C_B=5 give T_A=12 < 15=T_B; the product winner has the slower kernel.
I therefore split eligibility into two certificates. API/product timing is valid when semantic parity holds and hidden untimed conversion bytes are zero: E_product = S && H==0. Kernel/implementation timing additionally requires equal kernel-entry representation signatures: E_kernel = S && H==0 && (R_A == R_B), with R=(dtype, layout, alignment, endianness, preprocessing, shape).
For dense materializing conversion, there is also an exact traffic lower bound: B_min >= N*(b_source+b_target). On Digits (1797*64=115008 elements), f64→f32 requires at least 1,380,096 bytes of one-pass read/write traffic. This should be emitted in a conversion ledger rather than disappearing into a benchmark boundary.
Concrete direction: every canonical operation should record public-source and kernel-entry representation signatures plus (copies, bytes_read, bytes_written, hidden_untimed_bytes). Keep API-realistic rows with all required conversion work timed, and add equivalent-representation microbenchmarks where kernel attribution is needed. A published row should never have hidden untimed conversion bytes > 0.
Activity
godofecht commentedon Aug 25, 2026
I worked through the representation-parity part of this issue and archived the full derivation + Flow experiment in OpenProjects as
ml/representation-normalized-benchmark-certificates.mdand.flow.The key result is a non-identifiability theorem: if two implementations enter their timed kernels with different representations, end-to-end timing alone cannot identify which numerical kernel is faster. Example:
K_A=12,K_B=10, but conversion costsC_A=0,C_B=5giveT_A=12 < 15=T_B; the product winner has the slower kernel.I therefore split eligibility into two certificates. API/product timing is valid when semantic parity holds and hidden untimed conversion bytes are zero:
E_product = S && H==0. Kernel/implementation timing additionally requires equal kernel-entry representation signatures:E_kernel = S && H==0 && (R_A == R_B), withR=(dtype, layout, alignment, endianness, preprocessing, shape).For dense materializing conversion, there is also an exact traffic lower bound:
B_min >= N*(b_source+b_target). On Digits (1797*64=115008elements), f64→f32 requires at least1,380,096bytes of one-pass read/write traffic. This should be emitted in a conversion ledger rather than disappearing into a benchmark boundary.Concrete direction: every canonical operation should record public-source and kernel-entry representation signatures plus
(copies, bytes_read, bytes_written, hidden_untimed_bytes). Keep API-realistic rows with all required conversion work timed, and add equivalent-representation microbenchmarks where kernel attribution is needed. A published row should never have hidden untimed conversion bytes > 0.No hosted CI/workflows were touched.