You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
pareto_front() returns rows that another row dominates when scores tie on one objective #77
SearchResult.pareto_front() (and therefore get_best_result() and Search.best_result for any multi-objective search) keeps a row that ties the best row on one objective and is worse on another.
SearchResult.calculate_pareto_front keeps every row for which np.any(scores <= point) (MIN) or np.any(scores >= point) (MAX). A row is therefore dropped only when it is strictly worse than the current point on every objective. A row that is equal on one objective and worse on another survives, although the point dominates it.
The unit tests already state the intended definition, in the class docstring of TestCalculatePareteFrontMin:
For minimization: point A dominates point B if A is <= B on all objectives and strictly < on at least one.
Under that definition (1, 1) dominates (1, 5), but the same test then asserts the opposite (assert mask[2] is np.True_ # (1,5) is not dominated by (1,1) — equal on dim 0). The MAX test does the same for (5, 5) and (5, 1).
Why this matters for compiler-flag searches: one objective is often a count, such as spill bytes or registers, and many flag sets give the same value (often 0). With objectives (kernel time, spill bytes), every zero-spill row ends up on the returned "front", not just the fastest one. Measured on main (bf21d9e), checked against a brute-force dominance check:
Search(...).start() with 12 x 3 candidates and the objective (time_us rounded to 0.1, spill_bytes): in four runs pareto_front() returned 14, 12, 13 and 15 rows, while a brute-force check gave 1, 1, 1 and 2. Every extra row is a zero-spill row that is slower than the best one. (The engine was the repo's pure-Python scripted test engine, since the native engine is not available on macOS; the result does not depend on the engine because the ranking runs on the result table.)
Integer scores with ties (2–3 objectives, MIN and MAX, 1,000 cases): the output matched brute force in 24 of 1,000 cases.
The compileiq-validate-result skill re-measures every front candidate at 100 or more trials, so each dominated row costs a full validation run.
Fix: keep a row only if it is strictly better than the current point on some objective, or exactly equal to it on all objectives. That keeps the existing behavior that identical rows all stay on the front (test_identical_points_both_on_front). I have a patch with tests and can open a PR.
Activity