Skip to content

Match DataFrame columns by name and honour declared levels - #155

Closed
gitRasheed wants to merge 12 commits into
rasheed/mreg-performancefrom
rasheed/pandas-names-levels
Closed

gitRasheed wants to merge 12 commits into
rasheed/mreg-performancefrom
rasheed/pandas-names-levels

Conversation

@gitRasheed

@gitRasheed gitRasheed commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

nns_reg, nns_m_reg, nns_stack and nns_boost turned every input into a NumPy array first. That dropped pandas column names and categorical dtypes, so point_est and IVs.test were matched by position and declared level orders were lost, with no warning. On mtcars with point_est columns swapped, nns_reg returned 10.4 and 10.4 mpg where R returns 21.0 and 21.2. We keep the frames until encoding and follow R's matching and level rules.

Stacked on #154; merge with a merge commit.

Changes

  • Test columns are matched as R's .nns_match_predictor_names and .as_test_frame do. The same names are reordered, disjoint names or unnamed arrays are positional, and a partial overlap raises ValueError. nns_stack trained on [wt, hp, disp] and tested on [disp, hp, wt] returned 2714.95 and 2095.4 mpg for the first rows; it now realigns by name.
  • A Categorical, class_levels or factor_levels sets the level order, as R's factor does. Both arguments were deleted unread before. Undeclared labels are sorted, which is R's factor() order. On iris shuffled so virginica comes first, nns_stack out-of-fold accuracy goes from 0.837 to 0.985, equal to R's factor path.
  • Object columns of numbers stay numeric (Boston lstat ~ medv reported "R2" 0.0336, an accuracy), numeric-looking strings are factor levels, and missing labels raise "contains missing values" as in R instead of becoming a level.
  • nns_m_reg follows NNS.M.reg: factor_2_dummy=True by default and working, string columns accepted, DataFrame names kept, and class_levels classifies instead of silently turning into a regression.
  • NNS.dim.red.variables and the dim.red coefficients carry predictor names instead of 0-based indices, and nns_boost keys feature.weights/feature.frequency by column name (lstat, crim, rm on Boston) instead of x1..xp. nns_boost gains factor_levels=.
  • When ALL wins Method 1 with a single predictor, NNS.reg.n.best is NaN, as R's NA, instead of n.
  • The nns_reg default-order copula target uses R's colMeans, so on R's sin grid the copula term is R's value exactly.
  • import nns; nns.nns_reg takes 0.14 s instead of 0.47 s, because scipy.interpolate loads only for smooth=True.
  • conventions.md, api_status.md, api_reference.md and the result TypedDicts describe the current engine instead of the removed legacy one.

Purely numeric inputs give bit-identical results. One divergence stays on purpose: R 13.3's NNS.stack codes character columns by first appearance, so its results depend on row order. The port sorts.

Tests

New tests/parity/test_labels.py with a typed data.frame R helper, covering factor x and y, numeric strings, point names, NNS.M.reg on raw character frames, stack names and levels, boost names, the NaN n.best case and prepare_factor_predictors. Two boost parity tests now build object arrays so their numeric column stays numeric. The parity cache encodes unpatched R 13.3.

The Boston factor-path practical example is marked xfail(strict=False). With chas as a real two-level factor, an ulp-level difference in the fold X* products moves one partition (n.best 7 against R's 5). #161 fixes those numerics and drops the mark.

Fixes #146

Update

This push makes nns_stack_fit keep its own copies of the training data and keyword arguments. Before, adding 100 to y after fitting moved every prediction by 100. Callables such as obj_fn pass through uncopied, so an objective that holds a lock or counts its calls still works.

Review

GPT-6.1 Sol reviewed this slice and raised two findings. We took both.

  • Decimal predictors became categorical, which broke predictions that worked before. Decimal counts as numeric again ([5.], as before), and numeric-looking strings stay categorical, with a test for each.
  • tests/parity/test_labels.py imported pandas unconditionally, but CI only installs pandas from Re-vendor NNS-core, bind its M.reg kernels and honour ncores #157. The file now skips without pandas; it was the only test file in these slices with an unconditional pandas import.

GPT-6.1 Sol reviewed the round-3 fixes. Taken: deep-copying kwargs also copied callables, so an obj_fn holding a lock raised TypeError and a stateful objective lost its calls; callables now pass through uncopied, with a test. A second Sol round found nothing.

The dependence that sets nns_reg's default order and smoothing level, and the
multivariate copula, used np.mean for the copula target. np.mean's pairwise sum
can differ from R's colMeans in the last ulp, which flips the copula's exact
equality shortcut. With _r_col_means the copula component on R's own sin grid
(seq(0, 4*pi, pi/100)) is now R's value exactly (0.46202136490371937); the
remaining difference there comes from nns_dep's cell means (native kernel).
nns_reg ran every input through np.asarray, so pandas column names and
categorical dtypes were lost before encoding:

- point_est columns were matched by position. mtcars x = [wt, hp] with
  point_est given as [hp, wt] returned 10.4, 10.4 instead of R's 21.0,
  21.2, with no warning. Columns are now matched as R's
  .nns_match_predictor_names does: the same names are reordered, names
  disjoint from the training names (or unnamed arrays) are positional,
  and a partial overlap raises ValueError. A named row (df.iloc[i]) is
  matched the same way.
- A pandas Categorical response or predictor was coded in sorted order.
  It now uses the declared categories, as R does for a factor (iris with
  levels virginica/versicolor/setosa: Point.est 3 2 1 and class.levels in
  that order, equal to R).
- class_levels and factor_levels were deleted unread. class_levels now
  declares the response levels (codes 1..K in that order) and implies
  classification; factor_levels declares predictor levels (a mapping by
  column name, one entry per column, or a flat list for one predictor).
  Integer codes 1..K together with class_levels/factor_levels label
  names are kept as codes, so the existing "codes plus labels" calls
  still work.
- An object-dtype numeric y (df.to_numpy() of a mixed frame) was treated
  as a classification target with string-sorted classes (Boston lstat ~
  medv: "R2" 0.0336, the accuracy). Object columns are now numeric when
  every element is a number; this applies to y and to predictors.
- Numeric-looking strings ("1", "10") in a string/object column were
  parsed as numbers; they are factor levels, as in R (R2 0.646 with
  dummies g_X1, g_X10, g_X2 like R instead of 0.990 as a number).
- None/NaN in a categorical predictor became a level "nan" (RPM column
  g_nan). It now raises "Predictor 'g' contains missing values." like R;
  missing labels in y or point_est raise too.

Undeclared string predictors keep sorted levels. R uses first-appearance
order for character predictors, so its fit depends on the row order (R2
0.9082 vs 0.9200 after shuffling mtcars); sorted is R's factor() order.
With the default factor_2_dummy=True this only permutes the dummy
columns.

The column handling lives in a new nns/_labels.py so nns_m_reg,
nns_stack and nns_boost can share it. Purely numeric inputs give
bit-identical results (A/B of univariate, multivariate, dim-red,
classification, CI and VAR fits against the previous commit); timings are
unchanged (multivariate Boston fit 0.22 s both).

Public results change only for the inputs above. Two boost parity tests
built their mixed factor/numeric matrix with np.column_stack of a string
and a float array, which makes every column a string; they now build an
object array so the numeric column stays numeric.

New parity tests (tests/parity/test_labels.py, with a generic typed
data.frame R helper in tests/_r.py) compare against NNS 13.3.
nns_m_reg diverged from R's NNS.M.reg API:

- The default was factor_2_dummy=False; R's is TRUE.
- factor_2_dummy=True raised NotImplementedError claiming R's raw factor
  path errors. In R 13.3 it works:
  NNS:::NNS.M.reg(data.frame(g = c("a","b","c",...), z), y) encodes g
  into g_a, g_b, g_c itself.
- x went through np.asarray(dtype=float64), so any string column crashed
  with "could not convert string to float", and DataFrame column names
  were replaced by V1..Vp (nns_reg kept them).
- class_levels was applied by recoding y to 1..K floats without setting
  type, so nns_m_reg(X, species, class_levels=[...]) silently became a
  regression (R2 0.987, y.hat 1.045...) with class.levels None, while the
  same call without class_levels classified (accuracy 1.0).

nns_m_reg now passes x, y, point_est, class_levels and the new
factor_levels straight to nns_reg_engine, which encodes predictors and
the response exactly as nns_reg does (R's .nns_reg_encode_predictors and
.nns_reg_type, which NNS.M.reg also calls). A new multivariate_only flag
keeps R's "NNS.M.reg requires at least two encoded predictors." check, so
a single factor column with several levels is accepted as in R. The dead
_validate_inputs helper holding a second copy of the stale guard is
removed.

Numeric matrices give the same results as before (the engine call is
unchanged for them; RPM/Fitted.xy keys stay V1..Vp for unnamed input).
Results change for DataFrames (named keys), string or factor columns
(now accepted) and class_levels (now classification with class.levels set;
integer codes 1..K plus their labels still work). The direct
factor_2_dummy=True invariant test and the deferred-guard registry are
updated: no production NotImplementedError guard remains. Parity with
NNS:::NNS.M.reg on a raw character frame and on a factor response is in
tests/parity/test_labels.py.
nns_stack flattened its inputs with np.asarray and deleted class_levels
and factor_levels unread:

- IVs.test columns were matched by position. mtcars trained on
  [wt, hp, disp] and tested with [disp, hp, wt] returned 2714.95,
  2095.4, ... mpg where R realigns by name. Test columns are now matched
  like R's .as_test_frame (same names reordered, disjoint names
  positional, partial overlap an error), including a named row.
- NNS.dim.red.variables were 0-based indices and
  NNS.dim.red.coefficients a bare array. They are now R's names: the
  predictor names (frame columns, V1..Vp for an unnamed matrix, X1 for a
  vector) and a dict from encoded column name to coefficient (factor
  dummies named <column>__<make.names(level)> as in R).
- A string response was coded by first appearance and a pandas
  Categorical lost its categories. On iris shuffled so virginica comes
  first this gave the codes virginica/setosa/versicolor, a non-ordinal
  map, and OOF accuracy 0.837 where R's factor path reaches 0.985. The
  response is now coded in declared order for a Categorical or
  class_levels (unused levels dropped, as R's droplevels), sorted order
  for strings (R's factor order and NNS.boost's), False < True for bool,
  sorted values for numbers. An object array of numbers is numeric.
  The shuffled-iris case now gives OOF accuracy 0.985/0.985 and test
  predictions identical to R's factor path, for string and for
  Categorical y.
- Factor predictors: "0"/"1" strings were parsed as a number (13 columns
  for Boston where R's factor chas gives 14), factor_levels was ignored,
  and levels were first-appearance and re-derived per fold, so a level
  absent from a fold's training rows dropped a column or raised "unseen
  level". Categorical columns and factor_levels now give factors in
  declared order, other label columns sorted levels, and every fold uses
  the full training levels (R keeps a factor's levels when subsetting),
  one dummy per level.
- Missing labels in predictors or the response raise "contains missing
  values" as in R instead of becoming a level "None".

Divergence kept on purpose: R's NNS.stack codes a character response and
character predictors by first appearance, so its result depends on the
row order (an R bug); the port uses sorted order.

nns_stack_fit now stores the training inputs as given, so names and
categories still apply in nns_stack_predict.

Purely numeric inputs: stack, reg, dim.red, weights and thresholds are
bit-identical to the previous commit (Boston, iris, ts_test, balance,
binary cases); only the two selection keys changed type. Timing is
unchanged (Boston 400x13 stack 0.35 s vs 0.34 s).

The Boston "factor path" practical example passed before only because
chas, given as "0"/"1" strings, was silently numeric. As a real factor
(two dummy columns, like R) the extra column changes the last ulp of the
Method-1 fold X* products (numpy @ vs R's column-sequential dgemv, plus
R's long-double cor), which moves one NNS partition: OBJfn.reg 5.2015 vs
R 5.2124 and n.best 7 vs 5 (stack RMSE 3.822 vs 3.739). That needs a
separate numerics fix, so the test is marked xfail(strict=False)
with that reason. Dimension reduction already matches R exactly there
(threshold 0.6589, dim.red RMSE 3.8613 on both sides).
nns_boost discarded pandas column names and categorical dtypes:

- feature.weights / feature.frequency were keyed x1..xp. R keys them by
  predictor name (lstat, crim, rm on Boston; V1..Vp for an unnamed
  matrix). They now carry R's names: frame column names, V1..Vp for an
  unnamed matrix.
- IVs.test columns were matched by position; they are now matched by
  name as in R (shared with nns_stack).
- A pandas Categorical predictor or response lost its level order, an
  object array of numbers as y became a classification target, and
  there was no way to declare predictor levels although the docs
  described factor_levels. Inputs are now prepared by nns_stack's
  _prepare_inputs: categorical / factor_levels columns keep their
  declared order with unused levels dropped (R's droplevels in
  Boost.R), other label columns use sorted levels (as Boost.R does),
  and the response follows R's boost coding (declared order,
  class_levels, or sorted labels). Factor columns are passed to every
  learner nns_reg fit and to the final nns_stack as level-coded columns
  with factor_levels, so they stay one-hot factors throughout.
- factor_levels= is added (same forms as nns_reg / nns_stack).

Purely numeric inputs give identical results, thresholds and epoch
draws (A/B on mtcars, iris and Boston subsets against the previous
commit; only the feature-name keys change from x<i> to V<i>). Timing
unchanged (Boston 300x6, 10 learner trials, 20 epochs: 1.85 s before,
1.78 s after).

The two factor-predictor boost parity tests now pass the declared
levels (low, mid, high) that the R reference uses. A new parity test
checks feature names, frequencies and results against NNS.boost on a
named data.frame.
When the all-points candidate won Method 1 with a single (encoded)
predictor, the port reported NNS.reg.n.best = n (160 on the audit's
pure-noise case), a number R never returns. R's Stack.R sets
reg_best_k <- NA when "all" wins and only the multivariate branch
replaces it with the RPM row count.

The univariate branch now leaves the value missing, so NNS.reg.n.best is
NaN as R's NA; the multivariate branch is unchanged (RPM row count, equal
to R). Only that reported scalar changes: the fit still uses
n.best = "all" and reg/stack predictions are unchanged. A parity test on
the audit's noise case (RandomState(5), 160 training rows) checks NA and
the reg predictions against R.
The docs and docstrings for nns_reg, nns_m_reg, nns_stack and nns_boost
described an earlier design:

- conventions.md said nns_reg maps to factor.2.dummy = FALSE (the
  default is TRUE); intervals use UPM.VaR/LPM.VaR and below-range
  pred.int rows are dropped (they are type-8 residual quantiles, one row
  per point); "mode_class" is accepted (it raises); direct
  nns_m_reg(factor_2_dummy=True) is rejected because R errors (R 13.3
  works, and so does the port now); stack uses NumPy's RNG and a
  deterministic cv_size=0.25 (it reproduces R's Mersenne-Twister stream
  and R's 5-fold default); ts_test trains on the tail rows (folds are
  rolling chronological); raw string labels are rejected and
  class_levels/factor_levels reproduce R ordering (strings were
  accepted and both arguments were ignored until the previous commits);
  boost keeps an n_features > 10 NotImplementedError guard (gone) and
  positional X1.. names.
- api_status.md / api_reference.md listed the removed guards, "exact
  stochastic sample parity is not expected", and 0-based
  NNS.dim.red.variables.
- The stack module docstring claimed NumPy-RNG distributional parity.
- RegFitted promised standard.errors, RegResult an "empty" Point.est and
  no class.levels; MRegPredInt used lower/upper.pred.int keys (the
  result uses pred.int.neg/pos) and MRegResult claimed point_only drops
  keys (all keys are present, with None values). nns_reg's public
  docstring documented no parameters.

These sections now describe the code as it is: R's data-frame input
rules (name matching, factor/character typing, declared vs sorted level
order and where that deliberately differs from R's first-appearance
order), class_levels / factor_levels, R-stream randomness, split rules,
named selection outputs and the NaN n.best case. The TypedDicts match the
returned keys (standard.errors stays NotRequired for the unused legacy
implementation, whose interval helper annotation is widened accordingly)
and nns_reg gets a parameter/return docstring. The smoothing paragraph is
left as is (separate smoothing fix). No code behaviour changes.
prepare_factor_predictors is public and documented but no test ever ran it.
Check its design (one dummy per level, as R's factor_2_dummy_FR) and that
nns_m_reg on the design and nns_reg on the categorical frame both reproduce
R's NNS.reg point estimates on a factor + numeric data frame.
Every other test imports `_r` and `_tolerances` through pytest's
pythonpath. `tests._r` only resolves when the repository root is on
sys.path, so `uv run pytest` failed to collect this file, and under
`python -m pytest` it loaded a second copy of the module with its own
cache.
Decimal is not a numbers.Real, so a Decimal column became a factor and
point_est=[2.5] raised "unseen level(s): 2.5"; it now predicts [5.] as before.
The CI test jobs at this point do not install pandas, so the module-level
import was a collection error; without pandas the file now reports 1 skipped.
They point at notes that don't live in this repo.
@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Incorporated through NNS R 14.0 integration #192, merged as bd19c3303222af46da79d8c97f258248eb5674d2. Verified exact head f1c17a301c27624d35998249ab9ee0adf213498e is an ancestor of main; original commits and authorship are preserved. The integration includes the subsequent R14 compatibility corrections and passed the complete GitHub Python 3.11-3.14 matrix. This supersedes the separate landing, not the work or review history.

@github-actions github-actions Bot closed this Nov 7, 2026
@github-actions
github-actions Bot deleted the rasheed/pandas-names-levels branch November 7, 2026 18:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant