Repository navigation
sd: cluster keeps hclust's leaf order, and the variables when nothing is peeled - #184
Closed
gitRasheed wants to merge 1 commit into
Closed
gitRasheed wants to merge 1 commit into
gitRasheed wants to merge 1 commit into
Conversation
… is peeled (R sd-cluster-fixes)
Mirrors R NNS.SD.cluster (OVVO-Financial/NNS sd-cluster-fixes).
The dendrogram's "order" was overwritten with each label's original column
position, which is not a leaf order of the tree, so a plot or cut drawn
from it misplaced the leaves. It is now hclust's own order: the last merge
expanded left before right, as R's hcass2 does, read from the R-convention
merge matrix the function already returns.
When min_cluster was at or above the number of columns nothing was peeled,
and the final-cluster step also required more than min_cluster columns, so
the result had no clusters. Any variables left now form the final cluster.
Before -> after (= R main):
nns_sd_cluster(4 columns, min_cluster=4) {} -> {Cluster_1: [X_1, ..., X_4]}
nns_sd_cluster(cbind(A = 1:5, B = 2:6), min_cluster=2)
{} -> {Cluster_1: [A, B]}
dendrogram order of the 12-column test matrix (min_cluster=2)
original positions -> [1, 2, 3, 4, 9, 10, 11, 12, 5, 6, 7, 8]
3 cache entries predate the R change and were refilled from the cache's R
build plus that change alone: the two NNS.SD.cluster entries with
min_cluster 4 and 5 and the dendrogram entry whose order moved. The parity
test for the too-few-variables warning used min_cluster=4, which no longer
warns; it now uses one column, a new entry from the same build. The other
8 NNS.SD.cluster entries came back identical.
Contributor
|
Incorporated through NNS R 14.0 integration #192, merged as |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two fixes to R's
NNS.SD.clusterfrom OVVO-Financial/NNS#126 (merged). The returned dendrogram'sorderwas overwritten with each label's original column position, which is not a leaf order of the tree. And whenmin_clusterwas at or above the number of columns nothing was peeled, and the final-cluster step also needed more thanmin_clustercolumns, so the result had no clusters. This slice mirrors both.Stacked on #183; merge with a merge commit.
Changes
"order"is hclust's own leaf order: the last merge expanded left before right, as R's hcass2 does, read from the R-convention merge matrix_r_hclust_mergealready builds. On a 12-column test matrix this gives R main's order [1, 2, 3, 4, 9, 10, 11, 12, 5, 6, 7, 8].if activeinstead oflen(active) > min_cluster).nns_sd_cluster(4 columns, min_cluster=4){} -> {Cluster_1: [X_1, ..., X_4]},cbind(A = 1:5, B = 2:6)withmin_cluster=2{} -> {Cluster_1: [A, B]}, both as on R main. A single column still warns that there are too few variables for a dendrogram.plot.dendrogramfor a two-leaf tree becauseplot.hclustrejects it. Python draws with scipy, which already handles two leaves, so there is nothing to mirror.Tests
min_cluster2 and 3, one column with and withoutdendrogram) and R main's leaf order and labels on a deterministic 12-column matrix. R's first test builds its data withrnormunderset.seed(123), so I took R main's values on a deterministic matrix instead.min_cluster >= ncolnow expect the single final cluster. The parity test of the too-few-variables warning usedmin_cluster=4on four columns, which now clusters, so it uses one column.min_cluster4 and 5, and the NNS.SD.cluster.dendrogram entry whose order moved ([1, 2, 3, 4] now). I refilled them from an R build of the cache's base plus only Regenerate R parity cache for NNS 13.1 #126's R/SD_Cluster.R hunk, which also produced the new one-column entry. Themin_cluster=4dendrogram entry is no longer used and is gone; the other 8 NNS.SD.cluster entries came back identical. docs/parity.md records it.NNS_DISABLE_NATIVE=1: 2913 passed, 29 skipped, 2 xpassed. The 3 xfails are mode: break tied discrete modes by gravity, not their mean (mirrors R #111) #178's strict ones and the 2 xpasses the existing non-strict live-R VAR xfails. tests/parity/test_practical_examples.py against the usual R library: 9 passed, 2 xfailed, as on mode: break tied discrete modes by gravity, not their mean (mirrors R #111) #178. ruff check is clean on every changed file.Review
GPT-6.1 Sol reviewed the #135, #127 and #126 mirrors together: SHIP.
Mirrors OVVO-Financial/NNS#126