Skip to content

Refresh datasets page from the republished 2.4 build - #511

Merged
sgosline merged 1 commit into
mainfrom
docs-2.4-v2-counts
Oct 14, 2026
Merged

sgosline merged 1 commit into
mainfrom
docs-2.4-v2-counts

Conversation

@jjacobson95

@jjacobson95 jjacobson95 commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Refreshes the Datasets Included page so its numbers reflect the republished 2.4 data (Figshare article 33990850, v2), and generates every table on the page from scripts/gen_dataset_stats.py.

Changes:

  • scripts/gen_dataset_stats.py: also writes dataset_summary_statistics.csv and dataset_curve_metrics_wide.csv. Fixes column parsing on rows with quoted fields, which miscounted samples for MPNST, Novartis and the cell-line datasets. Normalizes float sample IDs (3524.0) so Novartis experiments match its omics files.
  • docs/source/_static/: regenerated all three CSVs from the v2 build.
  • docs/source/datasets_included.rst: updated sample and drug counts in the Datasets and Modalities table and added delta_auc to the metric glossary.
  • docs/README.md: refresh instructions now cover all three CSVs.

NCI60 drugs increase from 23,267 to 55,143, reflecting the rebuilt drug files. MPNST samples change from 47 to 133 and Novartis from 348 to 386 due to the parsing fix.

Verified the docs build renders the updated tables.

Refs #337

@jjacobson95 jjacobson95 added this to the 2.4 Documentation milestone Oct 13, 2026
@jjacobson95 jjacobson95 added the documentation Improvements or additions to documentation label Oct 13, 2026
@jjacobson95 jjacobson95 self-assigned this Oct 13, 2026
@jjacobson95
jjacobson95 requested a review from sgosline October 13, 2026 20:19
Regenerate every number on the Datasets page from the build now served as
version 2 of Figshare article 33990850.

gen_dataset_stats.py now also writes dataset_summary_statistics.csv and
dataset_curve_metrics_wide.csv, which were previously produced by
notebooks/dataset_documentation.ipynb against hardcoded local paths and had
not been recomputed for 2.4. It also fixes two counting bugs:

- Lines with quoted fields (e.g. samples other_names "b1_4, b1_5, b1_6")
  were split on raw delimiters, shifting columns. This undercounted MPNST
  samples (47 vs 133) and Novartis (348 vs 386), overcounted the cell-line
  datasets by one, and garbled model_type values.
- Sample ids written as floats ("3524.0", novartis experiments) are
  normalized so they match the integer ids in the omics files.

Also add a delta_auc entry to the dose-response metric glossary and update
the refresh instructions in docs/README.md.
@sgosline
sgosline merged commit 432a205 into main Oct 14, 2026
2 checks passed

This branch was previously deployed

1 inactive deployment
github-pages — efefbe7f Deployed Oct 14, 2026 by jjacobson95 via deploy #31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants