Skip to content

Food.txt and CellLine.txt are never loaded: the compendium list in plugins/nodenorm/static.py is hard-coded #40

Closed

Description

@gaurav

NodeNorm ES cannot normalize any identifier whose clique lives in Food.txt or CellLine.txt, because the loader's list of compendia is hard-coded and does not include them.

What we see

On nodenorm-es.ci.transltr.io (Babel 2026jul22, NodeNorm ES 1.0.0), every one of these returns null; on the Redis-backed nodenormalization-sri.renci.org (the same Babel release, NodeNorm 2.5.1) they all resolve:

CURIE Redis NodeNorm ES NodeNorm
DRUGBANK:DB10501 "Apple" biolink:Food clique, 3 identifiers null
DRUGBANK:DB10542 "Goat milk" biolink:Food null
DRUGBANK:DB11158 "Pectin" biolink:Food null
UNII:14C97E680P "Wheat germ oil" biolink:Food null
CLO:0003684 "HeLa cell" biolink:CellLine null
CLO:0003685 "HeLa 229 cell" biolink:CellLine null

Sampling more broadly through NameRes: 166 of 166 DrugBank/UNII/CHEBI-led biolink:Food cliques and 103 of 103 CLO-led biolink:CellLine cliques we tried are absent from ES. The UMLS-led "Food" cliques (e.g. UMLS:C0016452) are present, but only because those live in umls.txt, which is loaded.

Why

plugins/nodenorm/static.py (identical in biothings/pending.api and biothings/annotation-hub) enumerates the compendia to download and upload by hand: NODENORM_FILE_COLLECTION, NODENORM_BIG_FILE_COLLECTION and NODENORM_UPLOAD_CHUNKS. Those lists hold 23 files. The 2026jul22 build at https://stars.renci.org/var/babel_outputs/2026jul22/compendia/ has 25:

  • Food.txt (89 KB, 285 cliques) is new in Babel 2026jul22 — a new compendium from the DrugBank retype, see the release notes.
  • CellLine.txt (6.5 MB) has existed since at least 2025sep1 — the plugin README's own directory listing shows it — but has never been in the list, so cell lines have never been normalizable through ES.

Nothing notices: the dumper downloads what the list says, the uploader uploads what it downloaded, and an identifier that is in no document is reported as unnormalizable, which is indistinguishable from a bad CURIE.

Impact

This is where a chunk of the ORION discrepancy reported by @EvanDietzMorris comes from: the Translator KG built on ITRB against NodeNorm ES has 0 nodes in every Food category combination that the RENCI build against Redis has (64 nodes), and 130 fewer nodes normalized overall.

Suggested fix

Take the file list from the release rather than from source: list the compendia/ directory (or read Babel's metadata/*.txt.yaml, one per compendium) and load every *.txt there, so the next new compendium is picked up automatically. Failing that, at least compare the directory listing against the hard-coded list at dump time and refuse to build an index that would silently drop a compendium.

Regression tests for this are in TranslatorSRI/babel-validation (tests/nodenorm/by_issue/biothings/); they will go green on the ES targets once the index contains these cliques.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions