Repository navigation
MoleculeNet datasets - #48
aditya0by0 wants to merge 69 commits into
Conversation
…ted entries leading to cross merge
…python-chebai-graph into feature/molenet_dataset
…python-chebai-graph into feature/molenet_dataset
…python-chebai-graph into feature/molenet_dataset
…python-chebai-graph into feature/molenet_dataset
How Missing Labels Are Handled in HiMol FinetuningHiMol uses a three-valued label encoding for classification: +1 (positive/active), -1 (negative/inactive), and 0 (missing/unmeasured). The Loss ComputationIn is_valid = y**2 > 0 # 0²=0 → invalid; (±1)²=1 → valid
loss_mat = criterion(pred.double(), (y+1)/2) # {-1,+1} → {0,1} for BCE
loss_mat = torch.where(is_valid, loss_mat, torch.zeros(...)) # zero out missing
loss = torch.sum(loss_mat) / torch.sum(is_valid) # normalize by valid countMissing labels are zeroed out in the loss matrix and excluded from the normalization denominator ( Metrics Computation
Note: The |
fixed in ChEB-AI/python-chebai#174 |
Reference: Also be PR can be abandoned and closed #53
…been added. The .gitignore file has been updated to include the new pretraining scripts.
|
I see some cleanup potential here: Some of the files are specific to the JCI paper and not relevant to a user that just wants to run a GNN model. This includes
Most of these files would actually be useful for reproducing the experiments, so I am not suggesting to delete them. Instead, we should put them on zenodo alongside the checkpoints / datasets as supplementary material. My point is mainly about removing them from git so we have a cleaner repository for future projects. |
Context: The dataset-specific token directories were originally introduced intentionally because we wanted to support different sets of tokens for different datasets (e.g., dataset-specific functional group tokens). See PR: #47 However, this design introduced a limitation when it came to transferring knowledge from pretrained models to downstream tasks. In particular, it prevented us from directly fine-tuning a model pretrained on ZINC on MoleculeNet benchmarks for the following reasons:
To address these issues, in I introduced a common The idea is to support two distinct workflows:
Therefore, having separate token directories is intentional rather than redundant. The For this reason, I would suggest retaining the current structure rather than moving everything to a single fixed vocabulary. |
I agree that there is room for cleanup here. However, I would suggest addressing the removal or reorganization of JCI-paper-specific files in a separate PR, as the scope of this PR is specifically focused on the MoleculeNet dataset changes. This would help keep the current PR focused and make the changes easier to review.
Similarly, I think we could address the cleanup and consolidation of configuration files in a separate PR.
To address the usability aspect, I have updated the README in this PR to provide clear instructions for reproducing our experiments ([README](https://github.com/ChEB-AI/python-chebai-graph/tree/feature/molenet_dataset)). I have specifically included references to the corresponding shell scripts so that users can reproduce the experiments without having to navigate through the configuration files or understand the various PyTorch Lightning-specific command-line arguments and hyperparameters. Given that we currently have only five such shell scripts, I think retaining them is reasonable. Moreover, documenting their usage in the README makes the reproduction process more straightforward and user-friendly.
I see this as a standalone helper script which lives outside of |
I agree that we should keep the structure as it is for now.
The scripts and README are indeed useful documentation for the experiments. However, if we merge this now (and move consolidation to another PR), this would put the paper-specific code on the dev branch. I think removing the scripts / config files now would help me with reviewing this PR. One solution I can suggest: We could tag the current version (including the paper-specific instructions) and refer to that version in the paper. This would allow to guide readers directly to the scripts, but keep it out of the regular versioning. |
|
@sfluegel05, I have made the suggested changes. Please review.
removed, only atmost 2 config files per dataset retained, as suggested
removed
removed
removed |
@sfluegel05, I gave this another thought, and I agree with your point. Ideally, we should maintain a single, shared token directory across all datasets rather than having separate token lists for each dataset. I don't think using a shared vocabulary introduces any data leakage or exposes information from the test or validation sets. The atom properties we use are fundamental and independent of any particular dataset. For example, we already know that atomic numbers range from 1 to 119. Even if a particular dataset contains only a small subset of these elements, that should not prevent us from using the full vocabulary rather than constructing a vocabulary exclusively from the training portion of that dataset. Similarly for functional groups, we already know from our FG detection algorithm that we can detect around 101 non-ring FG groups, and ring-related FG are fundamental as discussed earlier. Moreover, I don't expect using a full, shared vocabulary instead of a vocabulary generated from the training split to have any significant impact on the model's input representation or computational requirements. In both cases, the input property vectors are zero-padded to match the size of the largest tensor (203 in our case). Therefore, using the full vocabulary would primarily provide consistency across datasets without introducing any meaningful additional overhead. Hence, I will remove the tokens related to other datasets while retaining only |
Also merge: sfluegel05/HiMol#3