Beyond Predictive Accuracy: A Reliability-Aware Audit of Molecular Representations for Human Olfaction

• URL: http://arxiv.org/abs/2607.24848v1
• Date: Sat, 25 Jul 2026 03:38:14 GMT
• Title: Beyond Predictive Accuracy: A Reliability-Aware Audit of Molecular Representations for Human Olfaction
• Abstract summary: We present a reliability-aware audit of generic molecular representations for human olfaction across four claims.<n>Human three-attribute rating geometry, based on intensity, pleasantness, and familiarity, is reproducible across participant splits.<n>MoLFormer provides no clear incremental predictive value beyond a combined RDKit-Morgan baseline in either single-molecule dataset.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pretrained molecular encoders are commonly evaluated through downstream prediction, but predictive accuracy alone does not establish that a learned representation captures reproducible scientific structure, adds information beyond strong conventional baselines, or transfers out of distribution. We present a reliability-aware audit of generic molecular representations for human olfaction across four distinct claims: global perceptual geometry, incremental predictive value beyond chemistry, cross-dataset replication, and mixture transfer to unseen components. Using the Keller-Vosshall and Bierling single-molecule rating datasets and the Ma binary-mixture dataset, we compare MoLFormer and ChemBERTa against RDKit descriptors and Morgan fingerprints under identity-controlled and matched evaluations. Human three-attribute rating geometry, based on intensity, pleasantness, and familiarity, is reproducible across participant splits (median RSA 0.743 and 0.855), whereas model-human alignment is substantially weaker (RSA 0.019-0.158). Learned embeddings do not consistently outperform conventional representations in global alignment, and MoLFormer provides no clear incremental predictive value beyond a combined RDKit-Morgan baseline in either single-molecule dataset. Human geometry shows positive but incomplete agreement across 63 shared molecules (RSA 0.331; 95% bootstrap interval [0.204, 0.507]). Under one strict unseen-component mixture split, incremental effects are outcome- and representation-dependent, with all intervals crossing zero. These results establish empirical boundaries for the evaluated generic molecular encoders and motivate a broader evaluation principle: representation quality in scientific domains should be assessed separately for target reliability, structural alignment, incremental information, replication, and out-of-distribution transfer.

Related papers

• Impute-EM: Native Mixed-State Diffusion Models for Heterogeneous Data Imputation [71.74203645336385]
  Impute-EM alternates between imputing missing entries with the current model and refitting a diffusion backbone on completed data.<n>We characterize the update and show that the observed mask-indexed marginals match the targets at the limit.
  arXiv  Detail & Related papers  (2026-09-14T09:39:13Z)
• Aggregate-then-Calibrate for Human-centered Assessment with Theoretical Guarantees [15.471672928989138]
  Methods using only human judgments suffer from heterogeneous expertise and inconsistent rating scales.<n>We propose Aggregate-then-Calibrate (AtC), a two-stage framework that combines these complementary sources.<n>AtC consistently improves accuracy and robustness over human-only or model-only assessments.
  arXiv  Detail & Related papers  (2026-08-03T16:30:30Z)
• Chem-PerturBridge: a harmonized compendium of small molecule perturbation transcriptomic effects [4.860649440823431]
  Chem-PerturBridge is a harmonized multi-dataset resource comprising over 37k compounds, 136 cellular contexts, and 1.25M transcriptomic samples.<n>We use the resource to evaluate matched-condition agreement across datasets and replicate agreement within datasets.
  arXiv  Detail & Related papers  (2026-05-29T16:38:30Z)
• A Systematic Evaluation of Molecular Mixture Behavior Prediction [0.0]
  We propose an evaluation framework that decomposes mixture-property error into pure-compound and interaction (non-ideal) components.<n>We show that strong absolute accuracy can mask poor recovery of non-ideal mixture behavior.<n>These results identify transfer to unseen molecules as a central challenge in molecular mixture machine learning.
  arXiv  Detail & Related papers  (2026-05-28T09:58:59Z)
• TerraBind: Fast and Accurate Binding Affinity Prediction through Coarse Structural Representations [0.7891868017562221]
  We present TerraBind, a foundation model for protein-ligand structure and binding affinity prediction.<n>It achieves 26-fold faster inference than state-of-the-art methods while improving affinity prediction accuracy by $sim$20%.
  arXiv  Detail & Related papers  (2026-02-08T00:01:43Z)
• Diagnosing Heteroskedasticity and Resolving Multicollinearity Paradoxes in Physicochemical Property Prediction [0.2330146832085843]
  We analyzed 426,850 bioactive molecules from PubChem, ChEMBL, and eMolecules databases.<n>We found severe heteroskedasticity in linear models predicting computed logP values.<n>Tree-based ensemble methods proved inherently robust to heteroskedasticity while delivering superior predictive performance.
  arXiv  Detail & Related papers  (2025-12-31T05:32:13Z)
• Aligned Manifold Property and Topology Point Clouds for Learning Molecular Properties [55.2480439325792]
  This work introduces AMPTCR, a molecular surface representation that combines local quantum-derived scalar fields and custom topological descriptors within an aligned point cloud format.<n>For molecular weight, results confirm that AMPTCR encodes physically meaningful data, with a validation R2 of 0.87.<n>In the bacterial inhibition task, AMPTCR enables both classification and direct regression of E. coli inhibition values.
  arXiv  Detail & Related papers  (2025-07-22T04:35:50Z)
• Robust Molecular Property Prediction via Densifying Scarce Labeled Data [53.24886143129006]
  In drug discovery, compounds most critical for advancing research often lie beyond the training set.<n>We propose a novel bilevel optimization approach that leverages unlabeled data to interpolate between in-distribution (ID) and out-of-distribution (OOD) data.
  arXiv  Detail & Related papers  (2025-06-13T15:27:40Z)
• Collaborative Heterogeneous Causal Inference Beyond Meta-analysis [68.4474531911361]
  We propose a collaborative inverse propensity score estimator for causal inference with heterogeneous data. Our method shows significant improvements over the methods based on meta-analysis when heterogeneity increases.
  arXiv  Detail & Related papers  (2024-04-24T09:04:36Z)
• SE(3)-Invariant Multiparameter Persistent Homology for Chiral-Sensitive Molecular Property Prediction [1.534667887016089]
  We present a novel method for generating molecular fingerprints using multi parameter persistent homology (MPPH) This technique holds considerable significance for drug discovery and materials science, where precise molecular property prediction is vital. We demonstrate its superior performance over existing state-of-the-art methods in predicting molecular properties through extensive evaluations on the MoleculeNet benchmark.
  arXiv  Detail & Related papers  (2023-12-12T09:33:54Z)
• Prototype-based Aleatoric Uncertainty Quantification for Cross-modal Retrieval [139.21955930418815]
  Cross-modal Retrieval methods build similarity relations between vision and language modalities by jointly learning a common representation space. However, the predictions are often unreliable due to the Aleatoric uncertainty, which is induced by low-quality data, e.g., corrupt images, fast-paced videos, and non-detailed texts. We propose a novel Prototype-based Aleatoric Uncertainty Quantification (PAU) framework to provide trustworthy predictions by quantifying the uncertainty arisen from the inherent data ambiguity.
  arXiv  Detail & Related papers  (2023-09-29T09:41:19Z)
• Good Classifiers are Abundant in the Interpolating Regime [64.72044662855612]
  We develop a methodology to compute precisely the full distribution of test errors among interpolating classifiers. We find that test errors tend to concentrate around a small typical value $varepsilon*$, which deviates substantially from the test error of worst-case interpolating model. Our results show that the usual style of analysis in statistical learning theory may not be fine-grained enough to capture the good generalization performance observed in practice.
  arXiv  Detail & Related papers  (2020-06-22T21:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.