From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Models

• URL: http://arxiv.org/abs/2605.09949v1
• Date: Mon, 11 May 2026 03:53:49 GMT
• Title: From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Models
• Abstract summary: We present Pan-CORE, a family of autoregressive Transformer-based encoder-decoder models for SMILES translation.<n>We use high-temporal-resolution checkpoint analysis to investigate how chiral information is learned during training.<n>Findings show that SMILES translation can serve as a useful experimental system for mechanistic analysis of semantic emergence in CLMs.
• Score: 1.481663141554356
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Understanding how chemical language models (CLMs) learn chemical meaning from molecular string representations, rather than only surface-level string patterns, is an important question in chemical representation learning and machine learning for chemistry. Chirality provides a demanding test case: enantiomers can differ greatly in pharmacological activity and toxicity, yet CLMs often struggle to distinguish chiral configurations reliably. Here we present Pan-CORE (Pan-Chemical Omniscale Representation Engine), a family of autoregressive Transformer-based encoder-decoder models for SMILES translation, and use high-temporal-resolution checkpoint analysis to investigate how chiral information is learned during training. Across all tested Pan-CORE variants, we observe a reproducible jump-up in which chiral-token accuracy rises abruptly after a long plateau, suggesting that chiral learning stagnation is not explained by model capacity alone and instead reflects the complexity of chiral constraints. Analyses of attention dynamics, residual-stream trajectories, and latent-space geometry support an encoder-centered mechanism in which chiral-token representations undergo transient destabilization and reconstruction, seen as a V-shaped drop and recovery in vector norm and directional stability, together with a clear reorganization of chiral molecular representations in the latent space. Encoder-decoder cross-evaluation further supports the encoder-centered nature of the transition, and targeted attention-head ablation identifies a small set of chiral-sensitive heads whose removal selectively reduces chiral-token accuracy even in the fully trained model. These findings show that SMILES translation can serve as a useful experimental system for mechanistic analysis of semantic emergence in CLMs, with implications for interpretable chemical representation learning.

Related papers

• SCENT: Aligning Mass Spectra with Molecular Structure for Olfactory Perception [48.96518000981492]
  We explore direct electron ionization mass spectrometry (EI-MS) as an alternative input modality for olfactory prediction.<n>We contribute Spectrum-to-Chemical Embedding alignmeNT (SCENT), a multi-modal contrastive learning framework that aligns EI-MS representations with pretrained chemical structure embeddings.<n>On the multi-label odor descriptor prediction task, SCENT significantly outperforms MS-only baselines and achieves olfactory performance comparable to structure-based models.
  arXiv  Detail & Related papers  (2026-05-26T13:28:57Z)
• ChemVA: Advancing Large Language Models on Chemical Reaction Diagrams Understanding [56.94419746470799]
  Large Language Models (LLMs) have revolutionized scientific text processing, but exhibit a significant capability gap when interpreting chemical reaction diagrams.<n>We propose the Chemical Visual Activation (ChemVA) framework, which employs a Visual Anchor mechanism to ground functional groups.<n>We evaluate our approach on OCRD-Bench, a newly constructed dataset featuring dense visual-semantic contexts.
  arXiv  Detail & Related papers  (2026-05-17T01:12:50Z)
• Molecules Meet Language: Confound-Aware Representation Learning and Chemical Property Steering in Transformer-VAE Latent Spaces [0.9332987715848714]
  Molecular generative models often assume meaningful latent geometry, but apparent property predictability can reflect sequence-level shortcuts.<n>We study this issue in an unsupervised autoregressive Transformer-VAE trained on SELFIES.<n>We find chemically meaningful steering can emerge in entangled molecular latent spaces, but only when validated through decoded molecules.
  arXiv  Detail & Related papers  (2026-05-07T14:07:10Z)
• Learning Molecular Chirality via Chiral Determinant Kernels [21.293956625903352]
  Chiality is a fundamental molecular property that governs stereospecific behavior in chemistry and biology.<n>ChiDeK is a framework that integrates stereogenic information into molecular representation learning.<n>ChiDeK achieves substantial improvements over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-02-07T07:21:43Z)
• Causal Representation Meets Stochastic Modeling under Generic Geometry [49.24293444627916]
  We develop causal representation learning for continuous-time latent point processes.<n>We develop MUTATE, an identifiable variational autoencoder framework with a time-adaptive transition module.<n>Across simulated and empirical studies, we find that MUTATE can effectively answer scientific questions.
  arXiv  Detail & Related papers  (2026-02-04T20:40:53Z)
• Agentic reinforcement learning empowers next-generation chemical language models for molecular design and synthesis [51.83339196548892]
  ChemCraft is a novel framework that decouples chemical reasoning from knowledge storage.<n>ChemCraft achieves superior performance with minimal inference costs.<n>This work establishes a cost-effective and privacy-preserving paradigm for AI-aided chemistry.
  arXiv  Detail & Related papers  (2026-01-25T04:23:34Z)
• How well can off-the-shelf LLMs elucidate molecular structures from mass spectra using chain-of-thought reasoning? [51.286853421822705]
  Large language models (LLMs) have shown promise for reasoning-intensive scientific tasks, but their capability for chemical interpretation is still unclear.<n>We introduce a Chain-of-Thought (CoT) prompting framework and benchmark that evaluate how LLMs reason about mass spectral data to predict molecular structures.<n>Our evaluation across metrics of SMILES validity, formula consistency, and structural similarity reveals that while LLMs can produce syntactically valid and partially plausible structures, they fail to achieve chemical accuracy or link reasoning to correct molecular predictions.
  arXiv  Detail & Related papers  (2026-01-09T20:08:42Z)
• SIGMA: Scalable Spectral Insights for LLM Collapse [51.863164847253366]
  We introduce SIGMA (Spectral Inequalities for Gram Matrix Analysis), a unified framework for model collapse.<n>By utilizing benchmarks that deriving and deterministic bounds on the matrix's spectrum, SIGMA provides a mathematically grounded metric to track the contraction of the representation space.<n>We demonstrate that SIGMA effectively captures the transition towards states, offering both theoretical insights into the mechanics of collapse.
  arXiv  Detail & Related papers  (2026-01-06T19:47:11Z)
• Circuits, Features, and Heuristics in Molecular Transformers [0.056179939237156]
  We present a mechanistic analysis of autoregressive transformers trained on drug-like small molecules.<n>We identify computational patterns consistent with low-level syntactic parsing and more abstract chemical validity constraints.
  arXiv  Detail & Related papers  (2025-12-10T15:35:22Z)
• Aligned Manifold Property and Topology Point Clouds for Learning Molecular Properties [55.2480439325792]
  This work introduces AMPTCR, a molecular surface representation that combines local quantum-derived scalar fields and custom topological descriptors within an aligned point cloud format.<n>For molecular weight, results confirm that AMPTCR encodes physically meaningful data, with a validation R2 of 0.87.<n>In the bacterial inhibition task, AMPTCR enables both classification and direct regression of E. coli inhibition values.
  arXiv  Detail & Related papers  (2025-07-22T04:35:50Z)
• Pre-trained Molecular Language Models with Random Functional Group Masking [54.900360309677794]
  We propose a SMILES-based underlineem Molecular underlineem Language underlineem Model, which randomly masking SMILES subsequences corresponding to specific molecular atoms. This technique aims to compel the model to better infer molecular structures and properties, thus enhancing its predictive capabilities.
  arXiv  Detail & Related papers  (2024-11-03T01:56:15Z)
• Empowering Machines to Think Like Chemists: Unveiling Molecular Structure-Polarity Relationships with Hierarchical Symbolic Regression [1.6986628849901197]
  We introduce Unsupervised Hierarchical Symbolic Regression (UHiSR), combining hierarchical neural networks and symbolic regression. UHiSR automatically distills chemical-intuitive polarity indices, and discovers interpretable equations that link molecular structure to chromatographic behavior.
  arXiv  Detail & Related papers  (2024-01-25T02:48:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.