MoRA: On-the-fly Molecule-aware Low-Rank Adaptation Framework for LLM-based Multi-Modal Molecular Assistant

• URL: http://arxiv.org/abs/2510.12245v1
• Date: Tue, 14 Oct 2025 07:54:43 GMT
• Title: MoRA: On-the-fly Molecule-aware Low-Rank Adaptation Framework for LLM-based Multi-Modal Molecular Assistant
• Authors: Tao Yin, Xiaohong Zhang, Jiacheng Zhang, Li Huang, Zhibin Zhang, Yuansong Zeng, Jin Xie, Meng Yan,
• Abstract summary: We propose an instance-specific parameter space alignment approach for each molecule on-the-fly.<n>MoRA produces a unique set of low-rank adaptation weights for each input molecular graph.<n>Experiments demonstrate that MoRA's instance-specific dynamic adaptation outperforms statically adapted baselines.
• Score: 23.60380089071549
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Effectively integrating molecular graph structures with Large Language Models (LLMs) is a key challenge in drug discovery. Most existing multi-modal alignment methods typically process these structures by fine-tuning the LLM or adding a static adapter simultaneously. However, these approaches have two main limitations: (1) it optimizes a shared parameter space across all molecular inputs, limiting the model's ability to capture instance-specific structural features; and (2) fine-tuning the LLM for molecular tasks can lead to catastrophic forgetting, undermining its general reasoning capabilities. In this paper, instead of static task-oriented adaptation, we propose an instance-specific parameter space alignment approach for each molecule on-the-fly. To this end, we introduce Molecule-aware Low-Rank Adaptation (MoRA) that produces a unique set of low-rank adaptation weights for each input molecular graph. These weights are then dynamically injected into a frozen LLM, allowing the model to adapt its reasoning to the structure of each molecular input, while preserving the LLM's core knowledge. Extensive experiments demonstrate that on key molecular tasks, such as chemical reaction prediction and molecular captioning, MoRA's instance-specific dynamic adaptation outperforms statically adapted baselines, including a 14.1% relative improvement in reaction prediction exact match and a 22% reduction in error for quantum property prediction. The code is available at https://github.com/jk-sounds/MoRA.

Related papers

• Breaking the Modality Barrier: Generative Modeling for Accurate Molecule Retrieval from Mass Spectra [60.08608779794957]
  We propose GLMR, a Generative Language Model-based Retrieval framework.<n>In the pre-retrieval stage, a contrastive learning-based model identifies top candidate molecules as contextual priors for the input mass spectrum.<n>In the generative retrieval stage, these candidate molecules are integrated with the input mass spectrum to guide a generative model in producing refined molecular structures.
  arXiv  Detail & Related papers  (2025-11-09T07:25:53Z)
• mCLM: A Modular Chemical Language Model that Generates Functional and Makeable Molecules [61.161776104062774]
  We propose a modular Chemical-Language Model that understands both natural language descriptions of functions and molecular blocks.<n>mCLM front-loads synthesizability considerations while improving the predicted functions of molecules in a principled manner.
  arXiv  Detail & Related papers  (2025-05-18T22:52:39Z)
• AdaptMol: Adaptive Fusion from Sequence String to Topological Structure for Few-shot Drug Discovery [7.338199946027998]
  We present AdaptMol, a prototypical network integrating Adaptive multimodal fusion for representation.<n>This framework employs a dual-level attention mechanism to dynamically integrate global and local molecular features.<n>Experiments on three commonly used benchmarks under 5-shot and 10-shot settings demonstrate that AdaptMol achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-05-17T07:12:12Z)
• Text-Guided Multi-Property Molecular Optimization with a Diffusion Language Model [20.250683535089617]
  We propose a text-guided multi-property molecular optimization method utilizing transformer-based diffusion language model (TransDLM)<n>By fusing physically and chemically detailed semantics with specialized molecular representations, TransDLM effectively integrates diverse information sources to guide precise optimization.
  arXiv  Detail & Related papers  (2024-10-17T14:30:27Z)
• MolecularGPT: Open Large Language Model (LLM) for Few-Shot Molecular Property Prediction [20.92779223471019]
  We present MolecularGPT for few-shot molecular property prediction. It exhibits competitive in-context reasoning capabilities across 10 downstream evaluation datasets. It also excels state-of-the-art LLM baselines by up to 15.7% increase on classification accuracy and decrease of 17.9 on regression metrics (e.g., RMSE) under zero-shot.
  arXiv  Detail & Related papers  (2024-06-18T12:54:47Z)
• MolX: Enhancing Large Language Models for Molecular Understanding With A Multi-Modal Extension [44.97089022713424]
  Large Language Models (LLMs) with their strong task-handling capabilities have shown remarkable advancements across a spectrum of fields.<n>This study seeks to enhance the ability of LLMs to comprehend molecules by equipping them with a multi-modal external module, termed MolX.<n>A hand-crafted molecular fingerprint is incorporated to leverage its embedded domain knowledge.
  arXiv  Detail & Related papers  (2024-06-10T20:25:18Z)
• Molecule Design by Latent Prompt Transformer [76.2112075557233]
  This work explores the challenging problem of molecule design by framing it as a conditional generative modeling task. We propose a novel generative model comprising three components: (1) a latent vector with a learnable prior distribution; (2) a molecule generation model based on a causal Transformer, which uses the latent vector as a prompt; and (3) a property prediction model that predicts a molecule's target properties and/or constraint values using the latent prompt.
  arXiv  Detail & Related papers  (2024-02-27T03:33:23Z)
• Implicit Geometry and Interaction Embeddings Improve Few-Shot Molecular Property Prediction [53.06671763877109]
  We develop molecular embeddings that encode complex molecular characteristics to improve the performance of few-shot molecular property prediction. Our approach leverages large amounts of synthetic data, namely the results of molecular docking calculations. On multiple molecular property prediction benchmarks, training from the embedding space substantially improves Multi-Task, MAML, and Prototypical Network few-shot learning performance.
  arXiv  Detail & Related papers  (2023-02-04T01:32:40Z)
• Chemical-Reaction-Aware Molecule Representation Learning [88.79052749877334]
  We propose using chemical reactions to assist learning molecule representation. Our approach is proven effective to 1) keep the embedding space well-organized and 2) improve the generalization ability of molecule embeddings. Experimental results demonstrate that our method achieves state-of-the-art performance in a variety of downstream tasks.
  arXiv  Detail & Related papers  (2021-09-21T00:08:43Z)
• MIMOSA: Multi-constraint Molecule Sampling for Molecule Optimization [51.00815310242277]
  generative models and reinforcement learning approaches made initial success, but still face difficulties in simultaneously optimizing multiple drug properties. We propose the MultI-constraint MOlecule SAmpling (MIMOSA) approach, a sampling framework to use input molecule as an initial guess and sample molecules from the target distribution.
  arXiv  Detail & Related papers  (2020-10-05T20:18:42Z)
• Predicting drug properties with parameter-free machine learning: Pareto-Optimal Embedded Modeling (POEM) [0.13854111346209866]
  We describe a similarity-based method for predicting molecular properties. POEM is a non-parametric, supervised ML algorithm developed to generate reliable predictive models without need for optimization. We benchmark POEM relative to industry-standard ML algorithms and published results across 17 classifications tasks. POEM performs well in all cases and reduces the risk of overfitting.
  arXiv  Detail & Related papers  (2020-02-11T17:20:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.