C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs

• URL: http://arxiv.org/abs/2604.23061v1
• Date: Fri, 24 Apr 2026 23:11:44 GMT
• Title: C-MORAL: Controllable Multi-Objective Molecular Optimization with Reinforcement Alignment for LLMs
• Abstract summary: C-Moral is a reinforcement learning framework for controllable multi-objective molecular optimization.<n>Experiments on the C-MuMOInstruct benchmark show that C-Moral consistently outperforms state-of-the-art models.
• Score: 16.006924868560795
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models (LLMs) show promise for molecular optimization, but aligning them with selective and competing drug-design constraints remains challenging. We propose C-Moral, a reinforcement learning post-training framework for controllable multi-objective molecular optimization. C-Moral combines group-based relative optimization, property score alignment for heterogeneous objectives, and continuous non-linear reward aggregation to improve stability across competing properties. Experiments on the C-MuMOInstruct benchmark show that C-Moral consistently outperforms state-of-the-art models across both in-domain and out-of-domain settings, achieving the best Success Optimized Rate (SOR) of 48.9% on IND tasks and 39.5% on OOD tasks, while largely preserving scaffold similarity. These results suggest that RL post-training is an effective way to align molecular language models with continuous molecular design objectives. Our code and models are publicly available at https://github.com/Rwigie/C-MORAL.

Related papers

• CoMerge: Conflict-Driven Preference Optimization for Multi-Task Model Merging [59.836482609414816]
  We propose a conflict-driven preference optimization framework for model merging (CoMerge)<n>CoMerge enables the model to mitigate parameter-space conflicts while preserving task-specific capabilities.<n>Experiments show that CoMerge achieves an average normalized performance of 0.9968 on MergeBench.
  arXiv  Detail & Related papers  (2026-09-02T08:21:28Z)
• Be Your Own Teacher: Steering Protein Language Models via Unsupervised Reward Optimization [85.4863852505905]
  We introduce unsupervised reward optimization of protein language models (PLMs)<n>Our key insight is that task-agnostic rewards, which combine intrinsic model uncertainty with semantic consistency informed by protein representation models, exhibit strong correlation with controllability measures across base models and temperature regimes.<n>Our framework provides a scalable pathway toward controllable biomolecular design in settings where labeled preferences or experimental feedback are scarce or unavailable.
  arXiv  Detail & Related papers  (2026-06-17T11:42:01Z)
• Controllable Molecular Generative Foundation Models [36.63304478618632]
  We propose textbfControllable textbfMolecular Generative Foundation Models (CoMole)<n>By learning a motif-aware graph space, CoMole transfers pretrained structural priors into controllable generation.<n>We show that CoMole ranks first in controllability on all nine targets, reduces MAE by up to 48.2% relative to the strongest baselines, and maintains validity above 0.94 without rule-based correction or post-hoc filtering.
  arXiv  Detail & Related papers  (2026-05-14T19:27:01Z)
• Scaffold-Conditioned Preference Triplets for Controllable Molecular Optimization with Large Language Models [32.68069719676566]
  We introduce textbfScaffold-Conditioned Preference Triplets ( SCPT), a pipeline that constructs similarity-constrained triplets $langletextscaffold, textbetter, textworserangle$ via scaffold alignment and chemistry-driven filters for validity, synthesizability, and meaningful property gains.<n>Compared with representative non-LLM molecular optimization methods, SCPT-trained LLMs are better suited to scaffold-constrained and multi-objective optimization.
  arXiv  Detail & Related papers  (2026-04-14T06:38:07Z)
• Composable Score-based Graph Diffusion Model for Multi-Conditional Molecular Generation [85.58520120011269]
  We propose Composable Score-based Graph Diffusion model (CSGD), which extends score matching to discrete graphs via concrete scores.<n>We show that CSGD achieves state-of-the-art performance with a 15.3% average improvement in controllability over prior methods.<n>Our findings highlight the practical advantages of score-based modeling for discrete graph generation and its capacity for flexible, multi-property molecular design.
  arXiv  Detail & Related papers  (2025-09-11T13:37:56Z)
• Discrete Diffusion Trajectory Alignment via Stepwise Decomposition [80.66701883088935]
  We propose an offline preference optimization method to approach trajectory alignment for discrete diffusion models.<n>We decompose the problem into a set of stepwise alignment objectives by matching the per-step posterior.<n> Experiments across multiple domains including DNA sequence design, protein inverse folding, and language modeling consistently demonstrate the superiority of our approach.
  arXiv  Detail & Related papers  (2025-07-07T09:52:56Z)
• Large Language Models for Controllable Multi-property Multi-objective Molecule Optimization [2.152507712409726]
  We introduce C-MuMOInstruct, the first instruction-tuning dataset focused on multi-property optimization with explicit, property-specific objectives.<n>We develop GeLLMO-Cs, a series of instruction-tuned LLMs that can perform targeted property-specific optimization.<n>Our experiments show that GeLLMO-Cs consistently outperform strong baselines, achieving up to 126% higher success rate.
  arXiv  Detail & Related papers  (2025-05-29T20:29:14Z)
• Empower Structure-Based Molecule Optimization with Gradient Guided Bayesian Flow Networks [16.743187639189976]
  Structure-Based molecule optimization (SBMO) aims to optimize molecules with both continuous coordinates and discrete types against protein targets.<n>MolJO is a gradient-based framework that facilitates joint guidance signals across different modalities.<n>MolJO achieves state-of-the-art performance on CrossDocked 2020 benchmark.
  arXiv  Detail & Related papers  (2024-11-20T12:48:29Z)
• Decoding-Time Language Model Alignment with Multiple Objectives [116.42095026960598]
  Existing methods primarily focus on optimizing LMs for a single reward function, limiting their adaptability to varied objectives. Here, we propose $textbfmulti-objective decoding (MOD)$, a decoding-time algorithm that outputs the next token from a linear combination of predictions. We show why existing approaches can be sub-optimal even in natural settings and obtain optimality guarantees for our method.
  arXiv  Detail & Related papers  (2024-06-27T02:46:30Z)
• DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization [49.85944390503957]
  DecompOpt is a structure-based molecular optimization method based on a controllable and diffusion model. We show that DecompOpt can efficiently generate molecules with improved properties than strong de novo baselines.
  arXiv  Detail & Related papers  (2024-03-07T02:53:40Z)
• Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization [76.09576643028362]
  We present Multi-Objective Direct Preference Optimization (MODPO) for multiple alignment objectives. MODPO folds language modeling directly into reward modeling, training language models as implicit collective reward models. It theoretically yields the same optimal solutions as MORLHF but is practically more stable and efficient.
  arXiv  Detail & Related papers  (2023-10-05T17:35:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.