ControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization

• URL: http://arxiv.org/abs/2502.10631v1
• Date: Sat, 15 Feb 2025 01:49:35 GMT
• Title: ControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization
• Abstract summary: We introduce ControllableGPT, a controllable training framework for large language models.<n>It is inspired by the biological processes of growth and evolution, which involve the expansion, shrinking, and mutation of sequences.<n>It enables the precise management of specific locations and ranges within a sequence, while maintaining the integrity of any specified positions or subsequences.
• Score: 6.900025190052277
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large Language Models (LLMs) employ three popular training approaches: Masked Language Models (MLM), Causal Language Models (CLM), and Sequence-to-Sequence Models (seq2seq). However, each approach has its strengths and limitations, and faces challenges in addressing specific tasks that require controllable and bidirectional generation, such as drug optimization. To address this challenge, inspired by the biological processes of growth and evolution, which involve the expansion, shrinking, and mutation of sequences, we introduce ControllableGPT. This initiative represents the first effort to combine the advantages of MLM, CLM, and seq2seq into a single unified, controllable GPT framework. It enables the precise management of specific locations and ranges within a sequence, allowing for expansion, reduction, or mutation over chosen or random lengths, while maintaining the integrity of any specified positions or subsequences. In this work, we designed ControllableGPT for drug optimization from the ground up, which included proposing the Causally Masked Seq2seq (CMS) objective, developing the training corpus, introducing a novel pre-training approach, and devising a unique generation process. We demonstrate the effectiveness and controllability of ControllableGPT by conducting experiments on drug optimization tasks for both viral and cancer benchmarks, surpassing competing baselines.

Related papers

• Adopting Reinforcement Learning with Verifiable Rewards for Molecular Generation [17.176795343295854]
  Leveraging large language models for molecular generation has shown remarkable potential in chemical and drug design.<n>We propose textbfLLMol, a principled reinforcement learning framework that directly incorporates verifiable rewards for targeted molecule generation.
  arXiv  Detail & Related papers  (2026-07-21T12:34:59Z)
• Scaling Behavior Foundation Model for Humanoid Robots [77.31115920959364]
  Behavior Foundation Models (BFMs) have emerged as a promising solution to address challenges by leveraging large-scale behavioral data.<n>We show that our approach yields significant improvements in control fidelity and task generalization.<n>These results establish BFM as a principled and effective foundation for scalable and general-purpose humanoid control.
  arXiv  Detail & Related papers  (2026-07-16T16:08:27Z)
• Controllable Molecular Generative Foundation Models [36.63304478618632]
  We propose textbfControllable textbfMolecular Generative Foundation Models (CoMole)<n>By learning a motif-aware graph space, CoMole transfers pretrained structural priors into controllable generation.<n>We show that CoMole ranks first in controllability on all nine targets, reduces MAE by up to 48.2% relative to the strongest baselines, and maintains validity above 0.94 without rule-based correction or post-hoc filtering.
  arXiv  Detail & Related papers  (2026-05-14T19:27:01Z)
• SCULPT: Constraint-Guided Pruned MCTS that Carves Efficient Paths for Mathematical Reasoning [11.991985041067638]
  This paper introduces SCULPT, a constraint-guided approach for Monte Carlo Tree Search (MCTS)<n>SCULPT scores and prunes actions using a combination of symbolic checks (dimensional consistency, type compatibility, magnitude sanity, depth control, and diversity) and structural pattern guidance.<n>Overall, domain-aware constraints can improve accuracy while maintaining efficiency and stability.
  arXiv  Detail & Related papers  (2026-01-19T08:55:46Z)
• M^4olGen: Multi-Agent, Multi-Stage Molecular Generation under Precise Multi-Property Constraints [35.83366265234892]
  textbfM olGen is a fragment-level, retrieval-augmented, two-stage framework for molecule generation under multi-property constraints.<n>A dataset with reasoning chains of fragment edits and measured property deltas underpins both stages.<n>Experiments on generation under two sets of property constraints show consistent gains in validity and precise satisfaction of multi-property targets.
  arXiv  Detail & Related papers  (2026-01-15T07:18:05Z)
• Flow Density Control: Generative Optimization Beyond Entropy-Regularized Fine-Tuning [59.11663802446183]
  Flow and diffusion generative models can be adapted to optimize task-specific objectives while preserving prior information.<n>We introduce Flow Density Control (FDC), a simple algorithm that reduces this complex problem to a specific sequence of simpler fine-tuning tasks.<n>We derive convergence guarantees for the proposed scheme under realistic assumptions by leveraging recent understanding of mirror flows.
  arXiv  Detail & Related papers  (2025-11-27T17:19:01Z)
• OMGPT: A Sequence Modeling Framework for Data-driven Operational Decision Making [5.419799294989289]
  We build a Generative Pre-trained Transformer (GPT) model to solve sequential decision making tasks.<n>We first propose a general sequence modeling framework to cover several operational decision making tasks.<n>We then train a transformer-based neural network model (OMGPT) as a natural and powerful architecture for sequential modeling.
  arXiv  Detail & Related papers  (2025-05-19T15:33:03Z)
• Multi-Objective-Guided Discrete Flow Matching for Controllable Biological Sequence Design [5.64033072458324]
  We present Multi-Objective-Guided Discrete Flow Matching (MOG-DFM), a general framework to steer any pretrained discrete flow matching generator.<n>At each sampling step, MOG-DFM computes a hybrid rank-directional score for candidate transitions.<n>We also trained two unconditional discrete flow matching models, PepDFM for diverse peptide generation and EnhancerDFM for functional enhancer DNA generation.
  arXiv  Detail & Related papers  (2025-05-11T18:17:44Z)
• Inference-Time Intervention in Large Language Models for Reliable Requirement Verification [2.3759432635713895]
  Inference-time intervention techniques provide a promising alternative to fine-tuning. We demonstrate how interventions enable fine-grained control for automating the usually time-intensive requirement verification process. Our method achieves robust and reliable outputs, significantly improving over both a baseline model and a fine-tuning approach.
  arXiv  Detail & Related papers  (2025-03-18T10:49:36Z)
• Regulatory DNA sequence Design with Reinforcement Learning [56.20290878358356]
  We propose a generative approach that leverages reinforcement learning to fine-tune a pre-trained autoregressive model. We evaluate our method on promoter design tasks in two yeast media conditions and enhancer design tasks for three human cell types.
  arXiv  Detail & Related papers  (2025-03-11T02:33:33Z)
• Test-Time Alignment for Large Language Models via Textual Model Predictive Control [63.508812485566374]
  Textual Model Predictive Control (TMPC) is a novel predictive planning framework adapted for aligning Large Language Models at inference time.<n>TMPC is evaluated on three tasks with distinct segmentation properties: discourse-level translation, long-form response generation, and program synthesis.<n>Results demonstrate that TMPC consistently improves performance, highlighting the generality.
  arXiv  Detail & Related papers  (2025-02-28T07:24:33Z)
• ScaffoldGPT: A Scaffold-based Large Language Model for Drug Improvement [2.6198448284771443]
  ScaffoldGPT is a novel Large Language Model (LLM) designed for drug optimization based on molecular scaffolds.<n>Our work comprises three key components: (1) A three-stage drug optimization approach that integrates pretraining, finetuning, and decoding optimization.
  arXiv  Detail & Related papers  (2025-02-09T10:36:33Z)
• Zero-Shot Strategies for Length-Controllable Summarization [56.15356055672189]
  Large language models (LLMs) struggle with precise length control, particularly in zero-shot settings.<n>We conduct a comprehensive study evaluating LLMs' length control capabilities across multiple measures and propose practical methods to improve controllability.<n>Our experiments with LLaMA 3 reveal stark differences in length adherence across measures and highlight inherent biases of the model.
  arXiv  Detail & Related papers  (2024-12-31T02:53:27Z)
• Controlling the Mutation in Large Language Models for the Efficient Evolution of Algorithms [2.2485774453793037]
  This paper introduces a novel approach to mutation control within evolutionary frameworks, inspired by theory of genetic algorithms.<n>We propose dynamic mutation prompts that adaptively regulate mutation rates, leveraging a heavy-tailed power-law distribution to balance exploration and exploitation.<n>Experiments show that the introduction of these dynamic rates can improve the convergence speed and adaptability of LLaMEA.
  arXiv  Detail & Related papers  (2024-12-04T11:49:22Z)
• Preparing Spin Squeezed States via Adaptive Genetic Algorithm [9.168152138847445]
  We introduce a novel strategy employing an adaptive genetic algorithm (GA) for iterative optimization of control sequences to generate quantum nonclassical states. Inspired by Darwinian evolution, the algorithm iteratively refines control sequences using crossover, mutation, and elimination strategies. Our approach, compared to constant control schemes, yields a variety of control sequences capable of maintaining squeezing for the collective spin model.
  arXiv  Detail & Related papers  (2024-10-20T12:15:11Z)
• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• Adaptable Logical Control for Large Language Models [68.27725600175013]
  Ctrl-G is an adaptable framework that facilitates tractable and flexible control of model generation at inference time. We show that Ctrl-G, when applied to a TULU2-7B model, outperforms GPT3.5 and GPT4 on the task of interactive text editing.
  arXiv  Detail & Related papers  (2024-06-19T23:47:59Z)
• DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization [49.85944390503957]
  DecompOpt is a structure-based molecular optimization method based on a controllable and diffusion model. We show that DecompOpt can efficiently generate molecules with improved properties than strong de novo baselines.
  arXiv  Detail & Related papers  (2024-03-07T02:53:40Z)
• DECIDER: A Dual-System Rule-Controllable Decoding Framework for Language Generation [57.07295906718989]
  Constrained decoding approaches aim to control the meaning or style of text generated by a Pre-trained Language Model (PLM) using specific target words during inference. We propose a novel decoding framework, DECIDER, which enables us to program rules on how we complete tasks to control a PLM.
  arXiv  Detail & Related papers  (2024-03-04T11:49:08Z)
• Multi-modal Self-supervised Pre-training for Regulatory Genome Across Cell Types [75.65676405302105]
  We propose a simple yet effective approach for pre-training genome data in a multi-modal and self-supervised manner, which we call GeneBERT. We pre-train our model on the ATAC-seq dataset with 17 million genome sequences.
  arXiv  Detail & Related papers  (2021-10-11T12:48:44Z)
• Multi-Level Evolution Strategies for High-Resolution Black-Box Control [0.2320417845168326]
  This paper introduces a multi-level (m-lev) mechanism into Evolution Strategies (ESs) It addresses a class of global optimization problems that could benefit from fine discretization of their decision variables.
  arXiv  Detail & Related papers  (2020-10-04T09:24:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.