ControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization

• URL: http://arxiv.org/abs/2502.10631v1
• Date: Sat, 15 Feb 2025 01:49:35 GMT
• Title: ControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization
• Authors: Xuefeng Liu, Songhao Jiang, Bo Li, Rick Stevens,
• Abstract summary: We introduce ControllableGPT, a controllable training framework for large language models. It is inspired by the biological processes of growth and evolution, which involve the expansion, shrinking, and mutation of sequences. It enables the precise management of specific locations and ranges within a sequence, while maintaining the integrity of any specified positions or subsequences.
• Score: 6.900025190052277
• License:
• Abstract: Large Language Models (LLMs) employ three popular training approaches: Masked Language Models (MLM), Causal Language Models (CLM), and Sequence-to-Sequence Models (seq2seq). However, each approach has its strengths and limitations, and faces challenges in addressing specific tasks that require controllable and bidirectional generation, such as drug optimization. To address this challenge, inspired by the biological processes of growth and evolution, which involve the expansion, shrinking, and mutation of sequences, we introduce ControllableGPT. This initiative represents the first effort to combine the advantages of MLM, CLM, and seq2seq into a single unified, controllable GPT framework. It enables the precise management of specific locations and ranges within a sequence, allowing for expansion, reduction, or mutation over chosen or random lengths, while maintaining the integrity of any specified positions or subsequences. In this work, we designed ControllableGPT for drug optimization from the ground up, which included proposing the Causally Masked Seq2seq (CMS) objective, developing the training corpus, introducing a novel pre-training approach, and devising a unique generation process. We demonstrate the effectiveness and controllability of ControllableGPT by conducting experiments on drug optimization tasks for both viral and cancer benchmarks, surpassing competing baselines.

Related papers

• ScaffoldGPT: A Scaffold-based Large Language Model for Drug Improvement [2.6198448284771443]
  ScaffoldGPT is a novel Large Language Model (LLM) designed for drug optimization based on molecular scaffolds. Our work comprises three key components: (1) A three-stage drug optimization approach that integrates pretraining, finetuning, and decoding optimization.
  arXiv  Detail & Related papers  (2025-02-09T10:36:33Z)
• Zero-Shot Strategies for Length-Controllable Summarization [56.15356055672189]
  Large language models (LLMs) struggle with precise length control, particularly in zero-shot settings. We conduct a comprehensive study evaluating LLMs' length control capabilities across multiple measures and propose practical methods to improve controllability. Our experiments with LLaMA 3 reveal stark differences in length adherence across measures and highlight inherent biases of the model.
  arXiv  Detail & Related papers  (2024-12-31T02:53:27Z)
• Controlling the Mutation in Large Language Models for the Efficient Evolution of Algorithms [2.2485774453793037]
  This paper introduces a novel approach to mutation control within evolutionary frameworks, inspired by theory of genetic algorithms. We propose dynamic mutation prompts that adaptively regulate mutation rates, leveraging a heavy-tailed power-law distribution to balance exploration and exploitation. Experiments show that the introduction of these dynamic rates can improve the convergence speed and adaptability of LLaMEA.
  arXiv  Detail & Related papers  (2024-12-04T11:49:22Z)
• Preparing Spin Squeezed States via Adaptive Genetic Algorithm [9.168152138847445]
  We introduce a novel strategy employing an adaptive genetic algorithm (GA) for iterative optimization of control sequences to generate quantum nonclassical states. Inspired by Darwinian evolution, the algorithm iteratively refines control sequences using crossover, mutation, and elimination strategies. Our approach, compared to constant control schemes, yields a variety of control sequences capable of maintaining squeezing for the collective spin model.
  arXiv  Detail & Related papers  (2024-10-20T12:15:11Z)
• Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction [88.65168366064061]
  We introduce Discrete Denoising Posterior Prediction (DDPP), a novel framework that casts the task of steering pre-trained MDMs as a problem of probabilistic inference. Our framework leads to a family of three novel objectives that are all simulation-free, and thus scalable. We substantiate our designs via wet-lab validation, where we observe transient expression of reward-optimized protein sequences.
  arXiv  Detail & Related papers  (2024-10-10T17:18:30Z)
• Adaptable Logical Control for Large Language Models [68.27725600175013]
  Ctrl-G is an adaptable framework that facilitates tractable and flexible control of model generation at inference time. We show that Ctrl-G, when applied to a TULU2-7B model, outperforms GPT3.5 and GPT4 on the task of interactive text editing.
  arXiv  Detail & Related papers  (2024-06-19T23:47:59Z)
• DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization [49.85944390503957]
  DecompOpt is a structure-based molecular optimization method based on a controllable and diffusion model. We show that DecompOpt can efficiently generate molecules with improved properties than strong de novo baselines.
  arXiv  Detail & Related papers  (2024-03-07T02:53:40Z)
• DECIDER: A Dual-System Rule-Controllable Decoding Framework for Language Generation [57.07295906718989]
  Constrained decoding approaches aim to control the meaning or style of text generated by a Pre-trained Language Model (PLM) using specific target words during inference. We propose a novel decoding framework, DECIDER, which enables us to program rules on how we complete tasks to control a PLM.
  arXiv  Detail & Related papers  (2024-03-04T11:49:08Z)
• Multi-modal Self-supervised Pre-training for Regulatory Genome Across Cell Types [75.65676405302105]
  We propose a simple yet effective approach for pre-training genome data in a multi-modal and self-supervised manner, which we call GeneBERT. We pre-train our model on the ATAC-seq dataset with 17 million genome sequences.
  arXiv  Detail & Related papers  (2021-10-11T12:48:44Z)
• Multi-Level Evolution Strategies for High-Resolution Black-Box Control [0.2320417845168326]
  This paper introduces a multi-level (m-lev) mechanism into Evolution Strategies (ESs) It addresses a class of global optimization problems that could benefit from fine discretization of their decision variables.
  arXiv  Detail & Related papers  (2020-10-04T09:24:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.