ScaffoldGPT: A Scaffold-based Large Language Model for Drug Improvement

• URL: http://arxiv.org/abs/2502.06891v1
• Date: Sun, 09 Feb 2025 10:36:33 GMT
• Title: ScaffoldGPT: A Scaffold-based Large Language Model for Drug Improvement
• Authors: Xuefeng Liu, Songhao Jiang, Rick Stevens,
• Abstract summary: ScaffoldGPT is a novel Large Language Model (LLM) designed for drug optimization based on molecular scaffolds. Our work comprises three key components: (1) A three-stage drug optimization approach that integrates pretraining, finetuning, and decoding optimization.
• Score: 2.6198448284771443
• License:
• Abstract: Drug optimization has become increasingly crucial in light of fast-mutating virus strains and drug-resistant cancer cells. Nevertheless, it remains challenging as it necessitates retaining the beneficial properties of the original drug while simultaneously enhancing desired attributes beyond its scope. In this work, we aim to tackle this challenge by introducing ScaffoldGPT, a novel Large Language Model (LLM) designed for drug optimization based on molecular scaffolds. Our work comprises three key components: (1) A three-stage drug optimization approach that integrates pretraining, finetuning, and decoding optimization. (2) A uniquely designed two-phase incremental training approach for pre-training the drug optimization LLM-based generator on molecule scaffold with enhanced performance. (3) A token-level decoding optimization strategy, TOP-N, that enabling controlled, reward-guided generation using pretrained/finetuned LLMs. Finally, by conducting a comprehensive evaluation on COVID and cancer benchmarks, we demonstrate that SCAFFOLDGPT outperforms the competing baselines in drug optimization benchmarks, while excelling in preserving the original functional scaffold and enhancing desired properties.

Related papers

• ControllableGPT: A Ground-Up Designed Controllable GPT for Molecule Optimization [6.900025190052277]
  We introduce ControllableGPT, a controllable training framework for large language models. It is inspired by the biological processes of growth and evolution, which involve the expansion, shrinking, and mutation of sequences. It enables the precise management of specific locations and ranges within a sequence, while maintaining the integrity of any specified positions or subsequences.
  arXiv  Detail & Related papers  (2025-02-15T01:49:35Z)
• DrugImproverGPT: A Large Language Model for Drug Optimization with Fine-Tuning via Structured Policy Optimization [53.27954325490941]
  Finetuning a Large Language Model (LLM) is crucial for generating results towards specific objectives. This research introduces a novel reinforcement learning algorithm to finetune a drug optimization LLM-based generative model.
  arXiv  Detail & Related papers  (2025-02-11T04:00:21Z)
• The Dual-use Dilemma in LLMs: Do Empowering Ethical Capacities Make a Degraded Utility? [54.18519360412294]
  Large Language Models (LLMs) must balance between rejecting harmful requests for safety and accommodating legitimate ones for utility. This paper presents a Direct Preference Optimization (DPO) based alignment framework that achieves better overall performance. Our resulting model, LibraChem, outperforms leading LLMs including Claude-3, GPT-4o, and LLaMA-3 by margins of 13.44%, 7.16%, and 7.10% respectively.
  arXiv  Detail & Related papers  (2025-01-20T06:35:01Z)
• Structure-Based Molecule Optimization via Gradient-Guided Bayesian Update [16.743187639189976]
  Structure-based molecule optimization (SBMO) aims to optimize molecules with both continuous coordinates and discrete types against protein targets. MolJO is the first gradient-based SBMO framework that facilitates joint guidance signals across different modalities. MolJO achieves state-of-the-art performance on CrossDocked 2020 benchmark.
  arXiv  Detail & Related papers  (2024-11-20T12:48:29Z)
• Efficient Continual Pre-training by Mitigating the Stability Gap [68.49269649759005]
  We study the behavior of Large Language Models (LLMs) during continual pre-training. We propose three effective strategies to enhance LLM performance within a fixed compute budget. Our strategies improve the average medical task performance of the OpenLlama-3B model from 36.2% to 40.7% with only 40% of the original training budget.
  arXiv  Detail & Related papers  (2024-06-21T02:28:37Z)
• LightCPPgen: An Explainable Machine Learning Pipeline for Rational Design of Cell Penetrating Peptides [0.32985979395737786]
  We introduce an innovative approach for the de novo design of CPPs, leveraging the strengths of machine learning (ML) and optimization algorithms. Our strategy, named Light CPPgen, integrates a LightGBM-based predictive model with a genetic algorithm (GA) The GA solutions specifically target the candidate sequences' penetrability score, while trying to maximize similarity with the original non-penetrating peptide.
  arXiv  Detail & Related papers  (2024-05-31T10:57:25Z)
• LLM as a Complementary Optimizer to Gradient Descent: A Case Study in Prompt Tuning [69.95292905263393]
  We show that gradient-based and high-level LLMs can effectively collaborate a combined optimization framework. In this paper, we show that these complementary to each other and can effectively collaborate a combined optimization framework.
  arXiv  Detail & Related papers  (2024-05-30T06:24:14Z)
• Deep Lead Optimization: Leveraging Generative AI for Structural Modification [12.167178956742113]
  This review delves into the basic concepts, goals, conventional CADD techniques, and recent advancements in AIDD. We introduce a unified perspective based on constrained subgraph generation to harmonize the methodologies of de novo design and lead optimization.
  arXiv  Detail & Related papers  (2024-04-30T03:17:42Z)
• DecompOpt: Controllable and Decomposed Diffusion Models for Structure-based Molecular Optimization [49.85944390503957]
  DecompOpt is a structure-based molecular optimization method based on a controllable and diffusion model. We show that DecompOpt can efficiently generate molecules with improved properties than strong de novo baselines.
  arXiv  Detail & Related papers  (2024-03-07T02:53:40Z)
• Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark [166.40879020706151]
  This paper proposes a shift towards BP-free, zeroth-order (ZO) optimization as a solution for reducing memory costs during fine-tuning. Unlike traditional ZO-SGD methods, our work expands the exploration to a wider array of ZO optimization techniques. Our study unveils previously overlooked optimization principles, highlighting the importance of task alignment, the role of the forward gradient method, and the balance between algorithm complexity and fine-tuning performance.
  arXiv  Detail & Related papers  (2024-02-18T14:08:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.