Steering Protein Language Models

• URL: http://arxiv.org/abs/2509.07983v2
• Date: Fri, 12 Sep 2025 12:39:45 GMT
• Title: Steering Protein Language Models
• Authors: Long-Kai Huang, Rongyi Zhu, Bing He, Jianhua Yao,
• Abstract summary: Activation Steering is a technique originally developed for controlling text generation in Large Language Models.<n>We propose a simple yet effective method that employs activation editing to steer PLM outputs.<n>We show that our methods can be seamlessly integrated into both auto-encoding and autoregressive PLMs without requiring additional training.
• Score: 22.308373820985793
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Protein Language Models (PLMs), pre-trained on extensive evolutionary data from natural proteins, have emerged as indispensable tools for protein design. While powerful, PLMs often struggle to produce proteins with precisely specified functionalities or properties due to inherent challenges in controlling their outputs. In this work, we investigate the potential of Activation Steering, a technique originally developed for controlling text generation in Large Language Models (LLMs), to direct PLMs toward generating protein sequences with targeted properties. We propose a simple yet effective method that employs activation editing to steer PLM outputs, and extend this approach to protein optimization through a novel editing site identification module. Through comprehensive experiments on lysozyme-like sequence generation and optimization, we demonstrate that our methods can be seamlessly integrated into both auto-encoding and autoregressive PLMs without requiring additional training. These results highlight a promising direction for precise protein engineering using foundation models.

Related papers

• Self Distillation Fine-Tuning of Protein Language Models Improves Versatility in Protein Design [61.2846583160056]
  Supervised fine-tuning (SFT) is a standard approach for adapting large language models to specialized domains.<n>This is in part because high-quality annotated data are far more difficult to obtain for proteins than for natural language.<n>We present a simple and general recipe for fast SFT of PLMs, designed to improve the fidelity, reliability, and novelty of generated protein sequences.
  arXiv  Detail & Related papers  (2025-12-10T05:34:47Z)
• ProtTeX: Structure-In-Context Reasoning and Editing of Proteins with Large Language Models [8.520384176663423]
  Large language models have made remarkable progress in the field of molecular science.<n>In protein science, the amino acid sequence serves as the sole tokenizer for LLMs.<n>We introduce a novel framework, Prot, which tokenizes the protein sequences, structures, and textual information into a unified discrete space.
  arXiv  Detail & Related papers  (2025-03-11T08:43:05Z)
• Controllable Protein Sequence Generation with LLM Preference Optimization [19.28325662879149]
  We propose a novel controllable protein design method called CtrlProt.<n> Experiments demonstrate that CtrlProt can meet functionality and structural stability requirements effectively.
  arXiv  Detail & Related papers  (2025-01-25T00:59:12Z)
• Computational Protein Science in the Era of Large Language Models (LLMs) [54.35488233989787]
  Computational protein science is dedicated to revealing knowledge and developing applications within the protein sequence-structure-function paradigm.<n>Recently, Language Models (pLMs) have emerged as a milestone in AI due to their unprecedented language processing & generalization capability.
  arXiv  Detail & Related papers  (2025-01-17T16:21:18Z)
• Large Language Model is Secretly a Protein Sequence Optimizer [24.55348363931866]
  We consider the protein sequence engineering problem, which aims to find protein sequences with high fitness levels, starting from a given wild-type sequence.<n>We demonstrate large language models (LLMs), despite being trained on massive texts, are secretly protein sequences.
  arXiv  Detail & Related papers  (2025-01-16T03:44:16Z)
• Structure-Enhanced Protein Instruction Tuning: Towards General-Purpose Protein Understanding with LLMs [43.811432723460534]
  We introduce Structure-Enhanced Protein Instruction Tuning (SEPIT) framework to bridge this gap.<n>Our approach incorporates a novel structure-aware module into pLMs to enrich their structural knowledge.<n>We construct the largest and most comprehensive protein instruction dataset to date, which allows us to train and evaluate the general-purpose protein understanding model.
  arXiv  Detail & Related papers  (2024-10-04T16:02:50Z)
• ProLLM: Protein Chain-of-Thoughts Enhanced LLM for Protein-Protein Interaction Prediction [54.132290875513405]
  The prediction of protein-protein interactions (PPIs) is crucial for understanding biological functions and diseases. Previous machine learning approaches to PPI prediction mainly focus on direct physical interactions. We propose a novel framework ProLLM that employs an LLM tailored for PPI for the first time.
  arXiv  Detail & Related papers  (2024-03-30T05:32:42Z)
• ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training [82.37346937497136]
  We propose a versatile cross-modal large language model (LLM) for both protein-centric and protein-language tasks. ProtLLM features a unique dynamic protein mounting mechanism, enabling it to handle complex inputs. By developing a specialized protein vocabulary, we equip the model with the capability to predict not just natural language but also proteins from a vast pool of candidates.
  arXiv  Detail & Related papers  (2024-02-28T01:29:55Z)
• xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein [74.64101864289572]
  We propose a unified protein language model, xTrimoPGLM, to address protein understanding and generation tasks simultaneously.<n>xTrimoPGLM significantly outperforms other advanced baselines in 18 protein understanding benchmarks across four categories.<n>It can also generate de novo protein sequences following the principles of natural ones, and can perform programmable generation after supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-01-11T15:03:17Z)
• Structure-informed Language Models Are Protein Designers [69.70134899296912]
  We present LM-Design, a generic approach to reprogramming sequence-based protein language models (pLMs) We conduct a structural surgery on pLMs, where a lightweight structural adapter is implanted into pLMs and endows it with structural awareness. Experiments show that our approach outperforms the state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2023-02-03T10:49:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.