Two-Stage Fine-Tuning for Protein Sequence Generation with Targeted Amino-Acid Composition

• URL: http://arxiv.org/abs/2606.27939v1
• Date: Fri, 26 Jun 2026 10:29:42 GMT
• Title: Two-Stage Fine-Tuning for Protein Sequence Generation with Targeted Amino-Acid Composition
• Abstract summary: We study a constrained protein generation problem in which sequences must match a desired amino-acid composition profile.<n>The motivating application is synthetic feed protein design, where the AA composition of dietary proteins directly determines their nutritional value.
• Score: 0.007996601065633402
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Protein language models are standard priors for biological sequence generation, but steering them toward explicit distributional design targets remains largely unexplored. We study a constrained protein generation problem in which sequences must match a desired amino-acid (AA) composition profile while preserving plausible sequence statistics and diversity. The motivating application is synthetic feed protein design, where the AA composition of dietary proteins directly determines their nutritional value. We propose a two-stage pipeline in which domain-adaptive fine-tuning (FT) on an in-domain protein dataset is followed by iterative reward-weighted FT via reinforcement learning (RL) anchored against the FT model as a frozen reference. We evaluate the pipeline on two AA compositions and find that FT brings the average composition close to the target, while the subsequent RL enforces specific sequence constraints that FT alone cannot satisfy. We additionally evaluate the design choices of the proposed composition reward term against two baselines and an ablated variant, isolate the contribution of each training stage, and verify that AA composition alignment is achieved without degrading sequence quality.

Related papers

• SymFold: Synergizing Evolutionary and Structural Priors for Accurate Protein Inverse Folding [30.025725962157498]
  Protein inverse folding aims to recover amino acid sequences for a given 3D protein structure.<n>Current methods often follow a serial pipeline, in which a structure encoder predicts a coarse sequence, which is then refined by protein language models (PLMs)<n>Thanks to recent multimodal protein language models (MPLMs), we could directly encode structure to generate sequences with pretrained structural knowledge, but we observe that they are not effective for inverse folding.<n>We introduce a symmetric dual-path architecture that both leverages PLMs for pretrained sequence evolution knowledge and MPLMs for pretrained structural knowledge to iteratively guide protein sequence generation.
  arXiv  Detail & Related papers  (2026-09-01T14:57:24Z)
• AMix-2: Establishing Protein as a Native Modality in Large Language Models [88.00237394496698]
  We present AMix-2, a protein-text foundation model that establishes protein as a native modality in large language models (LLMs)<n>AMix-2 is built upon two key ideas: (1) a unified protein-text formulation that embeds natural language and protein sequence in a shared token space; and (2) a block-wise diffusion language modeling backbone.<n>We introduce ProteinArena, a comprehensive benchmark with time-aware and homology-aware protocols across various understanding and design tasks.
  arXiv  Detail & Related papers  (2026-05-29T07:58:08Z)
• Self Distillation Fine-Tuning of Protein Language Models Improves Versatility in Protein Design [61.2846583160056]
  Supervised fine-tuning (SFT) is a standard approach for adapting large language models to specialized domains.<n>This is in part because high-quality annotated data are far more difficult to obtain for proteins than for natural language.<n>We present a simple and general recipe for fast SFT of PLMs, designed to improve the fidelity, reliability, and novelty of generated protein sequences.
  arXiv  Detail & Related papers  (2025-12-10T05:34:47Z)
• S$^2$Drug: Bridging Protein Sequence and 3D Structure in Contrastive Representation Learning for Virtual Screening [72.89086338778098]
  We propose a two-stage framework for protein-ligand contrastive representation learning.<n>In the first stage, we perform protein sequence pretraining on ChemBL using an ESM2-based backbone.<n>In the second stage, we fine-tune on PDBBind by fusing sequence and structure information through a residue-level gating module.<n>This auxiliary task guides the model to accurately localize binding residues within the protein sequence and capture their 3D spatial arrangement.
  arXiv  Detail & Related papers  (2025-11-10T11:57:47Z)
• NS-Pep: De novo Peptide Design with Non-Standard Amino Acids [15.931688895952234]
  Non-standard amino acids (NSAAs) offer improved binding affinity and improved pharmacological properties.<n>Existing peptide design methods are limited to standard amino acids, leaving NSAA-aware design largely unexplored.<n>We introduce NS-Pep, a unified framework for co-designing peptide sequences and structures with NSAAs.
  arXiv  Detail & Related papers  (2025-10-01T16:04:06Z)
• Lightweight MSA Design Advances Protein Folding From Evolutionary Embeddings [51.731441632457226]
  Multiple sequence alignments (MSAs) underperform on low-homology and orphan proteins.<n>We introduce PLAME, a lightweight MSA design framework that generates MSAs that better support downstream folding.<n>On AlphaFold2 low-homology/orphan benchmarks, PLAME delivers state-of-the-art improvements in structure accuracy.
  arXiv  Detail & Related papers  (2025-06-17T04:11:30Z)
• PPDiff: Diffusing in Hybrid Sequence-Structure Space for Protein-Protein Complex Design [20.033392739225658]
  PPDiff is a diffusion model to jointly design the sequence and structure of binders for arbitrary protein targets.<n> PPDiff builds upon our developed Sequence Structure Inter Network with Causal attention layers.<n>The model is pretrained on PPBench and finetuned on two real-world applications.
  arXiv  Detail & Related papers  (2025-06-13T02:39:14Z)
• Protriever: End-to-End Differentiable Protein Homology Search for Fitness Prediction [28.150437140009025]
  Protriever is an end-to-end differentiable framework that learns to retrieve relevant homologs while simultaneously training for the target task.<n>We introduce Protriever, an end-to-end differentiable framework that learns to retrieve relevant homologs while simultaneously training for the target task.
  arXiv  Detail & Related papers  (2025-06-10T16:24:09Z)
• Protein Inverse Folding From Structure Feedback [78.27854221882572]
  We introduce a novel approach to fine-tune an inverse folding model using feedback from a protein folding model.<n>Our results on the CATH 4.2 test set demonstrate that DPO fine-tuning leads to a significant improvement in average TM-Score.
  arXiv  Detail & Related papers  (2025-06-03T16:02:12Z)
• PSC-CPI: Multi-Scale Protein Sequence-Structure Contrasting for Efficient and Generalizable Compound-Protein Interaction Prediction [63.50967073653953]
  Compound-Protein Interaction prediction aims to predict the pattern and strength of compound-protein interactions for rational drug discovery. Existing deep learning-based methods utilize only the single modality of protein sequences or structures. We propose a novel multi-scale Protein Sequence-structure Contrasting framework for CPI prediction.
  arXiv  Detail & Related papers  (2024-02-13T03:51:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.