AMix-2: Establishing Protein as a Native Modality in Large Language Models

• URL: http://arxiv.org/abs/2605.30963v1
• Date: Fri, 29 May 2026 07:58:08 GMT
• Title: AMix-2: Establishing Protein as a Native Modality in Large Language Models
• Abstract summary: We present AMix-2, a protein-text foundation model that establishes protein as a native modality in large language models (LLMs)<n>AMix-2 is built upon two key ideas: (1) a unified protein-text formulation that embeds natural language and protein sequence in a shared token space; and (2) a block-wise diffusion language modeling backbone.<n>We introduce ProteinArena, a comprehensive benchmark with time-aware and homology-aware protocols across various understanding and design tasks.
• Score: 88.00237394496698
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We present AMix-2, a protein-text foundation model that establishes protein as a native modality in large language models (LLMs), unifying protein understanding and sequence design within a single foundation model. AMix-2 is built upon two key ideas: (1) a unified protein-text formulation that embeds natural language and protein sequence in a shared token space, enabling one model to perform biological reasoning and conditional design instead of separate downstream task-specialized models; and (2) a block-wise diffusion language modeling backbone that combines causal generation across blocks with bidirectional context and iterative refinement within blocks. This scheme better matches the intrinsic nature of proteins than a strict left-to-right factorization. To evaluate protein foundation models under realistic generalization settings, we further introduce ProteinArena, a comprehensive benchmark with time-aware and homology-aware protocols across various understanding and design tasks, and with baselines covering classical bioinformatics tools, protein-specialized models and LLMs. On ProteinArena, AMix-2 outperforms frontier LLMs and demonstrates competitive performance to task-specific protein models. Controlled experiments further show that the diffusion-based paradigm generally surpasses its autoregressive counterpart, highlighting the advantage of flexible generation order for protein sequences. We release both AMix-2 and ProteinArena to facilitate open research in protein foundation models.

Related papers

• Demystify Protein Generation with Hierarchical Conditional Diffusion Models [17.174551222714722]
  We propose a novel conditional diffusion model for efficient end-to-end protein design guided by specified functions.<n>By generating representations at different levels simultaneously, our framework can effectively model the inherent hierarchical relations between different levels.<n>We also propose a Protein-MMD, a new reliable evaluation metric, to evaluate the quality of generated protein.
  arXiv  Detail & Related papers  (2025-07-24T17:34:02Z)
• ProtDAT: A Unified Framework for Protein Sequence Design from Any Protein Text Description [7.198238666986253]
  We propose a de novo fine-grained framework capable of designing proteins from any descriptive text input.<n>Prot DAT builds upon the inherent characteristics of protein data to unify sequences and text as a cohesive whole rather than separate entities.<n> Experimental results demonstrate that Prot DAT achieves the state-of-the-art performance in protein sequence generation, excelling in rationality, functionality, structural similarity, and validity.
  arXiv  Detail & Related papers  (2024-12-05T11:05:46Z)
• Long-context Protein Language Modeling Using Bidirectional Mamba with Shared Projection Layers [76.95505296417866]
  Self-supervised training of language models (LMs) has seen great success for protein sequences in learning meaningful representations and for generative drug design.<n>Most protein LMs are based on the Transformer architecture trained on individual proteins with short context lengths.<n>In this work, we propose LC-PLM based on an alternative protein LM architecture, BiMamba-S, built upon selective structured state-space models.
  arXiv  Detail & Related papers  (2024-10-29T16:43:28Z)
• Design Proteins Using Large Language Models: Enhancements and Comparative Analyses [12.140433802768733]
  We adopt a suite of pre-trained LLMs, including Mistral-7B1, Llama-2-7B2, Llama-3-8B3, and gemma-7B4, to produce valid protein sequences. We retrain these models to process protein-related data, ensuring the generation of biologically feasible protein structures. Our findings demonstrate that even with limited data, the adapted models exhibit efficiency comparable to established protein-focused models.
  arXiv  Detail & Related papers  (2024-08-12T08:17:27Z)
• Diffusion on language model encodings for protein sequence generation [0.5088559194265662]
  DiMA is a latent diffusion framework that operates on protein language model representations.<n>It consistently produces novel, high-quality and diverse protein sequences.<n>It supports conditional generation tasks including protein family-generation, motif scaffolding and infilling, and fold-specific sequence design.
  arXiv  Detail & Related papers  (2024-03-06T14:15:20Z)
• ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training [82.37346937497136]
  We propose a versatile cross-modal large language model (LLM) for both protein-centric and protein-language tasks. ProtLLM features a unique dynamic protein mounting mechanism, enabling it to handle complex inputs. By developing a specialized protein vocabulary, we equip the model with the capability to predict not just natural language but also proteins from a vast pool of candidates.
  arXiv  Detail & Related papers  (2024-02-28T01:29:55Z)
• xTrimoPGLM: Unified 100B-Scale Pre-trained Transformer for Deciphering the Language of Protein [74.64101864289572]
  We propose a unified protein language model, xTrimoPGLM, to address protein understanding and generation tasks simultaneously.<n>xTrimoPGLM significantly outperforms other advanced baselines in 18 protein understanding benchmarks across four categories.<n>It can also generate de novo protein sequences following the principles of natural ones, and can perform programmable generation after supervised fine-tuning.
  arXiv  Detail & Related papers  (2024-01-11T15:03:17Z)
• Progressive Multi-Modality Learning for Inverse Protein Folding [47.095862120116976]
  We propose a novel protein design paradigm called MMDesign, which leverages multi-modality transfer learning. MMDesign is the first framework that combines a pretrained structural module with a pretrained contextual module, using an auto-encoder (AE) based language model to incorporate prior protein semantic knowledge. Experimental results, only training with the small dataset, demonstrate that MMDesign consistently outperforms baselines on various public benchmarks.
  arXiv  Detail & Related papers  (2023-12-11T10:59:23Z)
• A Latent Diffusion Model for Protein Structure Generation [50.74232632854264]
  We propose a latent diffusion model that can reduce the complexity of protein modeling. We show that our method can effectively generate novel protein backbone structures with high designability and efficiency.
  arXiv  Detail & Related papers  (2023-05-06T19:10:19Z)
• Protein Sequence and Structure Co-Design with Equivariant Translation [19.816174223173494]
  Existing approaches generate both protein sequence and structure using either autoregressive models or diffusion models. We propose a new approach capable of protein sequence and structure co-design, which iteratively translates both protein sequence and structure into the desired state. Our model consists of a trigonometry-aware encoder that reasons geometrical constraints and interactions from context features. All protein amino acids are updated in one shot in each translation step, which significantly accelerates the inference process.
  arXiv  Detail & Related papers  (2022-10-17T06:00:12Z)
• Learning Geometrically Disentangled Representations of Protein Folding Simulations [72.03095377508856]
  This work focuses on learning a generative neural network on a structural ensemble of a drug-target protein. Model tasks involve characterizing the distinct structural fluctuations of the protein bound to various drug molecules. Results show that our geometric learning-based method enjoys both accuracy and efficiency for generating complex structural variations.
  arXiv  Detail & Related papers  (2022-05-20T19:38:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.