Cross-Granularity Representations for Biological Sequences: Insights from ESM and BiGCARP

• URL: http://arxiv.org/abs/2603.20825v1
• Date: Sat, 21 Mar 2026 13:57:24 GMT
• Title: Cross-Granularity Representations for Biological Sequences: Insights from ESM and BiGCARP
• Abstract summary: We investigate the integration of cross-granularity knowledge from models through a case study of BiGCARP, a Pfam domain-level model for biosynthetic gene clusters, and ESM, an amino acid-level protein language model.<n>Our findings highlight cross-granularity integration as a promising strategy for improving the performance and interpretability of biological foundation models.
• Score: 6.039609675618599
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in general-purpose foundation models have stimulated the development of large biological sequence models. While natural language shows symbolic granularity (characters, words, sentences), biological sequences exhibit hierarchical granularity whose levels (nucleotides, amino acids, protein domains, genes) further encode biologically functional information. In this paper, we investigate the integration of cross-granularity knowledge from models through a case study of BiGCARP, a Pfam domain-level model for biosynthetic gene clusters, and ESM, an amino acid-level protein language model. Using representation analysis tools and a set of probe tasks, we first explain why a straightforward cross-model embedding initialization fails to improve downstream performance in BiGCARP, and show that deeper-layer embeddings capture a more contextual and faithful representation of the model's learned knowledge. Furthermore, we demonstrate that representations at different granularities encode complementary biological knowledge, and that combining them yields measurable performance gains in intermediate-level prediction tasks. Our findings highlight cross-granularity integration as a promising strategy for improving both the performance and interpretability of biological foundation models.

Related papers

• Advancing Cancer Prognosis with Hierarchical Fusion of Genomic, Proteomic and Pathology Imaging Data from a Systems Biology Perspective [28.350872729693847]
  We propose HFGPI, a hierarchical fusion framework that models the biological progression from genes to proteins to histology images from a systems biology perspective.<n>Specifically, we introduce Molecular Tokenizer, a molecular encoding strategy that integrates identity embeddings with expression profiles to construct biologically informed representations for genes and proteins.<n>We then develop Gene-Regulated Protein Fusion (GRPF), which employs graph-aware cross-attention with structure-protein alignment to explicitly model gene-preserving regulatory relationships and generate gene-regulated protein representations.
  arXiv  Detail & Related papers  (2026-03-14T06:30:51Z)
• When Genes Speak: A Semantic-Guided Framework for Spatially Resolved Transcriptomics Data Clustering [26.67465778995387]
  SemST is a semantic-guided deep learning framework for spatial transcriptomics data clustering.<n>FSM module learns spot-specific affine transformations that empower the semantic embeddings to perform an element-wise calibration of the spatial features.<n> experiments on public spatial transcriptomics datasets show that SemST achieves state-of-the-art clustering performance.
  arXiv  Detail & Related papers  (2025-11-14T15:03:41Z)
• Discovering Interpretable Biological Concepts in Single-cell RNA-seq Foundation Models [3.810388351528255]
  Single-cell RNA-seq foundation models achieve strong performance on downstream tasks but remain black boxes.<n>Recent work has shown that sparse dictionary learning can extract concepts from deep learning models.<n>We introduce a novel concept-based interpretability framework for single-cell RNA-seq models.
  arXiv  Detail & Related papers  (2025-10-29T08:52:55Z)
• Life-Code: Central Dogma Modeling with Multi-Omics Sequence Unification [55.98854157265578]
  Life-Code is a comprehensive framework that spans different biological functions.<n>We propose a unified pipeline to integrate multi-omics data by reverse-transcribing RNA and reverse-translating amino acids into nucleotide-based sequences.<n>Life-Code achieves state-of-the-art results on various tasks across three omics, highlighting its potential for advancing multi-omics analysis and interpretation.
  arXiv  Detail & Related papers  (2025-02-11T06:53:59Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters.<n>Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks.<n>It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models [55.74944165932666]
  We introduce Biology-Instructions, the first large-scale instruction-tuning dataset for multi-omics biological sequences.<n>This dataset bridges large language models (LLMs) and complex biological sequence-related tasks, enhancing their versatility and reasoning.<n>We also highlight significant limitations of current state-of-the-art LLMs on multi-omics tasks without specialized training.
  arXiv  Detail & Related papers  (2024-12-26T12:12:23Z)
• An Evaluation of Large Language Models in Bioinformatics Research [52.100233156012756]
  We study the performance of large language models (LLMs) on a wide spectrum of crucial bioinformatics tasks. These tasks include the identification of potential coding regions, extraction of named entities for genes and proteins, detection of antimicrobial and anti-cancer peptides, molecular optimization, and resolution of educational bioinformatics problems. Our findings indicate that, given appropriate prompts, LLMs like GPT variants can successfully handle most of these tasks.
  arXiv  Detail & Related papers  (2024-02-21T11:27:31Z)
• Neural Echos: Depthwise Convolutional Filters Replicate Biological Receptive Fields [56.69755544814834]
  We present evidence suggesting that depthwise convolutional kernels are effectively replicating the biological receptive fields observed in the mammalian retina. We propose a scheme that draws inspiration from the biological receptive fields.
  arXiv  Detail & Related papers  (2024-01-18T18:06:22Z)
• Improving Biomedical Entity Linking with Retrieval-enhanced Learning [53.24726622142558]
  $k$NN-BioEL provides a BioEL model with the ability to reference similar instances from the entire training corpus as clues for prediction. We show that $k$NN-BioEL outperforms state-of-the-art baselines on several datasets.
  arXiv  Detail & Related papers  (2023-12-15T14:04:23Z)
• Tertiary Lymphoid Structures Generation through Graph-based Diffusion [54.37503714313661]
  In this work, we leverage state-of-the-art graph-based diffusion models to generate biologically meaningful cell-graphs. We show that the adopted graph diffusion model is able to accurately learn the distribution of cells in terms of their tertiary lymphoid structures (TLS) content.
  arXiv  Detail & Related papers  (2023-10-10T14:37:17Z)
• A Biologically Interpretable Two-stage Deep Neural Network (BIT-DNN) For Vegetation Recognition From Hyperspectral Imagery [3.708283803668841]
  This study proposes a novel interpretable deep learning model -- a biologically interpretable two-stage deep neural network (BIT-DNN) The proposed model has been compared with five state-of-the-art deep learning models.
  arXiv  Detail & Related papers  (2020-04-19T15:58:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.