eccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence Analysis

• URL: http://arxiv.org/abs/2506.18940v1
• Date: Sun, 22 Jun 2025 17:50:57 GMT
• Title: eccDNAMamba: A Pre-Trained Model for Ultra-Long eccDNA Sequence Analysis
• Abstract summary: Extrachromosomal circular DNA (eccDNA) plays key regulatory roles and contributes to oncogene overexpression in cancer.<n>No pre-trained models currently support full-length circular eccDNA for downstream analysis.<n>eccDNAMamba is the first bidirectional state-space encoder tailored for circular DNA sequences.
• Score: 5.86106644437914
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Extrachromosomal circular DNA (eccDNA) plays key regulatory roles and contributes to oncogene overexpression in cancer through high-copy amplification and long-range interactions. Despite advances in modeling, no pre-trained models currently support full-length circular eccDNA for downstream analysis. Existing genomic models are either limited to single-nucleotide resolution or hindered by the inefficiency of the quadratic attention mechanism. Here, we introduce eccDNAMamba, the first bidirectional state-space encoder tailored for circular DNA sequences. It combines forward and reverse passes for full-context representation learning with linear-time complexity, and preserves circular structure through a novel augmentation strategy. Tested on two real-world datasets, eccDNAMamba achieves strong classification performance and scales to sequences up to 200 Kbp, offering a robust and efficient framework for modeling circular genomes. Our codes are available at https://github.com/zzq1zh/GenAI-Lab.

Related papers

• Coupling Models for One-Step Discrete Generation [50.37847093567075]
  Coupling Models is a one-step discrete generative model that learns a direct coupling between discrete sequences and Gaussian latents.<n>It avoids complex continuous flows over the simplex and hand-specified data-to-noise couplings.<n>It reduces LM1B text-generation perplexity by 33% at its lowest-perplexity operating point.
  arXiv  Detail & Related papers  (2026-05-08T03:40:39Z)
• How Private Are DNA Embeddings? Inverting Foundation Model Representations of Genomic Sequences [0.45880283710344055]
  DNA foundation models have become transformative tools in bioinformatics and healthcare applications.<n>Embeddings-as-a-Service (EBERT) frameworks are increasingly being shared via Embeddings-as-a-Service.<n>This study evaluates the resilience of DNA foundation models to model inversion attacks.
  arXiv  Detail & Related papers  (2026-03-06T23:52:26Z)
• dnaHNet: A Scalable and Hierarchical Foundation Model for Genomic Sequence Learning [16.347455063831067]
  dnaHNet is a tokenizer-free autoregressive model that segments and models genomic sequences end-to-end.<n>Pretrained on prokaryotic genomes, dnaHNet outperforms leading architectures including StripedHyena2 in scaling and efficiency.<n>On zero-shot tasks, dnaHNet achieves superior performance in predicting protein variant fitness and gene essentiality, while automatically discovering hierarchical biological structures without supervision.
  arXiv  Detail & Related papers  (2026-02-11T07:48:21Z)
• Bidirectional Representations Augmented Autoregressive Biological Sequence Generation:Application in De Novo Peptide Sequencing [51.12821379640881]
  Autoregressive (AR) models offer holistic, bidirectional representations but face challenges with generative coherence and scalability.<n>We propose a hybrid framework enhancing AR generation by dynamically integrating rich contextual information from non-autoregressive mechanisms.<n>A novel cross-decoder attention module enables the AR decoder to iteratively query and integrate these bidirectional features.
  arXiv  Detail & Related papers  (2025-10-09T12:52:55Z)
• Hyperbolic Genome Embeddings [0.6656737591902598]
  We develop a novel application of hyperbolic CNNs that exploits the evolutionarily-informed structure of biological systems.<n>Our strategy circumvents the need for explicit phylogenetic mapping while discerning key properties of sequences.<n>Our approach even surpasses state-of-the-art performance on seven GUE benchmark datasets.
  arXiv  Detail & Related papers  (2025-07-29T10:06:17Z)
• Sequential-Parallel Duality in Prefix Scannable Models [68.39855814099997]
  Recent developments have given rise to various models, such as Gated Linear Attention (GLA) and Mamba.<n>This raises a natural question: can we characterize the full class of neural sequence models that support near-constant-time parallel evaluation and linear-time, constant-space sequential inference?
  arXiv  Detail & Related papers  (2025-06-12T17:32:02Z)
• JanusDNA: A Powerful Bi-directional Hybrid DNA Foundation Model [1.6128508494592848]
  Large language models (LLMs) have revolutionized natural language processing and are increasingly applied to other sequential data types.<n>We introduce JanusDNA, the first bidirectional DNA foundation model built upon a novel pretraining paradigm.<n>JanusDNA processes up to 1 million base pairs at single nucleotide resolution on a single 80GB GPU.
  arXiv  Detail & Related papers  (2025-05-22T20:10:55Z)
• Learning Genomic Structure from $k$-mers [2.07180164747172]
  We present a method for analyzing read data using contrastive learning.<n>An encoder model is trained to produce embeddings that cluster together sequences from the same genomic region.<n>The model can also be trained fully self-supervised on read data, enabling analysis without the need to construct a full genome assembly.
  arXiv  Detail & Related papers  (2025-05-22T13:46:18Z)
• Regulatory DNA sequence Design with Reinforcement Learning [56.20290878358356]
  We propose a generative approach that leverages reinforcement learning to fine-tune a pre-trained autoregressive model.<n>We evaluate our method on promoter design tasks in two yeast media conditions and enhancer design tasks for three human cell types.
  arXiv  Detail & Related papers  (2025-03-11T02:33:33Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters.<n>Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks.<n>It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Large-Scale Targeted Cause Discovery via Learning from Simulated Data [66.51307552703685]
  We propose a novel machine learning approach for inferring causal variables of a target variable from observations.<n>We train a neural network using supervised learning on simulated data to infer causality.<n> Empirical results demonstrate superior performance in identifying causal relationships within large-scale gene regulatory networks.
  arXiv  Detail & Related papers  (2024-08-29T02:21:11Z)
• DNABERT-S: Pioneering Species Differentiation with Species-Aware DNA Embeddings [7.822348354050447]
  We introduce DNABERT-S, a tailored genome model that develops species-aware embeddings to naturally cluster and segregate DNA sequences of different species. Emerged results on 23 diverse datasets show DNABERT-S's effectiveness, especially in realistic label-scarce scenarios.
  arXiv  Detail & Related papers  (2024-02-13T20:21:29Z)
• Embed-Search-Align: DNA Sequence Alignment using Transformer Models [2.48439258515764]
  We bridge the gap by framing the sequence alignment task for Transformer models as an "Embed-Search-Align" task.<n>A novel Reference-Free DNA Embedding model generates embeddings of reads and reference fragments, which are projected into a shared vector space.<n>DNA-ESA is 99% accurate when aligning 250-length reads onto a human genome (3gb), rivaling conventional methods such as Bowtie and BWA-Mem.
  arXiv  Detail & Related papers  (2023-09-20T06:30:39Z)
• DNAGPT: A Generalized Pre-trained Tool for Versatile DNA Sequence Analysis Tasks [14.931476374660944]
  DNAGPT is a generalized DNA pre-training model trained on over 200 billion base pairs from all mammals. By enhancing the classic GPT model with a binary classification task, a numerical regression task, and a comprehensive token language, DNAGPT can handle versatile DNA analysis tasks.
  arXiv  Detail & Related papers  (2023-07-11T06:30:43Z)
• HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution [76.97231739317259]
  We present HyenaDNA, a genomic foundation model pretrained on the human reference genome with context lengths of up to 1 million tokens at the single nucleotide-level. On fine-tuned benchmarks from the Nucleotide Transformer, HyenaDNA reaches state-of-the-art (SotA) on 12 of 18 datasets using a model with orders of magnitude less parameters and pretraining data.
  arXiv  Detail & Related papers  (2023-06-27T20:46:34Z)
• Mutual Exclusivity Training and Primitive Augmentation to Induce Compositionality [84.94877848357896]
  Recent datasets expose the lack of the systematic generalization ability in standard sequence-to-sequence models. We analyze this behavior of seq2seq models and identify two contributing factors: a lack of mutual exclusivity bias and the tendency to memorize whole examples. We show substantial empirical improvements using standard sequence-to-sequence models on two widely-used compositionality datasets.
  arXiv  Detail & Related papers  (2022-11-28T17:36:41Z)
• DNA-GCN: Graph convolutional networks for predicting DNA-protein binding [4.1600531290054]
  We build a sequence k-mer graph and learn DNA Graph Convolutional Network (DNA-GCN) for the whole dataset. DNA-GCN is with a one-hot representation for all nodes, and it then jointly learns the embeddings for both k-mers and sequences. We evaluate our model on 50 datasets from ENCODE.
  arXiv  Detail & Related papers  (2021-06-02T07:36:11Z)
• Measuring Model Complexity of Neural Networks with Curve Activation Functions [100.98319505253797]
  We propose the linear approximation neural network (LANN) to approximate a given deep model with curve activation function. We experimentally explore the training process of neural networks and detect overfitting. We find that the $L1$ and $L2$ regularizations suppress the increase of model complexity.
  arXiv  Detail & Related papers  (2020-06-16T07:38:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.