Evolutionary Curriculum Learning Improves Biological Sequence Modeling

• URL: http://arxiv.org/abs/2608.00697v2
• Date: Sun, 09 Aug 2026 17:20:28 GMT
• Title: Evolutionary Curriculum Learning Improves Biological Sequence Modeling
• Abstract summary: Variational autoencoders (VAEs) have emerged as powerful generative models for biological sequences, with applications ranging from disease variant prediction to functional RNA design.<n>Standard biological VAE training treats all sequences as exchangeable, ignoring the rich evolutionary structure that organizes sequences from evolutionarily close to highly divergent.<n>We propose Evolutionary Learning Curriculum (ECL), a training strategy that exploits this structure by progressively exposing the model to sequences of increasing evolutionary distance from sampled anchors.
• Score: 6.89005543900307
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Variational autoencoders (VAEs) trained on multiple sequence alignments (MSAs) have emerged as powerful generative models for biological sequences, with applications ranging from disease variant prediction to functional RNA design. However, standard biological VAE training treats all sequences as exchangeable, ignoring the rich evolutionary structure that organizes homologous sequences from evolutionarily close to highly divergent. We propose Evolutionary Curriculum Learning (ECL), a training strategy that exploits this structure by progressively exposing the model to sequences of increasing evolutionary distance from sampled anchors, following a power-law expansion schedule. Applied to two architecturally distinct VAE models and two biological domains--protein variant effect prediction with EVE and RNA family sequence generation with RfamGen--ECL improves downstream task performance across five random seeds per configuration. Mean ClinVar classification AUROC rises from 0.981 to 0.989 for p53; for PTEN, ECL attains 1.000 in every seed whereas the baseline is unstable (mean 0.905, falling as low as 0.54). For RNA, ECL raises mean covariance-model bit scores on all three families tested and exceeds its seed-matched baseline in 12 of 15 training runs, though with only three families the effect cannot be established as significant at the family level. Ablation experiments show that progressively expanding the sampled sequences by evolutionary distance outperforms fixed-size neighborhood sampling in addition to uniform random sampling. Evolutionary distance is therefore a useful inductive bias for ordering the training curriculum in biological sequence modeling.

Related papers

• Rigidity-Aware Geometric Pretraining for Protein Design and Conformational Ensembles [74.32932832937618]
  We introduce $textbfRigidSSL$ ($textitRigidity-Aware Self-Supervised Learning$), a geometric pretraining framework that front-loads geometry learning prior to finetuning.<n>Phase I (RigidSSL-Perturb) learns geometric priors from 432K structures from the AlphaFold Protein Structure Database with simulated perturbations.<n>Phase II (RigidSSL-MD) refines these representations on 1.3K molecular dynamics trajectories to capture physically realistic transitions.
  arXiv  Detail & Related papers  (2026-03-02T21:32:30Z)
• STRAND: Sequence-Conditioned Transport for Single-Cell Perturbations [31.08466183513241]
  STRAND is a generative model that predicts single-cell responses by conditioning on regulatory DNA sequence.<n>Representing perturbations by sequence, rather than by a fixed set of gene identifiers, supports zero-shot inference at loci not seen during training.<n>We evaluate STRAND on CRISPR perturbation datasets in K562, Jurkat, and RPE1 cells.
  arXiv  Detail & Related papers  (2026-02-10T00:57:38Z)
• Evolutionary Profiles for Protein Fitness Prediction [45.945064429964084]
  EvoIF fuses sequence-structure representations with evolutionary signals to yield calibrated probabilities for log-odds scoring.<n>On ProteinGym (217 mutational assays; >2.5M mutants), EvoIF and its MSA-enabled variant achieve state-of-the-art or competitive performance while using only 0.15% of the training depths.
  arXiv  Detail & Related papers  (2025-10-08T17:46:02Z)
• Regulatory DNA sequence Design with Reinforcement Learning [56.20290878358356]
  We propose a generative approach that leverages reinforcement learning to fine-tune a pre-trained autoregressive model.<n>We evaluate our method on promoter design tasks in two yeast media conditions and enhancer design tasks for three human cell types.
  arXiv  Detail & Related papers  (2025-03-11T02:33:33Z)
• UniGenX: a unified generative foundation model that couples sequence, structure and function to accelerate scientific design across proteins, molecules and materials [62.72989417755985]
  We present UniGenX, a unified generative model for function in natural systems.<n>UniGenX represents heterogeneous inputs as a mixed stream of symbolic and numeric tokens.<n>It achieves state-of-the-art or competitive performance for the function-aware generation across domains.
  arXiv  Detail & Related papers  (2025-03-09T16:43:07Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters.<n>Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks.<n>It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• DPLM-2: A Multimodal Diffusion Protein Language Model [75.98083311705182]
  We introduce DPLM-2, a multimodal protein foundation model that extends discrete diffusion protein language model (DPLM) to accommodate both sequences and structures. DPLM-2 learns the joint distribution of sequence and structure, as well as their marginals and conditionals. Empirical evaluation shows that DPLM-2 can simultaneously generate highly compatible amino acid sequences and their corresponding 3D structures.
  arXiv  Detail & Related papers  (2024-10-17T17:20:24Z)
• Semantically Rich Local Dataset Generation for Explainable AI in Genomics [0.716879432974126]
  Black box deep learning models trained on genomic sequences excel at predicting the outcomes of different gene regulatory mechanisms. We propose using Genetic Programming to generate datasets by evolving perturbations in sequences that contribute to their semantic diversity.
  arXiv  Detail & Related papers  (2024-07-03T10:31:30Z)
• Learning to Predict Mutation Effects of Protein-Protein Interactions by Microenvironment-aware Hierarchical Prompt Learning [78.38442423223832]
  We develop a novel codebook pre-training task, namely masked microenvironment modeling. We demonstrate superior performance and training efficiency over state-of-the-art pre-training-based methods in mutation effect prediction.
  arXiv  Detail & Related papers  (2024-05-16T03:53:21Z)
• Unsupervised language models for disease variant prediction [3.6942566104432886]
  We find that a single protein LM trained on broad sequence datasets can score pathogenicity for any gene variant zero-shot. We show that it achieves scoring performance comparable to the state of the art when evaluated on clinically labeled variants of disease-related genes.
  arXiv  Detail & Related papers  (2022-12-07T22:28:13Z)
• EvoVGM: A Deep Variational Generative Model for Evolutionary Parameter Estimation [0.0]
  We propose a method for a deep variational Bayesian generative model that jointly approximates the true posterior of local biological evolutionary parameters. We show the consistency and effectiveness of the method on synthetic sequence alignments simulated with several evolutionary scenarios and on a real virus sequence alignment.
  arXiv  Detail & Related papers  (2022-05-25T20:08:10Z)
• Epigenetic evolution of deep convolutional models [81.21462458089142]
  We build upon a previously proposed neuroevolution framework to evolve deep convolutional models. We propose a convolutional layer layout which allows kernels of different shapes and sizes to coexist within the same layer. The proposed layout enables the size and shape of individual kernels within a convolutional layer to be evolved with a corresponding new mutation operator.
  arXiv  Detail & Related papers  (2021-04-12T12:45:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.