Fair splits flip the leaderboard: CHANRG reveals limited generalization in RNA secondary-structure prediction

• URL: http://arxiv.org/abs/2603.22330v1
• Date: Fri, 20 Mar 2026 20:18:08 GMT
• Title: Fair splits flip the leaderboard: CHANRG reveals limited generalization in RNA secondary-structure prediction
• Abstract summary: Current benchmarks may overestimate generalization across RNA families.<n>We present the Comprehensive Hierarchical.<n>Non-coding RNA Groups of Non-coding RNAs (CHANRG), a benchmark of 170,083 structurally non-redundant RNAs.
• Score: 44.072730684426794
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Accurate prediction of RNA secondary structure underpins transcriptome annotation, mechanistic analysis of non-coding RNAs, and RNA therapeutic design. Recent gains from deep learning and RNA foundation models are difficult to interpret because current benchmarks may overestimate generalization across RNA families. We present the Comprehensive Hierarchical Annotation of Non-coding RNA Groups (CHANRG), a benchmark of 170{,}083 structurally non-redundant RNAs curated from more than 10 million sequences in Rfam~15.0 using structure-aware deduplication, genome-aware split design and multiscale structural evaluation. Across 29 predictors, foundation-model methods achieved the highest held-out accuracy but lost most of that advantage out of distribution, whereas structured decoders and direct neural predictors remained markedly more robust. This gap persisted after controlling for sequence length and reflected both loss of structural coverage and incorrect higher-order wiring. Together, CHANRG and a padding-free, symmetry-aware evaluation stack provide a stricter and batch-invariant framework for developing RNA structure predictors with demonstrable out-of-distribution robustness.

Related papers

• RIBOSPAN: A Long-Context RNA Foundation Model for Versatile RNA Modeling [26.58224679878256]
  We present RIBOSPAN, a bidirectional RNA foundation model pretrained with context lengths up to 10,240 nt.<n> RIBOSPAN combines dense bidirectional self-attention, single-nucleotide tokenization, and attention-isolated sequence packing to enable high-resolution modeling of complete long RNAs.<n>Frozen RNA-type evaluations show that RIBOSPAN learns state-of-the-art RNA representations, with a particularly clear advantage on long RNAs.
  arXiv  Detail & Related papers  (2026-08-24T06:30:00Z)
• RiboSphere: Learning Unified and Efficient Representations of RNA Structures [57.40815107640066]
  RNA backbones are highly flexible, non-canonical interactions are prevalent, and experimentally determined 3D structures are comparatively scarce.<n>We introduce emphRiboSphere, a framework that learns emphdiscrete geometric representations of RNA by combining vector quantization with flow matching.
  arXiv  Detail & Related papers  (2026-03-20T04:36:35Z)
• Deep Learning Framework for RNA Inverse Folding with Geometric Structure Potentials [0.0]
  I introduce a deep learning framework that integrates Geometric Vector Perceptron layers with a Transformer architecture to enable end-to-end RNA design.<n>I construct a dataset consisting of experimentally solved RNA 3D structures, filtered and deduplicated from the BGSU RNA list, and evaluate performance using both sequence recovery rate and TM-score.<n>My model achieves state-of-the-art performance, with recovery and TM-scores of 0.481 and 0.332, surpassing existing methods across diverse RNA families and length scales.
  arXiv  Detail & Related papers  (2025-12-31T15:43:12Z)
• Comprehensive benchmarking of large language models for RNA secondary structure prediction [0.0]
  RNA-LLM uses large datasets of RNA sequences to learn, in a self-supervised way, how to represent each RNA base with a semantically rich numerical vector.<n>Among them, predicting the secondary structure is a fundamental task for uncovering RNA functional mechanisms.<n>We present a comprehensive experimental analysis of several pre-trained RNA-LLM, comparing them for the RNA secondary structure prediction task in a unified deep learning framework.
  arXiv  Detail & Related papers  (2024-10-21T17:12:06Z)
• BEACON: Benchmark for Comprehensive RNA Tasks and Language Models [60.02663015002029]
  We introduce the first comprehensive RNA benchmark BEACON (textbfBEnchmtextbfArk for textbfCOmprehensive RtextbfNA Task and Language Models).<n>First, BEACON comprises 13 distinct tasks derived from extensive previous work covering structural analysis, functional studies, and engineering applications.<n>Second, we examine a range of models, including traditional approaches like CNNs, as well as advanced RNA foundation models based on language models, offering valuable insights into the task-specific performances of these models.<n>Third, we investigate the vital RNA language model components
  arXiv  Detail & Related papers  (2024-06-14T19:39:19Z)
• Kirigami: large convolutional kernels improve deep learning-based RNA secondary structure prediction [0.0]
  We introduce a novel fully convolutional neural network (FCN) architecture for predicting the secondary structure of ribonucleic acid (RNA) molecules. We employ deep learning to estimate the probability of base pairing between nucleotide residues. On a widely adopted, standardized test set comprised of 1,305 molecules, the accuracy of our method exceeds that of current state-of-the-art (SOTA) secondary structure prediction software.
  arXiv  Detail & Related papers  (2024-06-04T14:58:10Z)
• RDesign: Hierarchical Data-efficient Representation Learning for Tertiary Structure-based RNA Design [65.41144149958208]
  This study aims to systematically construct a data-driven RNA design pipeline. We crafted a benchmark dataset and designed a comprehensive structural modeling approach to represent the complex RNA tertiary structure. We incorporated extracted secondary structures with base pairs as prior knowledge to facilitate the RNA design process.
  arXiv  Detail & Related papers  (2023-01-25T17:19:49Z)
• Accurate RNA 3D structure prediction using a language model-based deep learning approach [50.193512039121984]
  RhoFold+ is an RNA language model-based deep learning method that accurately predicts 3D structures of single-chain RNAs from sequences.<n>RhoFold+ offers a fully automated end-to-end pipeline for RNA 3D structure prediction.
  arXiv  Detail & Related papers  (2022-07-04T17:15:35Z)
• RNA Secondary Structure Prediction By Learning Unrolled Algorithms [70.09461537906319]
  In this paper, we propose an end-to-end deep learning model, called E2Efold, for RNA secondary structure prediction. The key idea of E2Efold is to directly predict the RNA base-pairing matrix, and use an unrolled algorithm for constrained programming as the template for deep architectures to enforce constraints. With comprehensive experiments on benchmark datasets, we demonstrate the superior performance of E2Efold.
  arXiv  Detail & Related papers  (2020-02-13T23:21:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.