GenoBERT: A Language Model for Accurate Genotype Imputation

• URL: http://arxiv.org/abs/2604.00058v1
• Date: Tue, 31 Mar 2026 04:16:55 GMT
• Title: GenoBERT: A Language Model for Accurate Genotype Imputation
• Abstract summary: Genotype Bidirectional Representations from Transformers (GenoBERT) is a reference-free framework that tokenizes phased genotypes.<n>GenoBERT achieves the highest overall accuracy compared to four baseline methods.
• Score: 23.45811349566905
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Genotype imputation enables dense variant coverage for genome-wide association and risk-prediction studies, yet conventional reference-panel methods remain limited by ancestry bias and reduced rare-variant accuracy. We present Genotype Bidirectional Encoder Representations from Transformers (GenoBERT), a transformer-based, reference-free framework that tokenizes phased genotypes and uses a self-attention mechanism to capture both short- and long-range linkage disequilibrium (LD) dependencies. Benchmarking on two independent datasets including the Louisiana Osteoporosis Study (LOS) and the 1000 Genomes Project (1KGP) across ancestry groups and multiple genotype missingness levels (5-50%) shows that GenoBERT achieves the highest overall accuracy compared to four baseline methods (Beagle5.4, SCDA, BiU-Net, and STICI). At practical sparsity levels (up to 25% missing), GenoBERT attains high overall imputation accuracy ($r^2 approx 0.98$) across datasets, and maintains robust performance ($r^2 > 0.90$) even at 50% missingness. Experimental results across different ancestries confirm consistent gains across datasets, with resilience to small sample sizes and weak LD. A 128-SNP (single-nucleotide polymorphism) context window (approximately 100 Kb) is validated through LD-decay analyses as sufficient to capture local correlation structures. By eliminating reference-panel dependence while preserving high accuracy, GenoBERT provides a scalable and robust solution for genotype imputation and a foundation for downstream genomic modeling.

Related papers

• BioM-JEPA: joint-embedding prediction of graph-connected gene blocks in single cells [64.21963170814706]
  BioM-JEPA predicts aggregate representations of graph-connected gene blocks.<n>A student network infers each target-block representation from the remaining genes in a cell.<n>Across CellBench tasks, BioM-JEPA embeddings retained expression, pathway and neighbourhood information.
  arXiv  Detail & Related papers  (2026-08-06T11:58:29Z)
• SNPgen: Phenotype-Supervised Genotype Representation and Synthetic Data Generation via Latent Diffusion [6.660968772645983]
  SNPgen is a conditional latent diffusion framework for generating phenotype-supervised synthetic genotypes.<n> evaluated on UK Biobank individuals across four complex diseases.
  arXiv  Detail & Related papers  (2026-03-11T15:23:37Z)
• Morpho-Genomic Deep Learning for Ovarian Cancer Subtype and Gene Mutation Prediction from Histopathology [0.0]
  Current diagnostic methods are limited in their ability to reveal underlying genomic variations essential for precision oncology.<n>This study introduces a novel hybrid deep learning pipeline that integrates quantitative nuclear morphometry with deep convolutional image features.<n>The pipeline achieved a robust overall subtype classification accuracy of $84.2%$ (Macro AUC of $0.87 pm 0.03$)
  arXiv  Detail & Related papers  (2025-11-05T11:09:20Z)
• DNABERT-2: Fine-Tuning a Genomic Language Model for Colorectal Gene Enhancer Classification [0.0]
  DNABERT-2 is a transformer genomic language model that uses byte-pair encoding to learn variable-length tokens from DNA.<n>Gene enhancers control when and where genes switch on, yet their sequence diversity and tissue specificity make them hard to pinpoint in colorectal cancer.<n>This is the first study to apply a second-generation genomic language model with BPE tokenization to enhancer classification in colorectal cancer.
  arXiv  Detail & Related papers  (2025-09-28T16:10:03Z)
• Unlasting: Unpaired Single-Cell Multi-Perturbation Estimation by Dual Conditional Diffusion Implicit Bridges [68.98973318553983]
  We propose a framework based on Dual Diffusion Implicit Bridges (DDIB) to learn the mapping between different data distributions.<n>We integrate gene regulatory network (GRN) information to propagate perturbation signals in a biologically meaningful way.<n>We also incorporate a masking mechanism to predict silent genes, improving the quality of generated profiles.
  arXiv  Detail & Related papers  (2025-06-26T09:05:38Z)
• Advancing Tabular Stroke Modelling Through a Novel Hybrid Architecture and Feature-Selection Synergy [0.9999629695552196]
  The present work develops and validates a data-driven and interpretable machine-learning framework designed to predict strokes.<n>Ten routinely gathered demographic, lifestyle, and clinical variables were sourced from a public cohort of 4,981 records.<n>The proposed model achieved an accuracy rate of 97.2% and an F1-score of 97.15%, indicating a significant enhancement compared to the leading individual model.
  arXiv  Detail & Related papers  (2025-05-18T21:46:45Z)
• Efficient and Scalable Fine-Tune of Language Models for Genome Understanding [49.606093223945734]
  We present textscLingo: textscLanguage prefix ftextscIne-tuning for textscGentextscOmes. Unlike DNA foundation models, textscLingo strategically leverages natural language foundation models' contextual cues. textscLingo further accommodates numerous downstream fine-tune tasks by an adaptive rank sampling method.
  arXiv  Detail & Related papers  (2024-02-12T21:40:45Z)
• HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution [76.97231739317259]
  We present HyenaDNA, a genomic foundation model pretrained on the human reference genome with context lengths of up to 1 million tokens at the single nucleotide-level. On fine-tuned benchmarks from the Nucleotide Transformer, HyenaDNA reaches state-of-the-art (SotA) on 12 of 18 datasets using a model with orders of magnitude less parameters and pretraining data.
  arXiv  Detail & Related papers  (2023-06-27T20:46:34Z)
• DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genome [10.051595222470304]
  We argue that the computation and sample inefficiencies introduced by k-mer tokenization are primary obstacles in developing large genome foundational models. We provide conceptual and empirical insights into genome tokenization, building on which we propose to replace k-mer tokenization with Byte Pair$. We introduce DNABERT-2, a refined genome foundation model that adapts an efficient tokenizer and employs multiple strategies to overcome input length constraints.
  arXiv  Detail & Related papers  (2023-06-26T18:43:46Z)
• rfPhen2Gen: A machine learning based association study of brain imaging phenotypes to genotypes [71.1144397510333]
  We learned machine learning models to predict SNPs using 56 brain imaging QTs. SNPs within the known Alzheimer disease (AD) risk gene APOE had lowest RMSE for lasso and random forest. Random forests identified additional SNPs that were not prioritized by the linear models but are known to be associated with brain-related disorders.
  arXiv  Detail & Related papers  (2022-03-31T20:15:22Z)
• Brain Image Synthesis with Unsupervised Multivariate Canonical CSC$\ell_4$Net [122.8907826672382]
  We propose to learn dedicated features that cross both intre- and intra-modal variations using a novel CSC$ell_4$Net.
  arXiv  Detail & Related papers  (2021-03-22T05:19:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.