TaxDistill: Improving Metagenomic Taxonomic Annotation via Distilled Genomic Foundation Models

• URL: http://arxiv.org/abs/2605.28868v1
• Date: Fri, 22 May 2026 08:03:29 GMT
• Title: TaxDistill: Improving Metagenomic Taxonomic Annotation via Distilled Genomic Foundation Models
• Abstract summary: TaxDistill is a knowledge distillation framework for metagenomic classification.<n>We introduce GenomeOcean as the teacher network to extract deep semantic features and generate soft labels based on confidence.<n>Comprehensive experiments on seven diverse CAMI2 datasets demonstrate that TaxDistill outperforms existing baselines in most scenarios.
• Score: 3.756924024442221
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Metagenomic taxonomic annotation aims to identify the microbial origins of DNA fragments in environmental samples. Traditional methods that rely on sequence similarity are often constrained by the high microbial diversity and the incompleteness of reference databases, which has motivated the development of learning approaches such as Taxometer that perform post hoc correction to learn more informative metagenomic sequence representations. However, these methods typically rely on labels derived from similarity search tools during training, which inevitably introduces noise that can impair representation learning and degrade classification performance. To address this issue, we propose TaxDistill, a knowledge distillation framework for metagenomic classification. We introduce GenomeOcean, a 500M parameter genomic foundation model, as the teacher network to extract deep semantic features and generate soft labels based on confidence. By distilling this soft label information into a lightweight student network, TaxDistill effectively reduces the label noise introduced by initial retrieval tools. Comprehensive experiments on seven diverse CAMI2 datasets demonstrate that TaxDistill outperforms existing baselines in most scenarios. For instance, on the Gastrointestinal dataset, it improves the F1 score of MMseqs2 from 0.763 to 0.941, outperforming the Taxometer baseline. Overall, TaxDistill provides a reliable method for label correction in complex metagenomic analysis.

Related papers

• Hierarchy-Guided Multimodal Representation Learning for Taxonomic Inference [17.708423679075075]
  We present two end-to-end variants for hierarchy-aware multimodal learning.<n>CLiBD-HiR introduces Hierarchical Information Regularization (HiR) to shape embedding geometry across taxonomic levels.<n>CLiBD-HiR-Fuse trains a lightweight fusion predictor that supports image-only, DNA-only, or joint inference.
  arXiv  Detail & Related papers  (2026-03-26T15:47:03Z)
• iCD: A Implicit Clustering Distillation Mathod for Structural Information Mining [1.3573542141741506]
  implicit Clustering Distillation (iCD) is a simple and effective method that mines and transfers interpretable structural knowledge from logits.<n>Experiments on benchmark datasets demonstrate the effectiveness of iCD across diverse teacher-student architectures.
  arXiv  Detail & Related papers  (2025-09-16T01:16:13Z)
• Rethinking Self-Distillation: Label Averaging and Enhanced Soft Label Refinement with Partial Labels [10.696635172502141]
  Multi-round self-distillation effectively performs label averaging among instances with high feature correlations.<n>We introduce a novel, efficient single-round self-distillation method using refined partial labels from the teacher's top two softmax outputs.
  arXiv  Detail & Related papers  (2024-02-16T07:13:12Z)
• Label-Retrieval-Augmented Diffusion Models for Learning from Noisy Labels [61.97359362447732]
  Learning from noisy labels is an important and long-standing problem in machine learning for real applications. In this paper, we reformulate the label-noise problem from a generative-model perspective. Our model achieves new state-of-the-art (SOTA) results on all the standard real-world benchmark datasets.
  arXiv  Detail & Related papers  (2023-05-31T03:01:36Z)
• GaussianMLR: Learning Implicit Class Significance via Calibrated Multi-Label Ranking [0.0]
  We propose a novel multi-label ranking method: GaussianMLR. It aims to learn implicit class significance values that determine the positive label ranks. We show that our method is able to accurately learn a representation of the incorporated positive rank order.
  arXiv  Detail & Related papers  (2023-03-07T14:09:08Z)
• Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting [62.23057729112182]
  Differentiable score-based causal discovery methods learn a directed acyclic graph from observational data. We propose a model-agnostic framework to boost causal discovery performance by dynamically learning the adaptive weights for the Reweighted Score function, ReScore.
  arXiv  Detail & Related papers  (2023-03-06T14:49:59Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• Deep Contrastive Graph Representation via Adaptive Homotopy Learning [76.22904270821778]
  Homotopy model is an excellent tool exploited by diverse research works in the field of machine learning. We propose a novel adaptive homotopy framework (AH) in which the Maclaurin duality is employed. AH can be widely utilized to enhance the homotopy-based algorithm.
  arXiv  Detail & Related papers  (2021-06-17T04:46:04Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)
• Rectified Meta-Learning from Noisy Labels for Robust Image-based Plant Disease Diagnosis [64.82680813427054]
  Plant diseases serve as one of main threats to food security and crop production. One popular approach is to transform this problem as a leaf image classification task, which can be addressed by the powerful convolutional neural networks (CNNs) We propose a novel framework that incorporates rectified meta-learning module into common CNN paradigm to train a noise-robust deep network without using extra supervision information.
  arXiv  Detail & Related papers  (2020-03-17T09:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.