DExTeR: Weakly Semi-Supervised Object Detection with Class and Instance Experts for Medical Imaging

• URL: http://arxiv.org/abs/2601.13954v1
• Date: Tue, 20 Jan 2026 13:27:14 GMT
• Title: DExTeR: Weakly Semi-Supervised Object Detection with Class and Instance Experts for Medical Imaging
• Abstract summary: We introduce DExTeR (DETR with Experts), a transformer-based Point-to-Box regressor for medical imaging.<n>It encodes single-point annotations as object queries, refining feature extraction with the proposed class-guided deformable attention.<n>It achieves state-of-the-art performance across three datasets spanning different medical domains.
• Score: 7.5912342399227235
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Detecting anatomical landmarks in medical imaging is essential for diagnosis and intervention guidance. However, object detection models rely on costly bounding box annotations, limiting scalability. Weakly Semi-Supervised Object Detection (WSSOD) with point annotations proposes annotating each instance with a single point, minimizing annotation time while preserving localization signals. A Point-to-Box teacher model, trained on a small box-labeled subset, converts these point annotations into pseudo-box labels to train a student detector. Yet, medical imagery presents unique challenges, including overlapping anatomy, variable object sizes, and elusive structures, which hinder accurate bounding box inference. To overcome these challenges, we introduce DExTeR (DETR with Experts), a transformer-based Point-to-Box regressor tailored for medical imaging. Built upon Point-DETR, DExTeR encodes single-point annotations as object queries, refining feature extraction with the proposed class-guided deformable attention, which guides attention sampling using point coordinates and class labels to capture class-specific characteristics. To improve discrimination in complex structures, it introduces CLICK-MoE (CLass, Instance, and Common Knowledge Mixture of Experts), decoupling class and instance representations to reduce confusion among adjacent or overlapping instances. Finally, we implement a multi-point training strategy which promotes prediction consistency across different point placements, improving robustness to annotation variability. DExTeR achieves state-of-the-art performance across three datasets spanning different medical domains (endoscopy, chest X-rays, and endoscopic ultrasound) highlighting its potential to reduce annotation costs while maintaining high detection accuracy.

Related papers

• Leveraging Point Transformers for Detecting Anatomical Landmarks in Digital Dentistry [0.0]
  We present our experiments from the 3DTeethLand Grand Challenge at MICCAI 2024.<n>We designed a Point Transformer v3 inspired module to capture meaningful geometric and anatomical features, which are processed by a lightweight decoder to predict per-point distances.<n>We report promising results and discuss insights on learned feature interpretability.
  arXiv  Detail & Related papers  (2025-04-15T17:34:56Z)
• Tiny Object Detection with Single Point Supervision [48.88814240556747]
  We propose Point Teacher--the first end-to-end point-supervised method for robust tiny object detection in aerial images.<n>To handle label noise from scale ambiguity and location shifts in point annotations, Point Teacher employs the teacher-student architecture.<n>In this framework, random masking of image regions facilitates regression learning, enabling the teacher to transform noisy point annotations into coarse pseudo boxes.<n>In the second phase, these coarse pseudo boxes are refined using dynamic multiple instance learning, which adaptively selects the most reliable instance.
  arXiv  Detail & Related papers  (2024-12-08T07:13:17Z)
• Weakly-Supervised Cross-Domain Segmentation of Electron Microscopy with Sparse Point Annotation [1.124958340749622]
  We introduce a multitask learning framework to leverage correlations among the counting, detection, and segmentation tasks. We develop a cross-position cut-and-paste for label augmentation and an entropy-based pseudo-label selection. The proposed model is capable of significantly outperforming UDA methods and produces comparable performance as the supervised counterpart.
  arXiv  Detail & Related papers  (2024-03-31T12:22:23Z)
• A Global and Patch-wise Contrastive Loss for Accurate Automated Exudate Detection [12.669734891001667]
  Diabetic retinopathy (DR) is a leading global cause of blindness. Early detection of hard exudates plays a crucial role in identifying DR, which aids in treating diabetes and preventing vision loss. We present a novel supervised contrastive learning framework to optimize hard exudate segmentation.
  arXiv  Detail & Related papers  (2023-02-22T17:39:00Z)
• Domain Adaptive Segmentation of Electron Microscopy with Sparse Point Annotations [2.5137859989323537]
  We develop a highly annotation-efficient approach with competitive performance. We focus on weakly-supervised domain adaptation (WDA) with a type of extremely sparse and weak annotation. We show that our model with only 15% point annotations can achieve comparable performance as supervised models.
  arXiv  Detail & Related papers  (2022-10-24T10:50:37Z)
• A Benchmark for Weakly Semi-Supervised Abnormality Localization in Chest X-Rays [42.1336336144291]
  We propose to train the CXR abnormality localization framework via a weakly semi-supervised strategy, termed Point Beyond Class. The core idea behind our PBC is to learn a robust and accurate mapping from the point annotations to the bounding boxes. Experimental results on RSNA and VinDr-CXR datasets justify the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-09-05T14:36:07Z)
• Point-Teaching: Weakly Semi-Supervised Object Detection with Point Annotations [81.02347863372364]
  We present Point-Teaching, a weakly semi-supervised object detection framework. Specifically, we propose a Hungarian-based point matching method to generate pseudo labels for point annotated images. We propose a simple-yet-effective data augmentation, termed point-guided copy-paste, to reduce the impact of the unmatched points.
  arXiv  Detail & Related papers  (2022-06-01T07:04:38Z)
• Learning to Detect Instance-level Salient Objects Using Complementary Image Labels [55.049347205603304]
  We present the first weakly-supervised approach to the salient instance detection problem. We propose a novel weakly-supervised network with three branches: a Saliency Detection Branch leveraging class consistency information to locate candidate objects; a Boundary Detection Branch exploiting class discrepancy information to delineate object boundaries; and a Centroid Detection Branch using subitizing information to detect salient instance centroids.
  arXiv  Detail & Related papers  (2021-11-19T10:15:22Z)
• Label Cleaning Multiple Instance Learning: Refining Coarse Annotations on Single Whole-Slide Images [83.7047542725469]
  Annotating cancerous regions in whole-slide images (WSIs) of pathology samples plays a critical role in clinical diagnosis, biomedical research, and machine learning algorithms development. We present a method, named Label Cleaning Multiple Instance Learning (LC-MIL), to refine coarse annotations on a single WSI without the need of external training data. Our experiments on a heterogeneous WSI set with breast cancer lymph node metastasis, liver cancer, and colorectal cancer samples show that LC-MIL significantly refines the coarse annotations, outperforming the state-of-the-art alternatives, even while learning from a single slide.
  arXiv  Detail & Related papers  (2021-09-22T15:06:06Z)
• Weakly-Supervised Salient Object Detection via Scribble Annotations [54.40518383782725]
  We propose a weakly-supervised salient object detection model to learn saliency from scribble labels. We present a new metric, termed saliency structure measure, to measure the structure alignment of the predicted saliency maps. Our method not only outperforms existing weakly-supervised/unsupervised methods, but also is on par with several fully-supervised state-of-the-art models.
  arXiv  Detail & Related papers  (2020-03-17T12:59:50Z)
• Object Detection as a Positive-Unlabeled Problem [78.2955013126312]
  We propose treating object detection as a positive-unlabeled (PU) problem, which removes the assumption that unlabeled regions must be negative. We demonstrate that our proposed PU classification loss outperforms the standard PN loss on PASCAL VOC and MS COCO across a range of label missingness, as well as on Visual Genome and DeepLesion with full labels.
  arXiv  Detail & Related papers  (2020-02-11T20:49:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.