Tree-based Semantic Losses: Application to Sparsely-supervised Large Multi-class Hyperspectral Segmentation

• URL: http://arxiv.org/abs/2506.21150v1
• Date: Thu, 26 Jun 2025 11:20:46 GMT
• Title: Tree-based Semantic Losses: Application to Sparsely-supervised Large Multi-class Hyperspectral Segmentation
• Authors: Junwen Wang, Oscar Maccormac, William Rochford, Aaron Kujawa, Jonathan Shapey, Tom Vercauteren,
• Abstract summary: We introduce two tree-based semantic loss functions which take advantage of a hierarchical organisation of the labels.<n>Our proposed method reaches state-of-the-art performance on a sparsely annotated HSI dataset.
• Score: 3.1970844823805002
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Hyperspectral imaging (HSI) shows great promise for surgical applications, offering detailed insights into biological tissue differences beyond what the naked eye can perceive. Refined labelling efforts are underway to train vision systems to distinguish large numbers of subtly varying classes. However, commonly used learning methods for biomedical segmentation tasks penalise all errors equivalently and thus fail to exploit any inter-class semantics in the label space. In this work, we introduce two tree-based semantic loss functions which take advantage of a hierarchical organisation of the labels. We further incorporate our losses in a recently proposed approach for training with sparse, background-free annotations. Extensive experiments demonstrate that our proposed method reaches state-of-the-art performance on a sparsely annotated HSI dataset comprising $107$ classes organised in a clinically-defined semantic tree structure. Furthermore, our method enables effective detection of out-of-distribution (OOD) pixels without compromising segmentation performance on in-distribution (ID) pixels.

Related papers

• Label tree semantic losses for rich multi-class medical image segmentation [3.1970844823805002]
  We propose two tree-based semantic loss functions which take advantage of a hierarchical organisation of labels.<n>Experiments are reported on two medical and surgical image segmentation tasks, namely head MRI for whole brain parcellation (WBP) with full supervision and neurosurgical hyperspectral imaging (HSI) for scene understanding with sparse annotations.
  arXiv  Detail & Related papers  (2025-07-21T16:32:48Z)
• Collaborative Feature-Logits Contrastive Learning for Open-Set Semi-Supervised Object Detection [75.02249869573994]
  In open-set scenarios, the unlabeled dataset contains both in-distribution (ID) classes and out-of-distribution (OOD) classes.<n>Applying semi-supervised detectors in such settings can lead to misclassifying OOD class as ID classes.<n>We propose a simple yet effective method, termed Collaborative Feature-Logits Detector (CFL-Detector)
  arXiv  Detail & Related papers  (2024-11-20T02:57:35Z)
• Diagnostic Text-guided Representation Learning in Hierarchical Classification for Pathological Whole Slide Image [9.195096835877914]
  We introduce the concept of hierarchical pathological image classification and propose a representation learning called PathTree. PathTree considers the multi-classification of diseases as a binary tree structure. Our proposed PathTree is consistently competitive compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-11-16T05:35:39Z)
• OOD-SEG: Out-Of-Distribution detection for image SEGmentation with sparse multi-class positive-only annotations [4.9547168429120205]
  Deep neural networks in medical and surgical imaging face several challenges, two of which we aim to address in this work. First, acquiring complete pixel-level segmentation labels for medical images is time-consuming and requires domain expertise. Second, typical segmentation pipelines cannot detect out-of-distribution pixels, leaving them prone to spurious outputs during deployment.
  arXiv  Detail & Related papers  (2024-11-14T16:06:30Z)
• Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation [79.05949524349005]
  We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from saliency maps. We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps.
  arXiv  Detail & Related papers  (2024-03-02T10:03:21Z)
• Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems [61.11799513362704]
  We propose learning an additional screening mechanism to identify discriminative clues commonly seen across instances and classes. We show that a common rationale detector can be learned by simply exploiting the GradCAM induced from the SSL objective.
  arXiv  Detail & Related papers  (2023-03-03T02:07:40Z)
• Human-machine Interactive Tissue Prototype Learning for Label-efficient Histopathology Image Segmentation [18.755759024796216]
  Deep neural networks have greatly advanced histopathology image segmentation but usually require abundant data. We present a label-efficient tissue prototype dictionary building pipeline and propose to use the obtained prototypes to guide histopathology image segmentation. We show that our human-machine interactive tissue prototype learning method can achieve comparable segmentation performance as the fully-supervised baselines.
  arXiv  Detail & Related papers  (2022-11-26T06:17:21Z)
• Semi-Supervised Semantic Segmentation of Vessel Images using Leaking Perturbations [1.5791732557395552]
  Leaking GAN is a GAN-based semi-supervised architecture for retina vessel semantic segmentation. Our key idea is to pollute the discriminator by leaking information from the generator. This leads to more moderate generations that benefit the training of GAN.
  arXiv  Detail & Related papers  (2021-10-22T18:25:08Z)
• Leveraging Auxiliary Tasks with Affinity Learning for Weakly Supervised Semantic Segmentation [88.49669148290306]
  We propose a novel weakly supervised multi-task framework called AuxSegNet to leverage saliency detection and multi-label image classification as auxiliary tasks. Inspired by their similar structured semantics, we also propose to learn a cross-task global pixel-level affinity map from the saliency and segmentation representations. The learned cross-task affinity can be used to refine saliency predictions and propagate CAM maps to provide improved pseudo labels for both tasks.
  arXiv  Detail & Related papers  (2021-07-25T11:39:58Z)
• Towards Robust Partially Supervised Multi-Structure Medical Image Segmentation on Small-Scale Data [123.03252888189546]
  We propose Vicinal Labels Under Uncertainty (VLUU) to bridge the methodological gaps in partially supervised learning (PSL) under data scarcity. Motivated by multi-task learning and vicinal risk minimization, VLUU transforms the partially supervised problem into a fully supervised problem by generating vicinal labels. Our research suggests a new research direction in label-efficient deep learning with partial supervision.
  arXiv  Detail & Related papers  (2020-11-28T16:31:00Z)
• Panoptic Feature Fusion Net: A Novel Instance Segmentation Paradigm for Biomedical and Biological Images [91.41909587856104]
  We present a Panoptic Feature Fusion Net (PFFNet) that unifies the semantic and instance features in this work. Our proposed PFFNet contains a residual attention feature fusion mechanism to incorporate the instance prediction with the semantic features. It outperforms several state-of-the-art methods on various biomedical and biological datasets.
  arXiv  Detail & Related papers  (2020-02-15T09:19:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.