Foveation for Segmentation of Ultra-High Resolution Images

• URL: http://arxiv.org/abs/2007.15124v2
• Date: Fri, 31 Jul 2020 16:53:18 GMT
• Title: Foveation for Segmentation of Ultra-High Resolution Images
• Authors: Chen Jin, Ryutaro Tanno, Moucheng Xu, Thomy Mertzanidou, Daniel C. Alexander
• Abstract summary: We introduce foveation module, a learnable "dataloader" which adaptively chooses the appropriate configuration (FoV/resolution trade-off) of the input patch to feed to the downstream segmentation model. We demonstrate that the foveation module consistently improves segmentation performance over the cases trained with patches of fixed FoV/resolution trade-off.
• Score: 8.037287701125832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segmentation of ultra-high resolution images is challenging because of their enormous size, consisting of millions or even billions of pixels. Typical solutions include dividing input images into patches of fixed size and/or down-sampling to meet memory constraints. Such operations incur information loss in the field-of-view (FoV) i.e., spatial coverage and the image resolution. The impact on segmentation performance is, however, as yet understudied. In this work, we start with a motivational experiment which demonstrates that the trade-off between FoV and resolution affects the segmentation performance on ultra-high resolution images---and furthermore, its influence also varies spatially according to the local patterns in different areas. We then introduce foveation module, a learnable "dataloader" which, for a given ultra-high resolution image, adaptively chooses the appropriate configuration (FoV/resolution trade-off) of the input patch to feed to the downstream segmentation model at each spatial location of the image. The foveation module is jointly trained with the segmentation network to maximise the task performance. We demonstrate on three publicly available high-resolution image datasets that the foveation module consistently improves segmentation performance over the cases trained with patches of fixed FoV/resolution trade-off. Our approach achieves the SoTA performance on the DeepGlobe aerial image dataset. On the Gleason2019 histopathology dataset, our model achieves better segmentation accuracy for the two most clinically important and ambiguous classes (Gleason Grade 3 and 4) than the top performers in the challenge by 13.1% and 7.5%, and improves on the average performance of 6 human experts by 6.5% and 7.5%. Our code and trained models are available at $\text{https://github.com/lxasqjc/Foveation-Segmentation}$.

Related papers

• High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity [69.32473738284374]
  We propose DiffDIS, a diffusion-driven segmentation model that taps into the potential of the pre-trained U-Net within diffusion models. By leveraging the robust generalization capabilities and rich, versatile image representation prior to the SD models, we significantly reduce the inference time while preserving high-fidelity, detailed generation. Experiments on the DIS5K dataset demonstrate the superiority of DiffDIS, achieving state-of-the-art results through a streamlined inference process.
  arXiv  Detail & Related papers  (2024-10-14T02:49:23Z)
• UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation [64.01742988773745]
  An increasing privacy concern exists regarding training large-scale image segmentation models on unauthorized private data. We exploit the concept of unlearnable examples to make images unusable to model training by generating and adding unlearnable noise into the original images. We empirically verify the effectiveness of UnSeg across 6 mainstream image segmentation tasks, 10 widely used datasets, and 7 different network architectures.
  arXiv  Detail & Related papers  (2024-10-13T16:34:46Z)
• Semantic Segmentation Refiner for Ultrasound Applications with Zero-Shot Foundation Models [1.8142288667655782]
  We propose a prompt-less segmentation method harnessing the ability of segmentation foundation models to segment abstract shapes. Our method's advantages are brought to light in experiments on a small-scale musculoskeletal ultrasound images dataset.
  arXiv  Detail & Related papers  (2024-04-25T04:21:57Z)
• Dual-scale Enhanced and Cross-generative Consistency Learning for Semi-supervised Medical Image Segmentation [49.57907601086494]
  Medical image segmentation plays a crucial role in computer-aided diagnosis. We propose a novel Dual-scale Enhanced and Cross-generative consistency learning framework for semi-supervised medical image (DEC-Seg)
  arXiv  Detail & Related papers  (2023-12-26T12:56:31Z)
• Self-supervised Semantic Segmentation: Consistency over Transformation [3.485615723221064]
  We propose a novel self-supervised algorithm, textbfS$3$-Net, which integrates a robust framework based on the proposed Inception Large Kernel Attention (I-LKA) modules. We leverage deformable convolution as an integral component to effectively capture and delineate lesion deformations for superior object boundary definition. Our experimental results on skin lesion and lung organ segmentation tasks show the superior performance of our method compared to the SOTA approaches.
  arXiv  Detail & Related papers  (2023-08-31T21:28:46Z)
• Does image resolution impact chest X-ray based fine-grained Tuberculosis-consistent lesion segmentation? [3.3086274755158325]
  Deep learning models are reportedly trained on reduced image resolutions citing reasons for the lack of computational resources. This study investigated performance gains achieved through training an Inception-V3-based UNet model using various image/mask resolutions.
  arXiv  Detail & Related papers  (2023-01-10T15:34:39Z)
• High-Quality Entity Segmentation [110.55724145851725]
  CropFormer is designed to tackle the intractability of instance-level segmentation on high-resolution images. It improves mask prediction by fusing high-res image crops that provide more fine-grained image details and the full image. With CropFormer, we achieve a significant AP gain of $1.9$ on the challenging entity segmentation task.
  arXiv  Detail & Related papers  (2022-11-10T18:58:22Z)
• Semantically Accurate Super-Resolution Generative Adversarial Networks [2.0454959820861727]
  We propose a novel architecture and domain-specific feature loss to increase the performance of semantic segmentation. We show the proposed approach improves perceived image quality as well as quantitative segmentation accuracy across all prediction classes. This work demonstrates that jointly considering image-based and task-specific losses can improve the performance of both, and advances the state-of-the-art in semantic-aware super-resolution of aerial imagery.
  arXiv  Detail & Related papers  (2022-05-17T23:05:27Z)
• High Quality Segmentation for Ultra High-resolution Images [72.97958314291648]
  We propose the Continuous Refinement Model for the ultra high-resolution segmentation refinement task. Our proposed method is fast and effective on image segmentation refinement.
  arXiv  Detail & Related papers  (2021-11-29T11:53:06Z)
• Gated Fusion Network for Degraded Image Super Resolution [78.67168802945069]
  We propose a dual-branch convolutional neural network to extract base features and recovered features separately. By decomposing the feature extraction step into two task-independent streams, the dual-branch model can facilitate the training process.
  arXiv  Detail & Related papers  (2020-03-02T13:28:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.