Fitting Segmentation Networks on Varying Image Resolutions using Splatting

• URL: http://arxiv.org/abs/2206.06445v2
• Date: Wed, 15 Jun 2022 10:07:37 GMT
• Title: Fitting Segmentation Networks on Varying Image Resolutions using Splatting
• Authors: Mikael Brudfors and Yael Balbastre and John Ashburner and Geraint Rees and Parashkev Nachev and Sebastien Ourselin and M. Jorge Cardoso
• Abstract summary: We propose a splat layer, which automatically handles resolution mismatches in the input data. As the splat operator is the adjoint to the resampling operator, the mean-space prediction can be pulled back to the native label space. This model improves segmentation results compared to resampling as a pre-processing step.
• Score: 1.3792537518004493
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data used in image segmentation are not always defined on the same grid. This is particularly true for medical images, where the resolution, field-of-view and orientation can differ across channels and subjects. Images and labels are therefore commonly resampled onto the same grid, as a pre-processing step. However, the resampling operation introduces partial volume effects and blurring, thereby changing the effective resolution and reducing the contrast between structures. In this paper we propose a splat layer, which automatically handles resolution mismatches in the input data. This layer pushes each image onto a mean space where the forward pass is performed. As the splat operator is the adjoint to the resampling operator, the mean-space prediction can be pulled back to the native label space, where the loss function is computed. Thus, the need for explicit resolution adjustment using interpolation is removed. We show on two publicly available datasets, with simulated and real multi-modal magnetic resonance images, that this model improves segmentation results compared to resampling as a pre-processing step.

Related papers

• Restoring Images in Adverse Weather Conditions via Histogram Transformer [75.74328579778049]
  We propose an efficient Histogram Transformer (Histoformer) for restoring images affected by adverse weather. It is powered by a mechanism dubbed histogram self-attention, which sorts and segments spatial features into intensity-based bins. To boost histogram self-attention, we present a dynamic-range convolution enabling conventional convolution to conduct operation over similar pixels.
  arXiv  Detail & Related papers  (2024-07-14T11:59:22Z)
• Learning to Rank Patches for Unbiased Image Redundancy Reduction [80.93989115541966]
  Images suffer from heavy spatial redundancy because pixels in neighboring regions are spatially correlated. Existing approaches strive to overcome this limitation by reducing less meaningful image regions. We propose a self-supervised framework for image redundancy reduction called Learning to Rank Patches.
  arXiv  Detail & Related papers  (2024-03-31T13:12:41Z)
• Inpainting Normal Maps for Lightstage data [3.1002416427168304]
  This study introduces a novel method for inpainting normal maps using a generative adversarial network (GAN) Our approach extends previous general image inpainting techniques, employing a bow tie-like generator network and a discriminator network, with alternating training phases. Our findings suggest that the proposed model effectively generates high-quality, realistic inpainted normal maps, suitable for performance capture applications.
  arXiv  Detail & Related papers  (2024-01-16T03:59:07Z)
• Wavelet-based Unsupervised Label-to-Image Translation [9.339522647331334]
  We propose a new Unsupervised paradigm for SIS (USIS) that makes use of a self-supervised segmentation loss and whole image wavelet based discrimination. We test our methodology on 3 challenging datasets and demonstrate its ability to bridge the performance gap between paired and unpaired models.
  arXiv  Detail & Related papers  (2023-05-16T17:48:44Z)
• High-Fidelity Guided Image Synthesis with Latent Diffusion Models [50.39294302741698]
  The proposed approach outperforms the previous state-of-the-art by over 85.32% on the overall user satisfaction scores. Human user study results show that the proposed approach outperforms the previous state-of-the-art by over 85.32% on the overall user satisfaction scores.
  arXiv  Detail & Related papers  (2022-11-30T15:43:20Z)
• Accurate Image Restoration with Attention Retractable Transformer [50.05204240159985]
  We propose Attention Retractable Transformer (ART) for image restoration. ART presents both dense and sparse attention modules in the network. We conduct extensive experiments on image super-resolution, denoising, and JPEG compression artifact reduction tasks.
  arXiv  Detail & Related papers  (2022-10-04T07:35:01Z)
• Spatially-Adaptive Image Restoration using Distortion-Guided Networks [51.89245800461537]
  We present a learning-based solution for restoring images suffering from spatially-varying degradations. We propose SPAIR, a network design that harnesses distortion-localization information and dynamically adjusts to difficult regions in the image.
  arXiv  Detail & Related papers  (2021-08-19T11:02:25Z)
• LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimation [52.63874513999119]
  Cross-resolution image alignment is a key problem in multiscale giga photography. Existing deep homography methods neglecting the explicit formulation of correspondences between them, which leads to degraded accuracy in cross-resolution challenges. We propose a local transformer network embedded within a multiscale structure to explicitly learn correspondences between the multimodal inputs.
  arXiv  Detail & Related papers  (2021-06-08T02:51:45Z)
• Deep Contrastive Patch-Based Subspace Learning for Camera Image Signal Processing [5.678834480723395]
  We present a specific patch-based, local subspace deep neural network that improves Camera ISP to be robust to heterogeneous artifacts. We call our three-fold deep-trained model the Patch Subspace Learning Autoencoder (PSL-AE) PSL-AE encodes patches extracted from noisy a nd clean image pairs, with different artifact types or distortion levels, by contrastive learning.
  arXiv  Detail & Related papers  (2021-04-01T04:40:22Z)
• An Empirical Method to Quantify the Peripheral Performance Degradation in Deep Networks [18.808132632482103]
  convolutional neural network (CNN) kernels compound with each convolutional layer. Deeper and deeper networks combined with stride-based down-sampling means that the propagation of this region can end up covering a non-negligable portion of the image. Our dataset is constructed by inserting objects into high resolution backgrounds, thereby allowing us to crop sub-images which place target objects at specific locations relative to the image border. By probing the behaviour of Mask R-CNN across a selection of target locations, we see clear patterns of performance degredation near the image boundary, and in particular in the image corners.
  arXiv  Detail & Related papers  (2020-12-04T18:00:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.