Super-Resolution Based Patch-Free 3D Image Segmentation with
High-Frequency Guidance
- URL: http://arxiv.org/abs/2210.14645v2
- Date: Mon, 10 Jul 2023 07:53:28 GMT
- Title: Super-Resolution Based Patch-Free 3D Image Segmentation with
High-Frequency Guidance
- Authors: Hongyi Wang, Lanfen Lin, Hongjie Hu, Qingqing Chen, Yinhao Li, Yutaro
Iwamoto, Xian-Hua Han, Yen-Wei Chen, Ruofeng Tong
- Abstract summary: High resolution (HR) 3D images are widely used nowadays, such as medical images like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT)
- Score: 20.86089285980103
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: High resolution (HR) 3D images are widely used nowadays, such as medical
images like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT).
However, segmentation of these 3D images remains a challenge due to their high
spatial resolution and dimensionality in contrast to currently limited GPU
memory. Therefore, most existing 3D image segmentation methods use patch-based
models, which have low inference efficiency and ignore global contextual
information. To address these problems, we propose a super-resolution (SR)
based patch-free 3D image segmentation framework that can realize HR
segmentation from a global-wise low-resolution (LR) input. The framework
contains two sub-tasks, of which semantic segmentation is the main task and
super resolution is an auxiliary task aiding in rebuilding the high frequency
information from the LR input. To furthermore balance the information loss with
the LR input, we propose a High-Frequency Guidance Module (HGM), and design an
efficient selective cropping algorithm to crop an HR patch from the original
image as restoration guidance for it. In addition, we also propose a
Task-Fusion Module (TFM) to exploit the inter connections between segmentation
and SR task, realizing joint optimization of the two tasks. When predicting,
only the main segmentation task is needed, while other modules can be removed
for acceleration. The experimental results on two different datasets show that
our framework has a four times higher inference speed compared to traditional
patch-based methods, while its performance also surpasses other patch-based and
patch-free models.
Related papers
- HartleyMHA: Self-Attention in Frequency Domain for Resolution-Robust and
Parameter-Efficient 3D Image Segmentation [4.48473804240016]
We introduce the HartleyMHA model which is robust to training image resolution with efficient self-attention.
We modify the FNO by using the Hartley transform with shared parameters to reduce the model size by orders of magnitude.
When tested on the BraTS'19 dataset, it achieved superior robustness to training image resolution than other tested models with less than 1% of their model parameters.
arXiv Detail & Related papers (2023-10-05T18:44:41Z) - Mutual Information-driven Triple Interaction Network for Efficient Image
Dehazing [54.168567276280505]
We propose a novel Mutual Information-driven Triple interaction Network (MITNet) for image dehazing.
The first stage, named amplitude-guided haze removal, aims to recover the amplitude spectrum of the hazy images for haze removal.
The second stage, named phase-guided structure refined, devotes to learning the transformation and refinement of the phase spectrum.
arXiv Detail & Related papers (2023-08-14T08:23:58Z) - SwIPE: Efficient and Robust Medical Image Segmentation with Implicit Patch Embeddings [12.79344668998054]
We propose SwIPE (Segmentation with Implicit Patch Embeddings) to enable accurate local boundary delineation and global shape coherence.
We show that SwIPE significantly improves over recent implicit approaches and outperforms state-of-the-art discrete methods with over 10x fewer parameters.
arXiv Detail & Related papers (2023-07-23T20:55:11Z) - Hi-ResNet: A High-Resolution Remote Sensing Network for Semantic
Segmentation [7.216053041550996]
High-resolution remote sensing (HRS) semantic segmentation extracts key objects from high-resolution coverage areas.
objects of the same category within HRS images show significant differences in scale and shape across diverse geographical environments.
We propose a High-resolution remote sensing network (Hi-ResNet) with efficient network structure designs.
arXiv Detail & Related papers (2023-05-22T03:58:25Z) - Rank-Enhanced Low-Dimensional Convolution Set for Hyperspectral Image
Denoising [50.039949798156826]
This paper tackles the challenging problem of hyperspectral (HS) image denoising.
We propose rank-enhanced low-dimensional convolution set (Re-ConvSet)
We then incorporate Re-ConvSet into the widely-used U-Net architecture to construct an HS image denoising method.
arXiv Detail & Related papers (2022-07-09T13:35:12Z) - Memory-augmented Deep Unfolding Network for Guided Image
Super-resolution [67.83489239124557]
Guided image super-resolution (GISR) aims to obtain a high-resolution (HR) target image by enhancing the spatial resolution of a low-resolution (LR) target image under the guidance of a HR image.
Previous model-based methods mainly takes the entire image as a whole, and assume the prior distribution between the HR target image and the HR guidance image.
We propose a maximal a posterior (MAP) estimation model for GISR with two types of prior on the HR target image.
arXiv Detail & Related papers (2022-02-12T15:37:13Z) - High Quality Segmentation for Ultra High-resolution Images [72.97958314291648]
We propose the Continuous Refinement Model for the ultra high-resolution segmentation refinement task.
Our proposed method is fast and effective on image segmentation refinement.
arXiv Detail & Related papers (2021-11-29T11:53:06Z) - High-resolution Depth Maps Imaging via Attention-based Hierarchical
Multi-modal Fusion [84.24973877109181]
We propose a novel attention-based hierarchical multi-modal fusion network for guided DSR.
We show that our approach outperforms state-of-the-art methods in terms of reconstruction accuracy, running speed and memory efficiency.
arXiv Detail & Related papers (2021-04-04T03:28:33Z) - Polarimetric SAR Image Semantic Segmentation with 3D Discrete Wavelet
Transform and Markov Random Field [32.59900433812833]
We present a contextual PolSAR image semantic segmentation method in this paper.
With a newly defined channelwise consistent feature set as input, the 3D-DWT technique is employed to extract discriminative multi-scale features that are robust to speckle noise.
By simultaneously utilizing 3D-DWT features and MRF priors for the first time, contextual information is fully integrated during the segmentation to ensure accurate and smooth segmentation.
arXiv Detail & Related papers (2020-08-05T08:28:18Z) - An End-to-end Framework For Low-Resolution Remote Sensing Semantic
Segmentation [0.5076419064097732]
We propose an end-to-end framework that unites a super-resolution and a semantic segmentation module.
It allows the semantic segmentation network to conduct the reconstruction process, modifying the input image with helpful textures.
The results show that the framework is capable of achieving a semantic segmentation performance close to native high-resolution data.
arXiv Detail & Related papers (2020-03-17T21:41:22Z) - Spatial-Spectral Residual Network for Hyperspectral Image
Super-Resolution [82.1739023587565]
We propose a novel spectral-spatial residual network for hyperspectral image super-resolution (SSRNet)
Our method can effectively explore spatial-spectral information by using 3D convolution instead of 2D convolution, which enables the network to better extract potential information.
In each unit, we employ spatial and temporal separable 3D convolution to extract spatial and spectral information, which not only reduces unaffordable memory usage and high computational cost, but also makes the network easier to train.
arXiv Detail & Related papers (2020-01-14T03:34:55Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.