Light Field Image Super-Resolution with Transformers
- URL: http://arxiv.org/abs/2108.07597v1
- Date: Tue, 17 Aug 2021 12:58:11 GMT
- Title: Light Field Image Super-Resolution with Transformers
- Authors: Zhengyu Liang, Yingqian Wang, Longguang Wang, Jungang Yang, Shilin
Zhou
- Abstract summary: CNN-based methods have achieved remarkable performance in LF image SR.
We propose a simple but effective Transformer-based method for LF image SR.
Our method achieves superior SR performance with a small model size and low computational cost.
- Score: 11.104338786168324
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Light field (LF) image super-resolution (SR) aims at reconstructing
high-resolution LF images from their low-resolution counterparts. Although
CNN-based methods have achieved remarkable performance in LF image SR, these
methods cannot fully model the non-local properties of the 4D LF data. In this
paper, we propose a simple but effective Transformer-based method for LF image
SR. In our method, an angular Transformer is designed to incorporate
complementary information among different views, and a spatial Transformer is
developed to capture both local and long-range dependencies within each
sub-aperture image. With the proposed angular and spatial Transformers, the
beneficial information in an LF can be fully exploited and the SR performance
is boosted. We validate the effectiveness of our angular and spatial
Transformers through extensive ablation studies, and compare our method to
recent state-of-the-art methods on five public LF datasets. Our method achieves
superior SR performance with a small model size and low computational cost.
Related papers
- Mamba-based Light Field Super-Resolution with Efficient Subspace Scanning [48.99361249764921]
Transformer-based methods have demonstrated impressive performance in 4D light field (LF) super-resolution.
However, their quadratic complexity hinders the efficient processing of high resolution 4D inputs.
We propose a Mamba-based Light Field Super-Resolution method, named MLFSR, by designing an efficient subspace scanning strategy.
arXiv Detail & Related papers (2024-06-23T11:28:08Z) - Efficient Visual State Space Model for Image Deblurring [83.57239834238035]
Convolutional neural networks (CNNs) and Vision Transformers (ViTs) have achieved excellent performance in image restoration.
We propose a simple yet effective visual state space model (EVSSM) for image deblurring.
arXiv Detail & Related papers (2024-05-23T09:13:36Z) - Enhancing Low-light Light Field Images with A Deep Compensation Unfolding Network [52.77569396659629]
This paper presents the deep compensation network unfolding (DCUNet) for restoring light field (LF) images captured under low-light conditions.
The framework uses the intermediate enhanced result to estimate the illumination map, which is then employed in the unfolding process to produce a new enhanced result.
To properly leverage the unique characteristics of LF images, this paper proposes a pseudo-explicit feature interaction module.
arXiv Detail & Related papers (2023-08-10T07:53:06Z) - Physics-Informed Ensemble Representation for Light-Field Image
Super-Resolution [12.156009287223382]
We analyze the coordinate transformation of the light field (LF) imaging process to reveal the geometric relationship in the LF images.
We introduce a new LF subspace of virtual-slit images (VSI) that provide sub-pixel information complementary to sub-aperture images.
To super-resolve image structures from undersampled LF data, we propose a geometry-aware decoder, named EPIXformer.
arXiv Detail & Related papers (2023-05-31T16:27:00Z) - Learning Non-Local Spatial-Angular Correlation for Light Field Image
Super-Resolution [36.69391399634076]
Exploiting spatial-angular correlation is crucial to light field (LF) image super-resolution (SR)
We propose a simple yet effective method to learn the non-local spatial-angular correlation for LF image SR.
Our method can fully incorporate the information from all angular views while achieving a global receptive field along the epipolar line.
arXiv Detail & Related papers (2023-02-16T03:40:40Z) - Real-World Light Field Image Super-Resolution via Degradation Modulation [59.68036846233918]
We propose a simple yet effective method for real-world LF image SR.
A practical LF degradation model is developed to formulate the degradation process of real LF images.
A convolutional neural network is designed to incorporate the degradation prior to the SR process.
arXiv Detail & Related papers (2022-06-13T14:44:46Z) - Deep Selective Combinatorial Embedding and Consistency Regularization
for Light Field Super-resolution [93.95828097088608]
Light field (LF) images acquired by hand-held devices usually suffer from low spatial resolution.
The high-dimensionality characteristic and complex geometrical structure of LF images make the problem more challenging than traditional single-image SR.
We propose a novel learning-based LF spatial SR framework to explore the coherence among LF sub-aperture images.
Experimental results over both synthetic and real-world LF datasets demonstrate the significant advantage of our approach over state-of-the-art methods.
arXiv Detail & Related papers (2020-09-26T08:34:37Z) - Light Field Image Super-Resolution Using Deformable Convolution [46.03974092854241]
We propose a deformable convolution network (i.e., LF-DFnet) to handle the disparity problem for LF image SR.
Our LF-DFnet can generate high-resolution images with more faithful details and achieve state-of-the-art reconstruction accuracy.
arXiv Detail & Related papers (2020-07-07T15:07:33Z) - Light Field Spatial Super-resolution via Deep Combinatorial Geometry
Embedding and Structural Consistency Regularization [99.96632216070718]
Light field (LF) images acquired by hand-held devices usually suffer from low spatial resolution.
The high-dimensional spatiality characteristic and complex geometrical structure of LF images make the problem more challenging than traditional single-image SR.
We propose a novel learning-based LF framework, in which each view of an LF image is first individually super-resolved.
arXiv Detail & Related papers (2020-04-05T14:39:57Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.