LGFN: Lightweight Light Field Image Super-Resolution using Local Convolution Modulation and Global Attention Feature Extraction

• URL: http://arxiv.org/abs/2409.17759v1
• Date: Thu, 26 Sep 2024 11:53:25 GMT
• Title: LGFN: Lightweight Light Field Image Super-Resolution using Local Convolution Modulation and Global Attention Feature Extraction
• Authors: Zhongxin Yu, Liang Chen, Zhiyun Zeng, Kunping Yang, Shaofei Luo, Shaorui Chen, Cheng Zhong,
• Abstract summary: We propose a lightweight model named LGFN which integrates the local and global features of different views and the features of different channels for LF image SR. Our model has a parameter of 0.45M and a FLOPs of 19.33G which has achieved a competitive effect.
• Score: 5.461017270708014
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Capturing different intensity and directions of light rays at the same scene Light field (LF) can encode the 3D scene cues into a 4D LF image which has a wide range of applications (i.e. post-capture refocusing and depth sensing). LF image super-resolution (SR) aims to improve the image resolution limited by the performance of LF camera sensor. Although existing methods have achieved promising results the practical application of these models is limited because they are not lightweight enough. In this paper we propose a lightweight model named LGFN which integrates the local and global features of different views and the features of different channels for LF image SR. Specifically owing to neighboring regions of the same pixel position in different sub-aperture images exhibit similar structural relationships we design a lightweight CNN-based feature extraction module (namely DGCE) to extract local features better through feature modulation. Meanwhile as the position beyond the boundaries in the LF image presents a large disparity we propose an efficient spatial attention module (namely ESAM) which uses decomposable large-kernel convolution to obtain an enlarged receptive field and an efficient channel attention module (namely ECAM). Compared with the existing LF image SR models with large parameter our model has a parameter of 0.45M and a FLOPs of 19.33G which has achieved a competitive effect. Extensive experiments with ablation studies demonstrate the effectiveness of our proposed method which ranked the second place in the Track 2 Fidelity & Efficiency of NTIRE2024 Light Field Super Resolution Challenge and the seventh place in the Track 1 Fidelity.

Related papers

• LFIC-DRASC: Deep Light Field Image Compression Using Disentangled Representation and Asymmetrical Strip Convolution [51.909036244222904]
  We propose an end-to-end deep LF Image Compression method using Disentangled Representation and Asymmetrical Strip Convolution. Experimental results demonstrate that the proposed LFIC-DRASC achieves an average of 20.5% bit rate reductions.
  arXiv  Detail & Related papers  (2024-09-18T05:33:42Z)
• Diffusion-based Light Field Synthesis [50.24624071354433]
  LFdiff is a diffusion-based generative framework tailored for LF synthesis. We propose DistgUnet, a disentanglement-based noise estimation network, to harness comprehensive LF representations. Extensive experiments demonstrate that LFdiff excels in synthesizing visually pleasing and disparity-controllable light fields.
  arXiv  Detail & Related papers  (2024-02-01T13:13:16Z)
• LFSRDiff: Light Field Image Super-Resolution via Diffusion Models [18.20217829625834]
  Light field (LF) image super-resolution (SR) is a challenging problem due to its inherent ill-posed nature. mainstream LF image SR methods typically adopt a deterministic approach, generating only a single output supervised by pixel-wise loss functions. We introduce LFSRDiff, the first diffusion-based LF image SR model, by incorporating the LF disentanglement mechanism.
  arXiv  Detail & Related papers  (2023-11-27T07:31:12Z)
• Enhancing Low-light Light Field Images with A Deep Compensation Unfolding Network [52.77569396659629]
  This paper presents the deep compensation network unfolding (DCUNet) for restoring light field (LF) images captured under low-light conditions. The framework uses the intermediate enhanced result to estimate the illumination map, which is then employed in the unfolding process to produce a new enhanced result. To properly leverage the unique characteristics of LF images, this paper proposes a pseudo-explicit feature interaction module.
  arXiv  Detail & Related papers  (2023-08-10T07:53:06Z)
• Probabilistic-based Feature Embedding of 4-D Light Fields for Compressive Imaging and Denoising [62.347491141163225]
  4-D light field (LF) poses great challenges in achieving efficient and effective feature embedding. We propose a probabilistic-based feature embedding (PFE), which learns a feature embedding architecture by assembling various low-dimensional convolution patterns. Our experiments demonstrate the significant superiority of our methods on both real-world and synthetic 4-D LF images.
  arXiv  Detail & Related papers  (2023-06-15T03:46:40Z)
• Sub-Aperture Feature Adaptation in Single Image Super-resolution Model for Light Field Imaging [17.721259583120396]
  This paper proposes an adaptation module in a pretrained Single Image Super Resolution (SISR) network to leverage the powerful SISR model. It is an adaptation in the SISR network to further exploit the spatial and angular information in LF images to improve the super resolution performance.
  arXiv  Detail & Related papers  (2022-07-25T03:43:56Z)
• Disentangling Light Fields for Super-Resolution and Disparity Estimation [67.50796924758221]
  Light field (LF) cameras record both intensity and directions of light rays, and encode 3D scenes into 4D LF images. It is challenging for convolutional neural networks (CNNs) to process LF images since the spatial and angular information are highly inter-twined with varying disparities. We propose a generic mechanism to disentangle these coupled information for LF image processing.
  arXiv  Detail & Related papers  (2022-02-22T01:04:41Z)
• Light Field Image Super-Resolution with Transformers [11.104338786168324]
  CNN-based methods have achieved remarkable performance in LF image SR. We propose a simple but effective Transformer-based method for LF image SR. Our method achieves superior SR performance with a small model size and low computational cost.
  arXiv  Detail & Related papers  (2021-08-17T12:58:11Z)
• Deep Selective Combinatorial Embedding and Consistency Regularization for Light Field Super-resolution [93.95828097088608]
  Light field (LF) images acquired by hand-held devices usually suffer from low spatial resolution. The high-dimensionality characteristic and complex geometrical structure of LF images make the problem more challenging than traditional single-image SR. We propose a novel learning-based LF spatial SR framework to explore the coherence among LF sub-aperture images. Experimental results over both synthetic and real-world LF datasets demonstrate the significant advantage of our approach over state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-26T08:34:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.