Learned Focused Plenoptic Image Compression with Microimage Preprocessing and Global Attention

• URL: http://arxiv.org/abs/2305.00489v1
• Date: Sun, 30 Apr 2023 14:24:56 GMT
• Title: Learned Focused Plenoptic Image Compression with Microimage Preprocessing and Global Attention
• Authors: Kedeng Tong, Xin Jin, Yuqing Yang, Chen Wang, Jinshi Kang, Fan Jiang
• Abstract summary: Focused plenoptic cameras can record spatial and angular information of the light field (LF) simultaneously. The existing plenoptic image compression methods present ineffectiveness to the captured images due to the complex micro-textures generated by the microlens relay imaging and long-distance correlations among the microimages. A lossy end-to-end learning architecture is proposed to compress the focused plenoptic images efficiently.
• Score: 17.05466366805901
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Focused plenoptic cameras can record spatial and angular information of the light field (LF) simultaneously with higher spatial resolution relative to traditional plenoptic cameras, which facilitate various applications in computer vision. However, the existing plenoptic image compression methods present ineffectiveness to the captured images due to the complex micro-textures generated by the microlens relay imaging and long-distance correlations among the microimages. In this paper, a lossy end-to-end learning architecture is proposed to compress the focused plenoptic images efficiently. First, a data preprocessing scheme is designed according to the imaging principle to remove the sub-aperture image ineffective pixels in the recorded light field and align the microimages to the rectangular grid. Then, the global attention module with large receptive field is proposed to capture the global correlation among the feature maps using pixel-wise vector attention computed in the resampling process. Also, a new image dataset consisting of 1910 focused plenoptic images with content and depth diversity is built to benefit training and testing. Extensive experimental evaluations demonstrate the effectiveness of the proposed approach. It outperforms intra coding of HEVC and VVC by an average of 62.57% and 51.67% bitrate reduction on the 20 preprocessed focused plenoptic images, respectively. Also, it achieves 18.73% bitrate saving and generates perceptually pleasant reconstructions compared to the state-of-the-art end-to-end image compression methods, which benefits the applications of focused plenoptic cameras greatly. The dataset and code are publicly available at https://github.com/VincentChandelier/GACN.

Related papers

• Latent Space Imaging [15.435034286180295]
  We propose to follow a similar approach for the development of artificial vision systems. Latent Space Imaging is a new paradigm that, through a combination of optics and software, directly encodes the image information into the semantically rich latent space of a generative model. We demonstrate this new principle through an initial hardware prototype based on the single pixel camera.
  arXiv  Detail & Related papers  (2024-07-09T17:17:03Z)
• Towards Real-World Focus Stacking with Deep Learning [97.34754533628322]
  We introduce a new dataset consisting of 94 high-resolution bursts of raw images with focus bracketing. This dataset is used to train the first deep learning algorithm for focus stacking capable of handling bursts of sufficient length for real-world applications.
  arXiv  Detail & Related papers  (2023-11-29T17:49:33Z)
• Pixel-Inconsistency Modeling for Image Manipulation Localization [63.54342601757723]
  Digital image forensics plays a crucial role in image authentication and manipulation localization. This paper presents a generalized and robust manipulation localization model through the analysis of pixel inconsistency artifacts. Experiments show that our method successfully extracts inherent pixel-inconsistency forgery fingerprints.
  arXiv  Detail & Related papers  (2023-09-30T02:54:51Z)
• Rank-Enhanced Low-Dimensional Convolution Set for Hyperspectral Image Denoising [50.039949798156826]
  This paper tackles the challenging problem of hyperspectral (HS) image denoising. We propose rank-enhanced low-dimensional convolution set (Re-ConvSet) We then incorporate Re-ConvSet into the widely-used U-Net architecture to construct an HS image denoising method.
  arXiv  Detail & Related papers  (2022-07-09T13:35:12Z)
• Pixel Distillation: A New Knowledge Distillation Scheme for Low-Resolution Image Recognition [124.80263629921498]
  We propose Pixel Distillation that extends knowledge distillation into the input level while simultaneously breaking architecture constraints. Such a scheme can achieve flexible cost control for deployment, as it allows the system to adjust both network architecture and image quality according to the overall requirement of resources.
  arXiv  Detail & Related papers  (2021-12-17T14:31:40Z)
• MC-Blur: A Comprehensive Benchmark for Image Deblurring [127.6301230023318]
  In most real-world images, blur is caused by different factors, e.g., motion and defocus. We construct a new large-scale multi-cause image deblurring dataset (called MC-Blur) Based on the MC-Blur dataset, we conduct extensive benchmarking studies to compare SOTA methods in different scenarios.
  arXiv  Detail & Related papers  (2021-12-01T02:10:42Z)
• Beyond Joint Demosaicking and Denoising: An Image Processing Pipeline for a Pixel-bin Image Sensor [0.883717274344425]
  Pixel binning is considered one of the most prominent solutions to tackle the hardware limitation of smartphone cameras. In this paper, we tackle the challenges of joint demosaicing and denoising (JDD) on such an image sensor by introducing a novel learning-based method. The proposed method is guided by a multi-term objective function, including two novel perceptual losses to produce visually plausible images.
  arXiv  Detail & Related papers  (2021-04-19T15:41:28Z)
• A Novel Upsampling and Context Convolution for Image Semantic Segmentation [0.966840768820136]
  Recent methods for semantic segmentation often employ an encoder-decoder structure using deep convolutional neural networks. We propose a dense upsampling convolution method based on guided filtering to effectively preserve the spatial information of the image in the network. We report a new record of 82.86% and 81.62% of pixel accuracy on ADE20K and Pascal-Context benchmark datasets, respectively.
  arXiv  Detail & Related papers  (2021-03-20T06:16:42Z)
• Deep Burst Super-Resolution [165.90445859851448]
  We propose a novel architecture for the burst super-resolution task. Our network takes multiple noisy RAW images as input, and generates a denoised, super-resolved RGB image as output. In order to enable training and evaluation on real-world data, we additionally introduce the BurstSR dataset.
  arXiv  Detail & Related papers  (2021-01-26T18:57:21Z)
• Convolutional Autoencoder for Blind Hyperspectral Image Unmixing [0.0]
  spectral unmixing is a technique to decompose a mixed pixel into two fundamental representatives: endmembers and abundances. In this paper, a novel architecture is proposed to perform blind unmixing on hyperspectral images.
  arXiv  Detail & Related papers  (2020-11-18T17:41:31Z)
• PlenoptiCam v1.0: A light-field imaging framework [8.467466998915018]
  Light-field cameras play a vital role for rich 3-D information retrieval in narrow range depth sensing applications. Key obstacle in composing light-fields from exposures taken by a plenoptic camera is to calibrate computationally, align and rearrange four-dimensional image data. Several attempts have been proposed to enhance the overall image quality by tailoring pipelines dedicated to particular plenoptic cameras.
  arXiv  Detail & Related papers  (2020-10-14T09:23:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.