Point Cloud-Assisted Neural Image Compression

• URL: http://arxiv.org/abs/2412.11771v1
• Date: Mon, 16 Dec 2024 13:44:26 GMT
• Title: Point Cloud-Assisted Neural Image Compression
• Authors: Ziqun Li, Qi Zhang, Xiaofeng Huang, Zhao Wang, Siwei Ma, Wei Yan,
• Abstract summary: In this paper, we increase image compression performance with the assistance of point cloud. We propose the point cloud-assisted neural image (PCA-NIC) to enhance the preservation of image texture and structure. Our work is the first to improve image compression performance using point cloud and achieves state-of-the-art performance.
• Score: 35.46346027449056
• License:
• Abstract: High-efficient image compression is a critical requirement. In several scenarios where multiple modalities of data are captured by different sensors, the auxiliary information from other modalities are not fully leveraged by existing image-only codecs, leading to suboptimal compression efficiency. In this paper, we increase image compression performance with the assistance of point cloud, which is widely adopted in the area of autonomous driving. We first unify the data representation for both modalities to facilitate data processing. Then, we propose the point cloud-assisted neural image codec (PCA-NIC) to enhance the preservation of image texture and structure by utilizing the high-dimensional point cloud information. We further introduce a multi-modal feature fusion transform module (MMFFT) to capture more representative image features, remove redundant information between channels and modalities that are not relevant to the image content. Our work is the first to improve image compression performance using point cloud and achieves state-of-the-art performance.

Related papers

• Sparse Point Clouds Assisted Learned Image Compression [22.991782666573933]
  We propose a new framework that uses sparse point clouds to assist in learned image compression in the autonomous driving scenario. Our proposed framework is compatible with various mainstream learned image compression models, and we validate our approach using different existing image compression methods.
  arXiv  Detail & Related papers  (2024-12-20T10:14:12Z)
• Stable Diffusion is a Natural Cross-Modal Decoder for Layered AI-generated Image Compression [7.643300240138419]
  We introduce a scalable cross-modal compression framework that incorporates multiple human-comprehensible modalities. Our framework encodes images into a layered bitstream consisting of a semantic layer that delivers high-level semantic information. Our method proficiently restores both semantic and visual details, competing against baseline approaches at extremely lows.
  arXiv  Detail & Related papers  (2024-12-17T15:01:35Z)
• Rendering-Oriented 3D Point Cloud Attribute Compression using Sparse Tensor-based Transformer [52.40992954884257]
  3D visualization techniques have fundamentally transformed how we interact with digital content. Massive data size of point clouds presents significant challenges in data compression. We propose an end-to-end deep learning framework that seamlessly integrates PCAC with differentiable rendering.
  arXiv  Detail & Related papers  (2024-11-12T16:12:51Z)
• 3D Point Cloud Compression with Recurrent Neural Network and Image Compression Methods [0.0]
  Storing and transmitting LiDAR point cloud data is essential for many AV applications. Due to the sparsity and unordered structure of the data, it is difficult to compress point cloud data to a low volume. We propose a new 3D-to-2D transformation which allows compression algorithms to efficiently exploit spatial correlations.
  arXiv  Detail & Related papers  (2024-02-18T19:08:19Z)
• Perceptual Image Compression with Cooperative Cross-Modal Side Information [53.356714177243745]
  We propose a novel deep image compression method with text-guided side information to achieve a better rate-perception-distortion tradeoff. Specifically, we employ the CLIP text encoder and an effective Semantic-Spatial Aware block to fuse the text and image features.
  arXiv  Detail & Related papers  (2023-11-23T08:31:11Z)
• You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
  Learned image compression (LIC) methods have experienced significant progress during recent years. LIC methods fail to explicitly explore the image structure and texture components crucial for image compression. We present DA-Mask that samples visible patches based on the structure and texture of original images. We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
  arXiv  Detail & Related papers  (2023-06-27T15:36:22Z)
• A Unified Image Preprocessing Framework For Image Compression [5.813935823171752]
  We propose a unified image compression preprocessing framework, called Kuchen, to improve the performance of existing codecs. The framework consists of a hybrid data labeling system along with a learning-based backbone to simulate personalized preprocessing. Results demonstrate that the modern codecs optimized by our unified preprocessing framework constantly improve the efficiency of the state-of-the-art compression.
  arXiv  Detail & Related papers  (2022-08-15T10:41:00Z)
• The Devil Is in the Details: Window-based Attention for Image Compression [58.1577742463617]
  Most existing learned image compression models are based on Convolutional Neural Networks (CNNs) In this paper, we study the effects of multiple kinds of attention mechanisms for local features learning, then introduce a more straightforward yet effective window-based local attention block. The proposed window-based attention is very flexible which could work as a plug-and-play component to enhance CNN and Transformer models.
  arXiv  Detail & Related papers  (2022-03-16T07:55:49Z)
• Variable Rate Compression for Raw 3D Point Clouds [5.107705550575662]
  We propose a novel variable rate deep compression architecture that operates on raw 3D point cloud data. Our network is capable of explicitly processing point clouds and generating a compressed description.
  arXiv  Detail & Related papers  (2022-02-28T15:15:39Z)
• Implicit Neural Representations for Image Compression [103.78615661013623]
  Implicit Neural Representations (INRs) have gained attention as a novel and effective representation for various data types. We propose the first comprehensive compression pipeline based on INRs including quantization, quantization-aware retraining and entropy coding. We find that our approach to source compression with INRs vastly outperforms similar prior work.
  arXiv  Detail & Related papers  (2021-12-08T13:02:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.