A Joint Visual Compression and Perception Framework for Neuralmorphic Spiking Camera

• URL: http://arxiv.org/abs/2503.02725v1
• Date: Tue, 04 Mar 2025 15:44:33 GMT
• Title: A Joint Visual Compression and Perception Framework for Neuralmorphic Spiking Camera
• Authors: Kexiang Feng, Chuanmin Jia, Siwei Ma, Wen Gao,
• Abstract summary: We present the notion of Spike Coding for Intelligence (SCI), wherein spike sequences are compressed and optimized for both bit-rate and task performance.<n>We achieve an average 17.25% BD-rate reduction compared to SOTA codecs and a 4.3% accuracy improvement over SpiReco for spike-based classification.
• Score: 42.74887012434441
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The advent of neuralmorphic spike cameras has garnered significant attention for their ability to capture continuous motion with unparalleled temporal resolution.However, this imaging attribute necessitates considerable resources for binary spike data storage and transmission.In light of compression and spike-driven intelligent applications, we present the notion of Spike Coding for Intelligence (SCI), wherein spike sequences are compressed and optimized for both bit-rate and task performance.Drawing inspiration from the mammalian vision system, we propose a dual-pathway architecture for separate processing of spatial semantics and motion information, which is then merged to produce features for compression.A refinement scheme is also introduced to ensure consistency between decoded features and motion vectors.We further propose a temporal regression approach that integrates various motion dynamics, capitalizing on the advancements in warping and deformation simultaneously.Comprehensive experiments demonstrate our scheme achieves state-of-the-art (SOTA) performance for spike compression and analysis.We achieve an average 17.25% BD-rate reduction compared to SOTA codecs and a 4.3% accuracy improvement over SpiReco for spike-based classification, with 88.26% complexity reduction and 42.41% inference time saving on the encoding side.

Related papers

• GANCompress: GAN-Enhanced Neural Image Compression with Binary Spherical Quantization [0.0]
  GANCompress is a novel neural compression framework that combines Binary Spherical Quantization (BSQ) with Generative Adversarial Networks (GANs)<n>We show that GANCompress achieves substantial improvement in compression efficiency -- reducing file sizes by up to 100x with minimal visual distortion.
  arXiv  Detail & Related papers  (2025-05-19T00:18:27Z)
• Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion [28.61304513668606]
  ResULIC is a residual-guided ultra lowrate image compression system.<n>It incorporates residual signals into both semantic retrieval and the diffusion-based generation process.<n>It achieves superior objective and subjective performance compared to state-of-the-art diffusion-based methods.
  arXiv  Detail & Related papers  (2025-05-13T06:51:23Z)
• 4DGC: Rate-Aware 4D Gaussian Compression for Efficient Streamable Free-Viewpoint Video [56.04182926886754]
  3D Gaussian Splatting (3DGS) has substantial potential for enabling photorealistic Free-Viewpoint Video (FVV) experiences. Existing methods typically handle dynamic 3DGS representation and compression separately, motion information and the rate-distortion trade-off during training. We propose 4DGC, a rate-aware 4D Gaussian compression framework that significantly reduces storage size while maintaining superior RD performance for FVV.
  arXiv  Detail & Related papers  (2025-03-24T08:05:27Z)
• MambaIC: State Space Models for High-Performance Learned Image Compression [53.991726013454695]
  A high-performance image compression algorithm is crucial for real-time information transmission across numerous fields. Inspired by the effectiveness of state space models (SSMs) in capturing long-range dependencies, we leverage SSMs to address computational inefficiency in existing methods. We propose an enhanced image compression approach through refined context modeling, which we term MambaIC.
  arXiv  Detail & Related papers  (2025-03-16T11:32:34Z)
• Pathology Image Compression with Pre-trained Autoencoders [52.208181380986524]
  Whole Slide Images in digital histopathology pose significant storage, transmission, and computational efficiency challenges. Standard compression methods, such as JPEG, reduce file sizes but fail to preserve fine-grained phenotypic details critical for downstream tasks. In this work, we repurpose autoencoders (AEs) designed for Latent Diffusion Models as an efficient learned compression framework for pathology images.
  arXiv  Detail & Related papers  (2025-03-14T17:01:17Z)
• HEMGS: A Hybrid Entropy Model for 3D Gaussian Splatting Data Compression [23.015728369640136]
  3D Gaussian Splatting (3DGS) is popular for 3D modeling and image rendering, but this creates big challenges in data storage and transmission.<n>We propose a hybrid entropy model for 3DGS data compression, which comprises two primary components, a hyperprior network and an autoregressive network.<n>Our method achieves about 40% average reduction in size while maintaining the rendering quality over our baseline method and achieving state-of-the-art compression results.
  arXiv  Detail & Related papers  (2024-11-27T16:08:59Z)
• Sensitivity Decouple Learning for Image Compression Artifacts Reduction [39.67836965306397]
  We propose to decouple the intrinsic attributes into two complementary features for artifacts reduction. In our proposed DAGN, we develop a cross-feature fusion module to maintain the consistency of compression-insensitive features. Our method achieves an average 2.06 dB PSNR gains on BSD500, outperforming state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-05-15T12:29:35Z)
• Compression-Realized Deep Structural Network for Video Quality Enhancement [78.13020206633524]
  This paper focuses on the task of quality enhancement for compressed videos. Most of the existing methods lack a structured design to optimally leverage the priors within compression codecs. A new paradigm is urgently needed for a more conscious'' process of quality enhancement.
  arXiv  Detail & Related papers  (2024-05-10T09:18:17Z)
• Semantic Ensemble Loss and Latent Refinement for High-Fidelity Neural Image Compression [58.618625678054826]
  This study presents an enhanced neural compression method designed for optimal visual fidelity. We have trained our model with a sophisticated semantic ensemble loss, integrating Charbonnier loss, perceptual loss, style loss, and a non-binary adversarial loss. Our empirical findings demonstrate that this approach significantly improves the statistical fidelity of neural image compression.
  arXiv  Detail & Related papers  (2024-01-25T08:11:27Z)
• ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression [18.05997169440533]
  We propose ConvNeXt-ChARM, an efficient ConvNeXt-based transform coding framework, paired with a compute-efficient channel-wise auto-regressive auto-regressive. We show that ConvNeXt-ChARM brings consistent and significant BD-rate (PSNR) reductions estimated on average to 5.24% and 1.22% over the versatile video coding (VVC) reference encoder (VTM-18.0) and the state-of-the-art learned image compression method SwinT-ChARM.
  arXiv  Detail & Related papers  (2023-07-12T11:45:54Z)
• SpikeCodec: An End-to-end Learned Compression Framework for Spiking Camera [48.194993029659685]
  Bio-inspired spike camera with continuous motion recording capability has attracted tremendous attention. We propose a novel learned spike compression framework using scene recovery, variational auto-encoder plus spike simulator. To our knowledge, it is the first data-trained model for efficient and robust spike stream compression.
  arXiv  Detail & Related papers  (2023-06-25T03:11:21Z)
• Learned Video Compression via Heterogeneous Deformable Compensation Network [78.72508633457392]
  We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance. More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
  arXiv  Detail & Related papers  (2022-07-11T02:31:31Z)
• DeepCompress: Efficient Point Cloud Geometry Compression [1.808877001896346]
  We propose a more efficient deep learning-based encoder architecture for point clouds compression. We show that incorporating the learned activation function from Efficient Neural Image Compression (CENIC) yields dramatic gains in efficiency and performance. Our proposed modifications outperform the baseline approaches by a small margin in terms of Bjontegard delta rate and PSNR values.
  arXiv  Detail & Related papers  (2021-06-02T23:18:11Z)
• Generalized Octave Convolutions for Learned Multi-Frequency Image Compression [20.504561050200365]
  We propose the first learned multi-frequency image compression and entropy coding approach. It is based on the recently developed octave convolutions to factorize the latents into high and low frequency (resolution) components. We show that the proposed generalized octave convolution can improve the performance of other auto-encoder-based computer vision tasks.
  arXiv  Detail & Related papers  (2020-02-24T01:35:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.