Exploring Effective Mask Sampling Modeling for Neural Image Compression
- URL: http://arxiv.org/abs/2306.05704v1
- Date: Fri, 9 Jun 2023 06:50:20 GMT
- Title: Exploring Effective Mask Sampling Modeling for Neural Image Compression
- Authors: Lin Liu, Mingming Zhao, Shanxin Yuan, Wenlong Lyu, Wengang Zhou,
Houqiang Li, Yanfeng Wang, Qi Tian
- Abstract summary: Most existing neural image compression methods rely on side information from hyperprior or context models to eliminate spatial redundancy.
Inspired by the mask sampling modeling in recent self-supervised learning methods for natural language processing and high-level vision, we propose a novel pretraining strategy for neural image compression.
Our method achieves competitive performance with lower computational complexity compared to state-of-the-art image compression methods.
- Score: 171.35596121939238
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image compression aims to reduce the information redundancy in images. Most
existing neural image compression methods rely on side information from
hyperprior or context models to eliminate spatial redundancy, but rarely
address the channel redundancy. Inspired by the mask sampling modeling in
recent self-supervised learning methods for natural language processing and
high-level vision, we propose a novel pretraining strategy for neural image
compression. Specifically, Cube Mask Sampling Module (CMSM) is proposed to
apply both spatial and channel mask sampling modeling to image compression in
the pre-training stage. Moreover, to further reduce channel redundancy, we
propose the Learnable Channel Mask Module (LCMM) and the Learnable Channel
Completion Module (LCCM). Our plug-and-play CMSM, LCMM, LCCM modules can apply
to both CNN-based and Transformer-based architectures, significantly reduce the
computational cost, and improve the quality of images. Experiments on the
public Kodak and Tecnick datasets demonstrate that our method achieves
competitive performance with lower computational complexity compared to
state-of-the-art image compression methods.
Related papers
- CMamba: Learned Image Compression with State Space Models [31.10785880342252]
We propose a hybrid Convolution and State Space Models (SSMs) based image compression framework to achieve superior rate-distortion performance.
Specifically, CMamba introduces two key components: a Content-Adaptive SSM (CA-SSM) module and a Context-Aware Entropy (CAE) module.
Experimental results demonstrate that CMamba achieves superior rate-distortion performance.
arXiv Detail & Related papers (2025-02-07T15:07:04Z) - CALLIC: Content Adaptive Learning for Lossless Image Compression [64.47244912937204]
CALLIC sets a new state-of-the-art (SOTA) for learned lossless image compression.
We propose a content-aware autoregressive self-attention mechanism by leveraging convolutional gating operations.
During encoding, we decompose pre-trained layers, including depth-wise convolutions, using low-rank matrices and then adapt the incremental weights on testing image by Rate-guided Progressive Fine-Tuning (RPFT)
RPFT fine-tunes with gradually increasing patches that are sorted in descending order by estimated entropy, optimizing learning process and reducing adaptation time.
arXiv Detail & Related papers (2024-12-23T10:41:18Z) - Cross-Scan Mamba with Masked Training for Robust Spectral Imaging [51.557804095896174]
We propose the Cross-Scanning Mamba, named CS-Mamba, that employs a Spatial-Spectral SSM for global-local balanced context encoding.
Experiment results show that our CS-Mamba achieves state-of-the-art performance and the masked training method can better reconstruct smooth features to improve the visual quality.
arXiv Detail & Related papers (2024-08-01T15:14:10Z) - Progressive Learning with Visual Prompt Tuning for Variable-Rate Image
Compression [60.689646881479064]
We propose a progressive learning paradigm for transformer-based variable-rate image compression.
Inspired by visual prompt tuning, we use LPM to extract prompts for input images and hidden features at the encoder side and decoder side, respectively.
Our model outperforms all current variable image methods in terms of rate-distortion performance and approaches the state-of-the-art fixed image compression methods trained from scratch.
arXiv Detail & Related papers (2023-11-23T08:29:32Z) - Image Compression and Decompression Framework Based on Latent Diffusion
Model for Breast Mammography [0.0]
This research presents a novel framework for the compression and decompression of medical images utilizing the Latent Diffusion Model (LDM)
The LDM represents advancement over the denoising diffusion probabilistic model (DDPM) with a potential to yield superior image quality.
A possible application of LDM and Torchvision for image upscaling has been explored using medical image data.
arXiv Detail & Related papers (2023-10-08T22:08:59Z) - You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
Learned image compression (LIC) methods have experienced significant progress during recent years.
LIC methods fail to explicitly explore the image structure and texture components crucial for image compression.
We present DA-Mask that samples visible patches based on the structure and texture of original images.
We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
arXiv Detail & Related papers (2023-06-27T15:36:22Z) - Modeling Image Quantization Tradeoffs for Optimal Compression [0.0]
Lossy compression algorithms target tradeoffs by quantizating high frequency data to increase compression rates.
We propose a new method of optimizing quantization tables using Deep Learning and a minimax loss function.
arXiv Detail & Related papers (2021-12-14T07:35:22Z) - Modeling Lost Information in Lossy Image Compression [72.69327382643549]
Lossy image compression is one of the most commonly used operators for digital images.
We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
arXiv Detail & Related papers (2020-06-22T04:04:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.