Exploring Effective Mask Sampling Modeling for Neural Image Compression
- URL: http://arxiv.org/abs/2306.05704v1
- Date: Fri, 9 Jun 2023 06:50:20 GMT
- Title: Exploring Effective Mask Sampling Modeling for Neural Image Compression
- Authors: Lin Liu, Mingming Zhao, Shanxin Yuan, Wenlong Lyu, Wengang Zhou,
Houqiang Li, Yanfeng Wang, Qi Tian
- Abstract summary: Most existing neural image compression methods rely on side information from hyperprior or context models to eliminate spatial redundancy.
Inspired by the mask sampling modeling in recent self-supervised learning methods for natural language processing and high-level vision, we propose a novel pretraining strategy for neural image compression.
Our method achieves competitive performance with lower computational complexity compared to state-of-the-art image compression methods.
- Score: 171.35596121939238
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Image compression aims to reduce the information redundancy in images. Most
existing neural image compression methods rely on side information from
hyperprior or context models to eliminate spatial redundancy, but rarely
address the channel redundancy. Inspired by the mask sampling modeling in
recent self-supervised learning methods for natural language processing and
high-level vision, we propose a novel pretraining strategy for neural image
compression. Specifically, Cube Mask Sampling Module (CMSM) is proposed to
apply both spatial and channel mask sampling modeling to image compression in
the pre-training stage. Moreover, to further reduce channel redundancy, we
propose the Learnable Channel Mask Module (LCMM) and the Learnable Channel
Completion Module (LCCM). Our plug-and-play CMSM, LCMM, LCCM modules can apply
to both CNN-based and Transformer-based architectures, significantly reduce the
computational cost, and improve the quality of images. Experiments on the
public Kodak and Tecnick datasets demonstrate that our method achieves
competitive performance with lower computational complexity compared to
state-of-the-art image compression methods.
Related papers
- Zero-Shot Image Compression with Diffusion-Based Posterior Sampling [34.50287066865267]
This work addresses the gap by harnessing the image prior learned by existing pre-trained diffusion models for solving the task of lossy image compression.
Our method, PSC (Posterior Sampling-based Compression), utilizes zero-shot diffusion-based posterior samplers.
PSC achieves competitive results compared to established methods, paving the way for further exploration of pre-trained diffusion models and posterior samplers for image compression.
arXiv Detail & Related papers (2024-07-13T14:24:22Z) - Transferable Learned Image Compression-Resistant Adversarial Perturbations [66.46470251521947]
Adversarial attacks can readily disrupt the image classification system, revealing the vulnerability of DNN-based recognition tasks.
We introduce a new pipeline that targets image classification models that utilize learned image compressors as pre-processing modules.
arXiv Detail & Related papers (2024-01-06T03:03:28Z) - Progressive Learning with Visual Prompt Tuning for Variable-Rate Image
Compression [60.689646881479064]
We propose a progressive learning paradigm for transformer-based variable-rate image compression.
Inspired by visual prompt tuning, we use LPM to extract prompts for input images and hidden features at the encoder side and decoder side, respectively.
Our model outperforms all current variable image methods in terms of rate-distortion performance and approaches the state-of-the-art fixed image compression methods trained from scratch.
arXiv Detail & Related papers (2023-11-23T08:29:32Z) - Image Compression and Decompression Framework Based on Latent Diffusion
Model for Breast Mammography [0.0]
This research presents a novel framework for the compression and decompression of medical images utilizing the Latent Diffusion Model (LDM)
The LDM represents advancement over the denoising diffusion probabilistic model (DDPM) with a potential to yield superior image quality.
A possible application of LDM and Torchvision for image upscaling has been explored using medical image data.
arXiv Detail & Related papers (2023-10-08T22:08:59Z) - You Can Mask More For Extremely Low-Bitrate Image Compression [80.7692466922499]
Learned image compression (LIC) methods have experienced significant progress during recent years.
LIC methods fail to explicitly explore the image structure and texture components crucial for image compression.
We present DA-Mask that samples visible patches based on the structure and texture of original images.
We propose a simple yet effective masked compression model (MCM), the first framework that unifies LIC and LIC end-to-end for extremely low-bitrate compression.
arXiv Detail & Related papers (2023-06-27T15:36:22Z) - Neural Data-Dependent Transform for Learned Image Compression [72.86505042102155]
We build a neural data-dependent transform and introduce a continuous online mode decision mechanism to jointly optimize the coding efficiency for each individual image.
The experimental results show the effectiveness of the proposed neural-syntax design and the continuous online mode decision mechanism.
arXiv Detail & Related papers (2022-03-09T14:56:48Z) - Modeling Image Quantization Tradeoffs for Optimal Compression [0.0]
Lossy compression algorithms target tradeoffs by quantizating high frequency data to increase compression rates.
We propose a new method of optimizing quantization tables using Deep Learning and a minimax loss function.
arXiv Detail & Related papers (2021-12-14T07:35:22Z) - Learned Image Compression with Gaussian-Laplacian-Logistic Mixture Model
and Concatenated Residual Modules [22.818632387206257]
Two key components of learned image compression are the entropy model of the latent representations and the encoding/decoding network architectures.
We propose a more flexible discretized Gaussian-Laplacian-Logistic mixture model (GLLMM) for the latent representations.
In the encoding/decoding network design part, we propose a residual blocks (CRB) where multiple residual blocks are serially connected with additional shortcut connections.
arXiv Detail & Related papers (2021-07-14T02:54:22Z) - Modeling Lost Information in Lossy Image Compression [72.69327382643549]
Lossy image compression is one of the most commonly used operators for digital images.
We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
arXiv Detail & Related papers (2020-06-22T04:04:56Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.