Image Compression with Encoder-Decoder Matched Semantic Segmentation
- URL: http://arxiv.org/abs/2101.09642v2
- Date: Sat, 30 Jan 2021 05:50:57 GMT
- Title: Image Compression with Encoder-Decoder Matched Semantic Segmentation
- Authors: Trinh Man Hoang, Jinjia Zhou, Yibo Fan
- Abstract summary: layered image compression is a promising direction.
Some works transmit the semantic segment together with the compressed image data.
We propose a new layered image compression framework with encoder matched semantic segmentation (EDMS)
The proposed EDMS framework can get up to 35.31% BD-rate reduction over the HEVC-based (BPG) encoding time saving.
- Score: 15.536056887418676
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: In recent years, layered image compression is demonstrated to be a promising
direction, which encodes a compact representation of the input image and apply
an up-sampling network to reconstruct the image. To further improve the quality
of the reconstructed image, some works transmit the semantic segment together
with the compressed image data. Consequently, the compression ratio is also
decreased because extra bits are required for transmitting the semantic
segment. To solve this problem, we propose a new layered image compression
framework with encoder-decoder matched semantic segmentation (EDMS). And then,
followed by the semantic segmentation, a special convolution neural network is
used to enhance the inaccurate semantic segment. As a result, the accurate
semantic segment can be obtained in the decoder without requiring extra bits.
The experimental results show that the proposed EDMS framework can get up to
35.31% BD-rate reduction over the HEVC-based (BPG) codec, 5% bitrate, and 24%
encoding time saving compare to the state-of-the-art semantic-based image
codec.
Related papers
- MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Model [78.4051835615796]
This paper proposes a method called Multimodal Image Semantic Compression.
It consists of an LMM encoder for extracting the semantic information of the image, a map encoder to locate the region corresponding to the semantic, an image encoder generates an extremely compressed bitstream, and a decoder reconstructs the image based on the above information.
It can achieve optimal consistency and perception results while saving perceptual 50%, which has strong potential applications in the next generation of storage and communication.
arXiv Detail & Related papers (2024-02-26T17:11:11Z) - Extreme Image Compression using Fine-tuned VQGANs [43.43014096929809]
We introduce vector quantization (VQ)-based generative models into the image compression domain.
The codebook learned by the VQGAN model yields a strong expressive capacity.
The proposed framework outperforms state-of-the-art codecs in terms of perceptual quality-oriented metrics.
arXiv Detail & Related papers (2023-07-17T06:14:19Z) - Cross Modal Compression: Towards Human-comprehensible Semantic
Compression [73.89616626853913]
Cross modal compression is a semantic compression framework for visual data.
We show that our proposed CMC can achieve encouraging reconstructed results with an ultrahigh compression ratio.
arXiv Detail & Related papers (2022-09-06T15:31:11Z) - Semantic Segmentation in Learned Compressed Domain [21.53261818914534]
We propose a method based on the compressed domain to improve segmentation tasks.
Two different modules are explored and analyzed to help the compressed representation be transformed as the features in the segmentation network.
arXiv Detail & Related papers (2022-09-03T07:59:34Z) - Reducing Redundancy in the Bottleneck Representation of the Autoencoders [98.78384185493624]
Autoencoders are a type of unsupervised neural networks, which can be used to solve various tasks.
We propose a scheme to explicitly penalize feature redundancies in the bottleneck representation.
We tested our approach across different tasks: dimensionality reduction using three different dataset, image compression using the MNIST dataset, and image denoising using fashion MNIST.
arXiv Detail & Related papers (2022-02-09T18:48:02Z) - A New Image Codec Paradigm for Human and Machine Uses [53.48873918537017]
A new scalable image paradigm for both human and machine uses is proposed in this work.
The high-level instance segmentation map and the low-level signal features are extracted with neural networks.
An image is designed and trained to achieve the general-quality image reconstruction with the 16-bit gray-scale profile and signal features.
arXiv Detail & Related papers (2021-12-19T06:17:38Z) - Enhanced Standard Compatible Image Compression Framework based on
Auxiliary Codec Networks [8.440333621142226]
We propose a novel standard compatible image compression framework based on Auxiliary Codec Networks (ACNs)
ACNs are designed to imitate image degradation operations of the existing, which delivers more accurate gradients to the compact representation network.
We demonstrate that our proposed framework based on JPEG and High Efficiency Video Coding (HEVC) standard substantially outperforms existing image compression algorithms in a standard compatible manner.
arXiv Detail & Related papers (2020-09-30T15:42:06Z) - Modeling Lost Information in Lossy Image Compression [72.69327382643549]
Lossy image compression is one of the most commonly used operators for digital images.
We propose a novel invertible framework called Invertible Lossy Compression (ILC) to largely mitigate the information loss problem.
arXiv Detail & Related papers (2020-06-22T04:04:56Z) - Learning Better Lossless Compression Using Lossy Compression [100.50156325096611]
We leverage the powerful lossy image compression algorithm BPG to build a lossless image compression system.
We model the distribution of the residual with a convolutional neural network-based probabilistic model that is conditioned on the BPG reconstruction.
Finally, the image is stored using the concatenation of the bitstreams produced by BPG and the learned residual coder.
arXiv Detail & Related papers (2020-03-23T11:21:52Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.