SG-JND: Semantic-Guided Just Noticeable Distortion Predictor For Image Compression
- URL: http://arxiv.org/abs/2408.04273v1
- Date: Thu, 8 Aug 2024 07:14:57 GMT
- Title: SG-JND: Semantic-Guided Just Noticeable Distortion Predictor For Image Compression
- Authors: Linhan Cao, Wei Sun, Xiongkuo Min, Jun Jia, Zicheng Zhang, Zijian Chen, Yucheng Zhu, Lizhou Liu, Qiubo Chen, Jing Chen, Guangtao Zhai,
- Abstract summary: Just noticeable distortion (JND) represents the threshold of distortion in an image that is minimally perceptible to the human visual system.
Traditional JND prediction methods only rely on pixel-level or sub-band level features.
We propose a Semantic-Guided JND network to leverage semantic information for JND prediction.
- Score: 50.2496399381438
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Just noticeable distortion (JND), representing the threshold of distortion in an image that is minimally perceptible to the human visual system (HVS), is crucial for image compression algorithms to achieve a trade-off between transmission bit rate and image quality. However, traditional JND prediction methods only rely on pixel-level or sub-band level features, lacking the ability to capture the impact of image content on JND. To bridge this gap, we propose a Semantic-Guided JND (SG-JND) network to leverage semantic information for JND prediction. In particular, SG-JND consists of three essential modules: the image preprocessing module extracts semantic-level patches from images, the feature extraction module extracts multi-layer features by utilizing the cross-scale attention layers, and the JND prediction module regresses the extracted features into the final JND value. Experimental results show that SG-JND achieves the state-of-the-art performance on two publicly available JND datasets, which demonstrates the effectiveness of SG-JND and highlight the significance of incorporating semantic information in JND assessment.
Related papers
- Perceptual Learned Image Compression via End-to-End JND-Based
Optimization [15.173265255635219]
Emerging Learned image Compression (LC) achieves significant improvements in coding efficiency by end-to-end training of neural networks for compression.
Perceptual optimization of LC to comply with the Human Visual System (HVS) is among such criteria, which has not been fully explored yet.
This paper proposes a novel framework to integrate Just Noticeable Distortion (JND) principles into LC.
arXiv Detail & Related papers (2024-02-05T09:45:38Z) - IQNet: Image Quality Assessment Guided Just Noticeable Difference
Prefiltering For Versatile Video Coding [0.9403328689534943]
Image prefiltering with just noticeable distortion (JND) improves coding efficiency in a visual way by filtering the perceptually redundant information prior to compression.
This paper proposes a fine-grained JND prefiltering dataset guided by image quality assessment for accurate block-level JND modeling.
arXiv Detail & Related papers (2023-12-15T13:58:10Z) - The First Comprehensive Dataset with Multiple Distortion Types for
Visual Just-Noticeable Differences [40.50003266570956]
This work establishes a generalized JND dataset with a coarse-to-fine JND selection, which contains 106 source images and 1,642 JND maps, covering 25 distortion types.
A fine JND selection is carried out on the JND candidates with a crowdsourced subjective assessment.
arXiv Detail & Related papers (2023-03-05T03:12:57Z) - Just Noticeable Difference Modeling for Face Recognition System [69.28990314553076]
We make the first attempt to study just noticeable difference (JND) for the automatic face recognition system.
We develop a novel JND prediction model to directly infer JND images for the FR system.
Experimental results have demonstrated that the proposed model achieves higher accuracy of JND map prediction.
arXiv Detail & Related papers (2022-09-13T10:06:36Z) - Semantic Image Synthesis via Diffusion Models [159.4285444680301]
Denoising Diffusion Probabilistic Models (DDPMs) have achieved remarkable success in various image generation tasks.
Recent work on semantic image synthesis mainly follows the emphde facto Generative Adversarial Nets (GANs)
arXiv Detail & Related papers (2022-06-30T18:31:51Z) - Deep Posterior Distribution-based Embedding for Hyperspectral Image
Super-resolution [75.24345439401166]
This paper focuses on how to embed the high-dimensional spatial-spectral information of hyperspectral (HS) images efficiently and effectively.
We formulate HS embedding as an approximation of the posterior distribution of a set of carefully-defined HS embedding events.
Then, we incorporate the proposed feature embedding scheme into a source-consistent super-resolution framework that is physically-interpretable.
Experiments over three common benchmark datasets demonstrate that PDE-Net achieves superior performance over state-of-the-art methods.
arXiv Detail & Related papers (2022-05-30T06:59:01Z) - Full RGB Just Noticeable Difference (JND) Modelling [69.42889006770018]
Just Noticeable Difference (JND) has many applications in multimedia signal processing.
We propose a JND model to generate the JND by taking the characteristics of full RGB channels into account.
An RGB-JND-NET is proposed, where the visual content in full RGB channels is used to extract features for JND generation.
arXiv Detail & Related papers (2022-03-01T17:16:57Z) - Towards Top-Down Just Noticeable Difference Estimation of Natural Images [65.14746063298415]
Just noticeable difference (JND) estimation mainly dedicates to modeling the visibility masking effects of different factors in spatial and frequency domains.
In this work, we turn to a dramatically different way to address these problems with a top-down design philosophy.
Our proposed JND model can achieve better performance than several latest JND models.
arXiv Detail & Related papers (2021-08-11T06:51:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.