Exploring Compressed Image Representation as a Perceptual Proxy: A Study

• URL: http://arxiv.org/abs/2401.07200v1
• Date: Sun, 14 Jan 2024 04:37:17 GMT
• Title: Exploring Compressed Image Representation as a Perceptual Proxy: A Study
• Authors: Chen-Hsiu Huang and Ja-Ling Wu
• Abstract summary: We propose an end-to-end learned image compression wherein the analysis transform is jointly trained with an object classification task. This study affirms that the compressed latent representation can predict human perceptual distance judgments with an accuracy comparable to a custom-tailored DNN-based quality metric.
• Score: 1.0878040851638
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We propose an end-to-end learned image compression codec wherein the analysis transform is jointly trained with an object classification task. This study affirms that the compressed latent representation can predict human perceptual distance judgments with an accuracy comparable to a custom-tailored DNN-based quality metric. We further investigate various neural encoders and demonstrate the effectiveness of employing the analysis transform as a perceptual loss network for image tasks beyond quality judgments. Our experiments show that the off-the-shelf neural encoder proves proficient in perceptual modeling without needing an additional VGG network. We expect this research to serve as a valuable reference developing of a semantic-aware and coding-efficient neural encoder.

Related papers

• Blind Image Quality Assessment via Transformer Predicted Error Map and Perceptual Quality Token [19.67014524146261]
  No-reference image quality assessment (NR-IQA) has gained increasing attention recently. We propose a Transformer based NR-IQA model using a predicted objective error map and perceptual quality token. Our proposed method outperforms the current state-of-the-art in both authentic and synthetic image databases.
  arXiv  Detail & Related papers  (2023-05-16T11:17:54Z)
• Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images [79.07468367923619]
  We develop convolutional neural generative coding (Conv-NGC) We implement a flexible neurobiologically-motivated algorithm that progressively refines latent state maps. We study the effectiveness of our brain-inspired neural system on the tasks of reconstruction and image denoising.
  arXiv  Detail & Related papers  (2022-11-22T06:42:41Z)
• Rate-Distortion in Image Coding for Machines [26.32381277880991]
  In many applications, such as surveillance, images are mostly transmitted for automated analysis, and rarely seen by humans. Traditional compression for this scenario has been shown to be inefficient in terms of bit-rate, likely due to the focus on human based distortion metrics. One way to create the machine side of such a scalable model is to perform feature matching of some intermediate layer in a Deep Neural Network performing the machine task.
  arXiv  Detail & Related papers  (2022-09-21T20:24:14Z)
• Toward a Geometrical Understanding of Self-supervised Contrastive Learning [55.83778629498769]
  Self-supervised learning (SSL) is one of the premier techniques to create data representations that are actionable for transfer learning in the absence of human annotations. Mainstream SSL techniques rely on a specific deep neural network architecture with two cascaded neural networks: the encoder and the projector. In this paper, we investigate how the strength of the data augmentation policies affects the data embedding.
  arXiv  Detail & Related papers  (2022-05-13T23:24:48Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Image Quality Assessment using Contrastive Learning [50.265638572116984]
  We train a deep Convolutional Neural Network (CNN) using a contrastive pairwise objective to solve the auxiliary problem. We show through extensive experiments that CONTRIQUE achieves competitive performance when compared to state-of-the-art NR image quality models. Our results suggest that powerful quality representations with perceptual relevance can be obtained without requiring large labeled subjective image quality datasets.
  arXiv  Detail & Related papers  (2021-10-25T21:01:00Z)
• Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics [101.35754364753409]
  Video Coding for Machines (VCM) is committed to bridging to an extent separate research tracks of video/image compression and feature compression. This paper summarizes VCM methodology and philosophy based on existing academia and industrial efforts.
  arXiv  Detail & Related papers  (2021-10-18T12:42:13Z)
• Blind microscopy image denoising with a deep residual and multiscale encoder/decoder network [0.0]
  Deep multiscale convolutional encoder-decoder neural network is proposed. The proposed model reaches on average 38.38 of PSNR and 0.98 of SSIM on a test set of 57458 images.
  arXiv  Detail & Related papers  (2021-05-01T14:54:57Z)
• A Neuro-Inspired Autoencoding Defense Against Adversarial Perturbations [11.334887948796611]
  Deep Neural Networks (DNNs) are vulnerable to adversarial attacks. Most effective current defense is to train the network using adversarially perturbed examples. In this paper, we investigate a radically different, neuro-inspired defense mechanism.
  arXiv  Detail & Related papers  (2020-11-21T21:03:08Z)
• Semantics-aware Adaptive Knowledge Distillation for Sensor-to-Vision Action Recognition [131.6328804788164]
  We propose a framework, named Semantics-aware Adaptive Knowledge Distillation Networks (SAKDN), to enhance action recognition in vision-sensor modality (videos) The SAKDN uses multiple wearable-sensors as teacher modalities and uses RGB videos as student modality.
  arXiv  Detail & Related papers  (2020-09-01T03:38:31Z)
• Learning to Learn to Compress [25.23586503813838]
  We present an end-to-end meta-learned system for image compression. We propose a new training paradigm for learned image compression based on meta-learning.
  arXiv  Detail & Related papers  (2020-07-31T13:13:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.