End-to-end optimized image compression for machines, a study

• URL: http://arxiv.org/abs/2011.06409v1
• Date: Tue, 10 Nov 2020 20:10:43 GMT
• Title: End-to-end optimized image compression for machines, a study
• Authors: Lahiru D. Chamain, Fabien Racap\'e, Jean B\'egaint, Akshay Pushparaja, Simon Feltman
• Abstract summary: An increasing share of image and video content is analyzed by machines rather than viewed by humans. Conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception. neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model.
• Score: 3.0448872422956437
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: An increasing share of image and video content is analyzed by machines rather than viewed by humans, and therefore it becomes relevant to optimize codecs for such applications where the analysis is performed remotely. Unfortunately, conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception. However, neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model. In this paper, we propose to study an end-to-end framework enabling efficient image compression for remote machine task analysis, using a chain composed of a compression module and a task algorithm that can be optimized end-to-end. We show that it is possible to significantly improve the task accuracy when fine-tuning jointly the codec and the task networks, especially at low bit-rates. Depending on training or deployment constraints, selective fine-tuning can be applied only on the encoder, decoder or task network and still achieve rate-accuracy improvements over an off-the-shelf codec and task network. Our results also demonstrate the flexibility of end-to-end pipelines for practical applications.

Related papers

• Bridging the gap between image coding for machines and humans [20.017766644567036]
  In many use cases, such as surveillance, it is important that the visual quality is not drastically deteriorated by the compression process. Recent works on using neural network (NN) based ICM codecs have shown significant coding gains against traditional methods. We propose an effective decoder finetuning scheme based on adversarial training to significantly enhance the visual quality of ICM.
  arXiv  Detail & Related papers  (2024-01-19T14:49:56Z)
• VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision [59.632286735304156]
  It is more efficient to enhance/analyze the coded representations directly without decoding them into pixels. We propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis.
  arXiv  Detail & Related papers  (2023-06-19T03:04:57Z)
• Preprocessing Enhanced Image Compression for Machine Vision [14.895698385236937]
  We propose a preprocessing enhanced image compression method for machine vision tasks. Our framework is built upon the traditional non-differential codecs. Experimental results show our method achieves a better tradeoff between the coding and the performance of the downstream machine vision tasks by saving about 20%.
  arXiv  Detail & Related papers  (2022-06-12T03:36:38Z)
• An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic [72.35307086274912]
  High-dimension parameter model and large-scale mathematical calculation restrict execution efficiency, especially for Internet of Things (IoT) devices. We propose a new Deep Reinforcement Learning (DRL)-Soft Actor Critic for discrete (SAC-d), which generates the emphexit point, emphexit point, and emphcompressing bits by soft policy iterations. Based on the latency and accuracy aware reward design, such an computation can well adapt to the complex environment like dynamic wireless channel and arbitrary processing, and is capable of supporting the 5G URL
  arXiv  Detail & Related papers  (2022-01-09T09:31:50Z)
• Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics [101.35754364753409]
  Video Coding for Machines (VCM) is committed to bridging to an extent separate research tracks of video/image compression and feature compression. This paper summarizes VCM methodology and philosophy based on existing academia and industrial efforts.
  arXiv  Detail & Related papers  (2021-10-18T12:42:13Z)
• Quantized Neural Networks via {-1, +1} Encoding Decomposition and Acceleration [83.84684675841167]
  We propose a novel encoding scheme using -1, +1 to decompose quantized neural networks (QNNs) into multi-branch binary networks. We validate the effectiveness of our method on large-scale image classification, object detection, and semantic segmentation tasks.
  arXiv  Detail & Related papers  (2021-06-18T03:11:15Z)
• Multi-Exit Semantic Segmentation Networks [78.44441236864057]
  We propose a framework for converting state-of-the-art segmentation models to MESS networks. specially trained CNNs that employ parametrised early exits along their depth to save during inference on easier samples. We co-optimise the number, placement and architecture of the attached segmentation heads, along with the exit policy, to adapt to the device capabilities and application-specific requirements.
  arXiv  Detail & Related papers  (2021-06-07T11:37:03Z)
• End-to-end optimized image compression for multiple machine tasks [3.8323580808203785]
  We introduce 'Connectors' that are inserted between the decoder and the task algorithms to enable a direct transformation of the compressed content. We demonstrate the effectiveness of the proposed method by achieving significant rate-accuracy performance improvement for both image classification and object segmentation.
  arXiv  Detail & Related papers  (2021-03-06T19:09:05Z)
• Enhanced Standard Compatible Image Compression Framework based on Auxiliary Codec Networks [8.440333621142226]
  We propose a novel standard compatible image compression framework based on Auxiliary Codec Networks (ACNs) ACNs are designed to imitate image degradation operations of the existing, which delivers more accurate gradients to the compact representation network. We demonstrate that our proposed framework based on JPEG and High Efficiency Video Coding (HEVC) standard substantially outperforms existing image compression algorithms in a standard compatible manner.
  arXiv  Detail & Related papers  (2020-09-30T15:42:06Z)
• Analytic Simplification of Neural Network based Intra-Prediction Modes for Video Compression [10.08097582267397]
  This paper presents two ways to derive simplified intra-prediction from learnt models. It shows that these streamlined techniques can lead to efficient compression solutions.
  arXiv  Detail & Related papers  (2020-04-23T10:25:54Z)
• End-to-End Facial Deep Learning Feature Compression with Teacher-Student Enhancement [57.18801093608717]
  We propose a novel end-to-end feature compression scheme by leveraging the representation and learning capability of deep neural networks. In particular, the extracted features are compactly coded in an end-to-end manner by optimizing the rate-distortion cost. We verify the effectiveness of the proposed model with the facial feature, and experimental results reveal better compression performance in terms of rate-accuracy.
  arXiv  Detail & Related papers  (2020-02-10T10:08:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.