End-to-end optimized image compression for multiple machine tasks

• URL: http://arxiv.org/abs/2103.04178v1
• Date: Sat, 6 Mar 2021 19:09:05 GMT
• Title: End-to-end optimized image compression for multiple machine tasks
• Authors: Lahiru D. Chamain, Fabien Racap\'e, Jean B\'egaint, Akshay Pushparaja and Simon Feltman
• Abstract summary: We introduce 'Connectors' that are inserted between the decoder and the task algorithms to enable a direct transformation of the compressed content. We demonstrate the effectiveness of the proposed method by achieving significant rate-accuracy performance improvement for both image classification and object segmentation.
• Score: 3.8323580808203785
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: An increasing share of captured images and videos are transmitted for storage and remote analysis by computer vision algorithms, rather than to be viewed by humans. Contrary to traditional standard codecs with engineered tools, neural network based codecs can be trained end-to-end to optimally compress images with respect to a target rate and any given differentiable performance metric. Although it is possible to train such compression tools to achieve better rate-accuracy performance for a particular computer vision task, it could be practical and relevant to re-use the compressed bit-stream for multiple machine tasks. For this purpose, we introduce 'Connectors' that are inserted between the decoder and the task algorithms to enable a direct transformation of the compressed content, which was previously optimized for a specific task, to multiple other machine tasks. We demonstrate the effectiveness of the proposed method by achieving significant rate-accuracy performance improvement for both image classification and object segmentation, using the same bit-stream, originally optimized for object detection.

Related papers

• Bridging the gap between image coding for machines and humans [20.017766644567036]
  In many use cases, such as surveillance, it is important that the visual quality is not drastically deteriorated by the compression process. Recent works on using neural network (NN) based ICM codecs have shown significant coding gains against traditional methods. We propose an effective decoder finetuning scheme based on adversarial training to significantly enhance the visual quality of ICM.
  arXiv  Detail & Related papers  (2024-01-19T14:49:56Z)
• VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision [59.632286735304156]
  It is more efficient to enhance/analyze the coded representations directly without decoding them into pixels. We propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis.
  arXiv  Detail & Related papers  (2023-06-19T03:04:57Z)
• Semantic Segmentation in Learned Compressed Domain [21.53261818914534]
  We propose a method based on the compressed domain to improve segmentation tasks. Two different modules are explored and analyzed to help the compressed representation be transformed as the features in the segmentation network.
  arXiv  Detail & Related papers  (2022-09-03T07:59:34Z)
• Analysis of the Effect of Low-Overhead Lossy Image Compression on the Performance of Visual Crowd Counting for Smart City Applications [78.55896581882595]
  Lossy image compression techniques can reduce the quality of the images, leading to accuracy degradation. In this paper, we analyze the effect of applying low-overhead lossy image compression methods on the accuracy of visual crowd counting.
  arXiv  Detail & Related papers  (2022-07-20T19:20:03Z)
• Preprocessing Enhanced Image Compression for Machine Vision [14.895698385236937]
  We propose a preprocessing enhanced image compression method for machine vision tasks. Our framework is built upon the traditional non-differential codecs. Experimental results show our method achieves a better tradeoff between the coding and the performance of the downstream machine vision tasks by saving about 20%.
  arXiv  Detail & Related papers  (2022-06-12T03:36:38Z)
• A New Image Codec Paradigm for Human and Machine Uses [53.48873918537017]
  A new scalable image paradigm for both human and machine uses is proposed in this work. The high-level instance segmentation map and the low-level signal features are extracted with neural networks. An image is designed and trained to achieve the general-quality image reconstruction with the 16-bit gray-scale profile and signal features.
  arXiv  Detail & Related papers  (2021-12-19T06:17:38Z)
• Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics [101.35754364753409]
  Video Coding for Machines (VCM) is committed to bridging to an extent separate research tracks of video/image compression and feature compression. This paper summarizes VCM methodology and philosophy based on existing academia and industrial efforts.
  arXiv  Detail & Related papers  (2021-10-18T12:42:13Z)
• Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform [58.60004238261117]
  We propose a versatile deep image compression network based on Spatial Feature Transform (SFT arXiv:1804.02815) Our model covers a wide range of compression rates using a single model, which is controlled by arbitrary pixel-wise quality maps. The proposed framework allows us to perform task-aware image compressions for various tasks.
  arXiv  Detail & Related papers  (2021-08-21T17:30:06Z)
• How to Exploit the Transferability of Learned Image Compression to Conventional Codecs [25.622863999901874]
  We show how learned image coding can be used as a surrogate to optimize an image for encoding. Our approach can remodel a conventional image to adjust for the MS-SSIM distortion with over 20% rate improvement without any decoding overhead.
  arXiv  Detail & Related papers  (2020-12-03T12:34:51Z)
• End-to-end optimized image compression for machines, a study [3.0448872422956437]
  An increasing share of image and video content is analyzed by machines rather than viewed by humans. Conventional coding tools are challenging to specialize for machine tasks as they were originally designed for human perception. neural network based codecs can be jointly trained end-to-end with any convolutional neural network (CNN)-based task model.
  arXiv  Detail & Related papers  (2020-11-10T20:10:43Z)
• Discernible Image Compression [124.08063151879173]
  This paper aims to produce compressed images by pursuing both appearance and perceptual consistency. Based on the encoder-decoder framework, we propose using a pre-trained CNN to extract features of the original and compressed images. Experiments on benchmarks demonstrate that images compressed by using the proposed method can also be well recognized by subsequent visual recognition and detection models.
  arXiv  Detail & Related papers  (2020-02-17T07:35:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.