Machine vision-aware quality metrics for compressed image and video assessment

• URL: http://arxiv.org/abs/2411.06776v1
• Date: Mon, 11 Nov 2024 08:07:34 GMT
• Title: Machine vision-aware quality metrics for compressed image and video assessment
• Authors: Mikhail Dremin, Konstantin Kozhemyakov, Ivan Molodetskikh, Malakhov Kirill, Artur Sagitov, Dmitriy Vatolin,
• Abstract summary: Modern video-analysis efforts involve so much data that they necessitate machine-vision processing with minimal human intervention. This paper explores the effects of compression on detection and recognition algorithms. It introduces novel full-reference image/video-quality metrics for each task, tailored to machine vision.
• Score: 0.0
• License:
• Abstract: A main goal in developing video-compression algorithms is to enhance human-perceived visual quality while maintaining file size. But modern video-analysis efforts such as detection and recognition, which are integral to video surveillance and autonomous vehicles, involve so much data that they necessitate machine-vision processing with minimal human intervention. In such cases, the video codec must be optimized for machine vision. This paper explores the effects of compression on detection and recognition algorithms (objects, faces, and license plates) and introduces novel full-reference image/video-quality metrics for each task, tailored to machine vision. Experimental results indicate our proposed metrics correlate better with the machine-vision results for the respective tasks than do existing image/video-quality metrics.

Related papers

• VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision [59.632286735304156]
  It is more efficient to enhance/analyze the coded representations directly without decoding them into pixels. We propose a versatile neural video coding (VNVC) framework, which targets learning compact representations to support both reconstruction and direct enhancement/analysis.
  arXiv  Detail & Related papers  (2023-06-19T03:04:57Z)
• Perceptual Video Coding for Machines via Satisfied Machine Ratio Modeling [66.56355316611598]
  Satisfied Machine Ratio (SMR) is a metric that evaluates the perceptual quality of compressed images and videos for machines. SMR enables perceptual coding for machines and propels Video Coding for Machines from specificity to generality.
  arXiv  Detail & Related papers  (2022-11-13T03:16:36Z)
• Scalable Video Coding for Humans and Machines [42.870358996305356]
  We propose a scalable video coding framework that supports machine vision through its base layer bitstream and human vision via its enhancement layer bitstream. The proposed framework includes components from both conventional and Deep Neural Network (DNN)-based video coding.
  arXiv  Detail & Related papers  (2022-08-04T07:45:41Z)
• Video Coding for Machine: Compact Visual Representation Compression for Intelligent Collaborative Analytics [101.35754364753409]
  Video Coding for Machines (VCM) is committed to bridging to an extent separate research tracks of video/image compression and feature compression. This paper summarizes VCM methodology and philosophy based on existing academia and industrial efforts.
  arXiv  Detail & Related papers  (2021-10-18T12:42:13Z)
• End-to-end optimized image compression for multiple machine tasks [3.8323580808203785]
  We introduce 'Connectors' that are inserted between the decoder and the task algorithms to enable a direct transformation of the compressed content. We demonstrate the effectiveness of the proposed method by achieving significant rate-accuracy performance improvement for both image classification and object segmentation.
  arXiv  Detail & Related papers  (2021-03-06T19:09:05Z)
• Human-Machine Collaborative Video Coding Through Cuboidal Partitioning [26.70051123157869]
  We propose a video coding framework by leveraging on to the commonality that exists between human vision and machine vision applications using cuboids. Cuboids, estimated rectangular regions over a video frame, are computationally efficient, has a compact representation and object centric. Herein cuboidal feature descriptors are extracted from the current frame and then employed for accomplishing a machine vision task in the form of object detection.
  arXiv  Detail & Related papers  (2021-02-02T04:44:45Z)
• Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics [127.65410486227007]
  Video coding, which targets to compress and reconstruct the whole frame, and feature compression, which only preserves and transmits the most critical information, stand at two ends of the scale. Recent endeavors in imminent trends of video compression, e.g. deep learning based coding tools and end-to-end image/video coding, and MPEG-7 compact feature descriptor standards, promote the sustainable and fast development in their own directions. In this paper, thanks to booming AI technology, e.g. prediction and generation models, we carry out exploration in the new area, Video Coding for Machines (VCM), arising from the emerging MPEG
  arXiv  Detail & Related papers  (2020-01-10T17:24:13Z)
• An Emerging Coding Paradigm VCM: A Scalable Coding Approach Beyond Feature and Signal [99.49099501559652]
  Video Coding for Machine (VCM) aims to bridge the gap between visual feature compression and classical video coding. We employ a conditional deep generation network to reconstruct video frames with the guidance of learned motion pattern. By learning to extract sparse motion pattern via a predictive model, the network elegantly leverages the feature representation to generate the appearance of to-be-coded frames.
  arXiv  Detail & Related papers  (2020-01-09T14:18:18Z)
• Towards Coding for Human and Machine Vision: A Scalable Image Coding Approach [104.02201472370801]
  We come up with a novel image coding framework by leveraging both the compressive and the generative models. By introducing advanced generative models, we train a flexible network to reconstruct images from compact feature representations and the reference pixels. Experimental results demonstrate the superiority of our framework in both human visual quality and facial landmark detection.
  arXiv  Detail & Related papers  (2020-01-09T10:37:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.