ILCAS: Imitation Learning-Based Configuration-Adaptive Streaming for Live Video Analytics with Cross-Camera Collaboration

• URL: http://arxiv.org/abs/2308.10068v2
• Date: Thu, 2 Nov 2023 12:05:23 GMT
• Title: ILCAS: Imitation Learning-Based Configuration-Adaptive Streaming for Live Video Analytics with Cross-Camera Collaboration
• Authors: Duo Wu, Dayou Zhang, Miao Zhang, Ruoyu Zhang, Fangxin Wang, Shuguang Cui
• Abstract summary: This paper proposes the first imitation learning (IL) based configuration-adaptive live video analytics (VA) streaming system. ILCAS trains the agent with demonstrations collected from the expert which is designed as an offline optimal policy. experiments confirm the superiority of ILCAS compared with state-of-the-art solutions, with 2-20.9% improvement of mean accuracy and 19.9-85.3% reduction of chunk upload lag.
• Score: 53.29046841099947
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The high-accuracy and resource-intensive deep neural networks (DNNs) have been widely adopted by live video analytics (VA), where camera videos are streamed over the network to resource-rich edge/cloud servers for DNN inference. Common video encoding configurations (e.g., resolution and frame rate) have been identified with significant impacts on striking the balance between bandwidth consumption and inference accuracy and therefore their adaption scheme has been a focus of optimization. However, previous profiling-based solutions suffer from high profiling cost, while existing deep reinforcement learning (DRL) based solutions may achieve poor performance due to the usage of fixed reward function for training the agent, which fails to craft the application goals in various scenarios. In this paper, we propose ILCAS, the first imitation learning (IL) based configuration-adaptive VA streaming system. Unlike DRL-based solutions, ILCAS trains the agent with demonstrations collected from the expert which is designed as an offline optimal policy that solves the configuration adaption problem through dynamic programming. To tackle the challenge of video content dynamics, ILCAS derives motion feature maps based on motion vectors which allow ILCAS to visually ``perceive'' video content changes. Moreover, ILCAS incorporates a cross-camera collaboration scheme to exploit the spatio-temporal correlations of cameras for more proper configuration selection. Extensive experiments confirm the superiority of ILCAS compared with state-of-the-art solutions, with 2-20.9% improvement of mean accuracy and 19.9-85.3% reduction of chunk upload lag.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Multi-agent Assessment with QoS Enhancement for HD Map Updates in a Vehicular Network [8.853779271331508]
  Reinforcement Learning (RL) algorithms have been used to address the challenging problems in the offloading process of vehicular ad hoc networks (VANET) Deep Q-learning (DQN) and Actor-critic at the autonomous vehicle (AV) may lead to an increase in the computational load, causing a heavy burden on the computational devices and higher costs. In this paper, we assess the scalability of an application utilizing a Q-learning single-agent solution in a distributed multi-agent environment.
  arXiv  Detail & Related papers  (2024-07-31T09:17:09Z)
• Doubly-Dynamic ISAC Precoding for Vehicular Networks: A Constrained Deep Reinforcement Learning (CDRL) Approach [11.770137653756697]
  Integrated sensing and communication (ISAC) technology is essential for supporting vehicular networks. The communication channel in this scenario exhibits time variations, and the potential targets may move rapidly, resulting in double dynamics. We propose using constrained deep reinforcement learning to facilitate dynamic updates to the ISAC precoder.
  arXiv  Detail & Related papers  (2024-05-23T09:19:14Z)
• Edge Computing Enabled Real-Time Video Analysis via Adaptive Spatial-Temporal Semantic Filtering [18.55091203660391]
  This paper proposes a novel edge computing enabled real-time video analysis system for intelligent visual devices. The proposed system consists of a tracking-assisted object detection module (TAODM) and a region of interesting module (ROIM) TAODM adaptively determines the offloading decision to process each video frame locally with a tracking algorithm or to offload it to the edge server inferred by an object detection model.
  arXiv  Detail & Related papers  (2024-02-29T07:42:03Z)
• Digital Twin-Enhanced Deep Reinforcement Learning for Resource Management in Networks Slicing [46.65030115953947]
  We propose a framework consisting of a digital twin and reinforcement learning agents. Specifically, we propose to use the historical data and the neural networks to build a digital twin model to simulate the state variation law of the real environment. We also extend the framework to offline reinforcement learning, where solutions can be used to obtain intelligent decisions based solely on historical data.
  arXiv  Detail & Related papers  (2023-11-28T15:25:14Z)
• Learned Video Compression via Heterogeneous Deformable Compensation Network [78.72508633457392]
  We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance. More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
  arXiv  Detail & Related papers  (2022-07-11T02:31:31Z)
• STIP: A SpatioTemporal Information-Preserving and Perception-Augmented Model for High-Resolution Video Prediction [78.129039340528]
  We propose a Stemporal Information-Preserving and Perception-Augmented Model (STIP) to solve the above two problems. The proposed model aims to preserve thetemporal information for videos during the feature extraction and the state transitions. Experimental results show that the proposed STIP can predict videos with more satisfactory visual quality compared with a variety of state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-06-09T09:49:04Z)
• Enhancing Object Detection for Autonomous Driving by Optimizing Anchor Generation and Addressing Class Imbalance [0.0]
  This study presents an enhanced 2D object detector based on Faster R-CNN that is better suited for the context of autonomous vehicles. The proposed modifications over the Faster R-CNN do not increase computational cost and can easily be extended to optimize other anchor-based detection frameworks.
  arXiv  Detail & Related papers  (2021-04-08T16:58:31Z)
• Deep Learning-based Resource Allocation For Device-to-Device Communication [66.74874646973593]
  We propose a framework for the optimization of the resource allocation in multi-channel cellular systems with device-to-device (D2D) communication. A deep learning (DL) framework is proposed, where the optimal resource allocation strategy for arbitrary channel conditions is approximated by deep neural network (DNN) models. Our simulation results confirm that near-optimal performance can be attained with low time, which underlines the real-time capability of the proposed scheme.
  arXiv  Detail & Related papers  (2020-11-25T14:19:23Z)
• Video Face Super-Resolution with Motion-Adaptive Feedback Cell [90.73821618795512]
  Video super-resolution (VSR) methods have recently achieved a remarkable success due to the development of deep convolutional neural networks (CNN) In this paper, we propose a Motion-Adaptive Feedback Cell (MAFC), a simple but effective block, which can efficiently capture the motion compensation and feed it back to the network in an adaptive way.
  arXiv  Detail & Related papers  (2020-02-15T13:14:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.