Task-Oriented Communication for Edge Video Analytics

• URL: http://arxiv.org/abs/2211.14049v3
• Date: Mon, 1 Apr 2024 14:38:13 GMT
• Title: Task-Oriented Communication for Edge Video Analytics
• Authors: Jiawei Shao, Xinjie Zhang, Jun Zhang,
• Abstract summary: This paper proposes a task-oriented communication framework for edge video analytics. Multiple devices collect visual sensory data and transmit the informative features to an edge server for processing. We show that the proposed framework effectively encodes task-relevant information of video data and achieves a better rate-performance tradeoff than existing methods.
• Score: 11.03999024164301
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the development of artificial intelligence (AI) techniques and the increasing popularity of camera-equipped devices, many edge video analytics applications are emerging, calling for the deployment of computation-intensive AI models at the network edge. Edge inference is a promising solution to move the computation-intensive workloads from low-end devices to a powerful edge server for video analytics, but the device-server communications will remain a bottleneck due to the limited bandwidth. This paper proposes a task-oriented communication framework for edge video analytics, where multiple devices collect the visual sensory data and transmit the informative features to an edge server for processing. To enable low-latency inference, this framework removes video redundancy in spatial and temporal domains and transmits minimal information that is essential for the downstream task, rather than reconstructing the videos at the edge server. Specifically, it extracts compact task-relevant features based on the deterministic information bottleneck (IB) principle, which characterizes a tradeoff between the informativeness of the features and the communication cost. As the features of consecutive frames are temporally correlated, we propose a temporal entropy model (TEM) to reduce the bitrate by taking the previous features as side information in feature encoding. To further improve the inference performance, we build a spatial-temporal fusion module at the server to integrate features of the current and previous frames for joint inference. Extensive experiments on video analytics tasks evidence that the proposed framework effectively encodes task-relevant information of video data and achieves a better rate-performance tradeoff than existing methods.

Related papers

• Task-Oriented Real-time Visual Inference for IoVT Systems: A Co-design Framework of Neural Networks and Edge Deployment [61.20689382879937]
  Task-oriented edge computing addresses this by shifting data analysis to the edge. Existing methods struggle to balance high model performance with low resource consumption. We propose a novel co-design framework to optimize neural network architecture.
  arXiv  Detail & Related papers  (2024-10-29T19:02:54Z)
• Edge Computing Enabled Real-Time Video Analysis via Adaptive Spatial-Temporal Semantic Filtering [18.55091203660391]
  This paper proposes a novel edge computing enabled real-time video analysis system for intelligent visual devices. The proposed system consists of a tracking-assisted object detection module (TAODM) and a region of interesting module (ROIM) TAODM adaptively determines the offloading decision to process each video frame locally with a tracking algorithm or to offload it to the edge server inferred by an object detection model.
  arXiv  Detail & Related papers  (2024-02-29T07:42:03Z)
• Spatio-temporal Prompting Network for Robust Video Feature Extraction [74.54597668310707]
  Frametemporal is one of the main challenges in the field of video understanding. Recent approaches exploit transformer-based integration modules to obtain quality-of-temporal information. We present a neat and unified framework called N-Temporal Prompting Network (NNSTP) It can efficiently extract video features by adjusting the input features in the network backbone.
  arXiv  Detail & Related papers  (2024-02-04T17:52:04Z)
• Semantics-Driven Cloud-Edge Collaborative Inference [1.441340412842035]
  This paper proposes a semantics-driven cloud-edge collaborative approach for accelerating video inference. The method separates semantics extraction and recognition, allowing edge servers to only extract visual semantics from video frames. Experiments demonstrate significant improvements in end-to-end inference speed (up to 5x faster), throughput (up to 9 FPS), and reduced traffic volumes.
  arXiv  Detail & Related papers  (2023-09-27T06:53:09Z)
• Spatiotemporal Attention-based Semantic Compression for Real-time Video Recognition [117.98023585449808]
  We propose a temporal attention-based autoencoder (STAE) architecture to evaluate the importance of frames and pixels in each frame. We develop a lightweight decoder that leverages a 3D-2D CNN combined to reconstruct missing information. Experimental results show that ViT_STAE can compress the video dataset H51 by 104x with only 5% accuracy loss.
  arXiv  Detail & Related papers  (2023-05-22T07:47:27Z)
• Streaming Video Model [90.24390609039335]
  We propose to unify video understanding tasks into one streaming video architecture, referred to as Streaming Vision Transformer (S-ViT) S-ViT first produces frame-level features with a memory-enabled temporally-aware spatial encoder to serve frame-based video tasks. The efficiency and efficacy of S-ViT is demonstrated by the state-of-the-art accuracy in the sequence-based action recognition.
  arXiv  Detail & Related papers  (2023-03-30T08:51:49Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• CANS: Communication Limited Camera Network Self-Configuration for Intelligent Industrial Surveillance [8.360870648463653]
  Realtime and intelligent video surveillance via camera networks involve computation-intensive vision detection tasks with massive video data. Multiple video streams compete for limited communication resources on the link between edge devices and camera networks. An adaptive camera network self-configuration method (CANS) of video surveillance is proposed to cope with multiple video streams of heterogeneous quality of service.
  arXiv  Detail & Related papers  (2021-09-13T01:54:33Z)
• Multi-Exit Semantic Segmentation Networks [78.44441236864057]
  We propose a framework for converting state-of-the-art segmentation models to MESS networks. specially trained CNNs that employ parametrised early exits along their depth to save during inference on easier samples. We co-optimise the number, placement and architecture of the attached segmentation heads, along with the exit policy, to adapt to the device capabilities and application-specific requirements.
  arXiv  Detail & Related papers  (2021-06-07T11:37:03Z)
• VID-WIN: Fast Video Event Matching with Query-Aware Windowing at the Edge for the Internet of Multimedia Things [3.222802562733787]
  VID-WIN is an adaptive 2-stage allied windowing approach to accelerate video event analytics in an edge-cloud paradigm. VID-WIN exploits the video content and input knobs to accelerate the video inference process across nodes.
  arXiv  Detail & Related papers  (2021-04-27T10:08:40Z)
• Neural Compression and Filtering for Edge-assisted Real-time Object Detection in Challenged Networks [8.291242737118482]
  We focus on edge computing supporting remote object detection by means of Deep Neural Networks (DNNs) We develop a framework to reduce the amount of data transmitted over the wireless link. The proposed technique represents an effective intermediate option between local and edge computing in a parameter region.
  arXiv  Detail & Related papers  (2020-07-31T03:11:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.