CoVStream: Edge-Cloud Collaboration for Understanding of Long Video Streams

• URL: http://arxiv.org/abs/2606.22804v2
• Date: Thu, 25 Jun 2026 06:29:16 GMT
• Title: CoVStream: Edge-Cloud Collaboration for Understanding of Long Video Streams
• Abstract summary: CoVStream is the first edge-cloud collaborative framework for understanding long video streams.<n>It distills raw video streams into compact visual features and semantic captions for transmission to the cloud, minimizing bandwidth costs.<n>Experiments show that CoVStream reduces bandwidth usage by 87.6% while retaining 99.2% of the cloud baseline accuracy.
• Score: 49.92157301784352
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long, continuous video streams are an increasingly critical driver of multimedia intelligence. Existing efforts often handle long videos with a sample-encode-reason approach using large models. However, they overlook a crucial deployment fact: the stream is often produced by computationally constrained devices. This forces an untenable compromise: cloud offloading unlocks strong reasoning but incurs prohibitive bandwidth overhead, while on-device processing remains limited by edge hardware capacity. Therefore, we propose CoVStream, the first edge-cloud collaborative framework for understanding long video streams. The edge node distills raw video streams into compact visual features and semantic captions for transmission to the cloud, minimizing bandwidth costs, while the cloud server integrates this data into an entity graph and global visual context, activating the heavy reasoning model only when a user query arrives. Experiments on VideoMME-Long, LVBench, and RTV-Bench show that CoVStream reduces bandwidth usage by 87.6% while retaining 99.2% of the cloud baseline accuracy on LVBench.

Related papers

• ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding [16.068802214957014]
  Streaming video understanding is a critical capability for real-world applications.<n>ShallowStream is a novel framework that leverages the shallow layers of an MLLM to simultaneously perform frame encoding and retrieval index building.<n>ShallowStream achieves on par with the strongest existing streaming methods, while reducing per-frame prefill latency and 10-second end-to-end latency by up to 52.1x and 11.9x, respectively.
  arXiv  Detail & Related papers  (2026-09-02T16:14:48Z)
• StreamFlow: Dynamic Memory Flows for Streaming Video Understanding [65.10324190760602]
  StreamFlow is an efficient visual memory framework that enables dynamic, on-demand access to historical visual information.<n>It achieves state-of-the-art streaming video understanding performance, reaching 67.73% overall accuracy on StreamingBench.<n>Relative to the vanilla setting, it improves the visual attention score (VAS) by 59.1% while reducing end-to-end latency and peak memory by 50.4% and 21.1%, respectively.
  arXiv  Detail & Related papers  (2026-08-11T14:19:03Z)
• ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inference [28.470202983949886]
  ViCoStream is a stage-wise streaming framework that combines chunk-wise execution, visual token control, bounded visual attention, query-side retrieval, and memory costs.<n>We show that ViCoStream achieves 134 FPS video throughput and less than 50 ms TTFT on a single A100 GPU while maintaining accuracy close to full-ordinated baselines.
  arXiv  Detail & Related papers  (2026-06-18T06:57:31Z)
• Progressive Semantic Communication for Efficient Edge-Cloud Vision-Language Models [0.3004066195320147]
  Vision-Language Models (VLMs) are challenging to deploy on resource-constrained embedded platforms.<n>Full offloading inference to the cloud is often impractical in bandwidth-limited environments.<n>We propose a progressive semantic communication framework for edge-cloud VLM inference.
  arXiv  Detail & Related papers  (2026-04-29T10:16:06Z)
• CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference [4.835489391255295]
  We present CodecSight, a streaming video analytics system built on a key observation that video codecs already extract the temporal and spatial structure of each stream as a byproduct of compression.<n>CodecSight treats this metadata as a low-cost signal to unify runtime optimization across video decoding, visual processing, and LLM prefilling, with transmission reduction as an inherent benefit of operating directly on compressed bitstreams.<n> Experiments show that CodecSight achieves an improvement in throughput of up to 3$times$, and a reduction of up to 87% in GPU compute over state-of-the-art baselines, maintaining competitive accuracy with
  arXiv  Detail & Related papers  (2026-04-07T16:31:45Z)
• Thinking in Streaming Video [30.61790766076081]
  ThinkStream is a framework for streaming video reasoning based on a Watch--Think--Speak paradigm.<n> Reasoning-Compressed Streaming Memory (RCSM) treats intermediate reasoning traces as compact semantic memory.<n>Experiments on multiple streaming video benchmarks show that ThinkStream significantly outperforms existing online video models.
  arXiv  Detail & Related papers  (2026-03-13T12:33:36Z)
• Video Streaming Thinking: VideoLLMs Can Watch and Think Simultaneously [69.0264594684213]
  Video Streaming Thinking (VST) is a novel paradigm for streaming video understanding.<n>It supports a thinking while watching mechanism, which activates reasoning over incoming video clips during streaming.<n>VST improves timely comprehension and coherent cognition while preserving real-time responsiveness.
  arXiv  Detail & Related papers  (2026-03-12T17:59:51Z)
• Event-VStream: Event-Driven Real-Time Understanding for Long Video Streams [11.495597616926274]
  Event-VStream represents continuous video as a sequence of discrete, semantically coherent events.<n>System detects meaningful state transitions by integrating motion, semantic, and predictive cues.<n>System maintains around 70% GPT-5 win rate on 2-hour Ego4D streams.
  arXiv  Detail & Related papers  (2026-01-22T05:05:53Z)
• Flash-VStream: Efficient Real-Time Understanding for Long Video Streams [64.25549822010372]
  Flash-VStream is a video language model capable of processing extremely long videos and responding to user queries in real time.<n>Compared to existing models, Flash-VStream achieves significant reductions in inference latency.
  arXiv  Detail & Related papers  (2025-06-30T13:17:49Z)
• Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams [78.72965584414368]
  We present Flash-VStream, a video-language model that simulates the memory mechanism of human. Compared to existing models, Flash-VStream achieves significant reductions in latency inference and VRAM consumption. We propose VStream-QA, a novel question answering benchmark specifically designed for online video streaming understanding.
  arXiv  Detail & Related papers  (2024-06-12T11:07:55Z)
• Looking Backward: Streaming Video-to-Video Translation with Feature Banks [65.46145157488344]
  StreamV2V is a diffusion model that achieves real-time streaming video-to-video (V2V) translation with user prompts.<n>It can run 20 FPS on one A100 GPU, being 15x, 46x, 108x, and 158x faster than FlowVid, CoDeF, Rerender, and TokenFlow.
  arXiv  Detail & Related papers  (2024-05-24T17:53:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.