WVSC: Wireless Video Semantic Communication with Multi-frame Compensation

• URL: http://arxiv.org/abs/2503.21197v1
• Date: Thu, 27 Mar 2025 06:27:15 GMT
• Title: WVSC: Wireless Video Semantic Communication with Multi-frame Compensation
• Authors: Bingyan Xie, Yongpeng Wu, Yuxuan Shi, Biqian Feng, Wenjun Zhang, Jihong Park, Tony Q. S. Quek,
• Abstract summary: Existing wireless video transmission schemes directly conduct video coding in pixel level.<n>We propose a wireless video semantic communication framework, abbreviated as WVSC, which integrates the idea of semantic communication into wireless video transmission scenarios.
• Score: 56.63352157833874
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing wireless video transmission schemes directly conduct video coding in pixel level, while neglecting the inner semantics contained in videos. In this paper, we propose a wireless video semantic communication framework, abbreviated as WVSC, which integrates the idea of semantic communication into wireless video transmission scenarios. WVSC first encodes original video frames as semantic frames and then conducts video coding based on such compact representations, enabling the video coding in semantic level rather than pixel level. Moreover, to further reduce the communication overhead, a reference semantic frame is introduced to substitute motion vectors of each frame in common video coding methods. At the receiver, multi-frame compensation (MFC) is proposed to produce compensated current semantic frame with a multi-frame fusion attention module. With both the reference frame transmission and MFC, the bandwidth efficiency improves with satisfying video transmission performance. Experimental results verify the performance gain of WVSC over other DL-based methods e.g. DVSC about 1 dB and traditional schemes about 2 dB in terms of PSNR.

Related papers

• Generative Video Semantic Communication via Multimodal Semantic Fusion with Large Model [55.71885688565501]
  We propose a scalable generative video semantic communication framework that extracts and transmits semantic information to achieve high-quality video reconstruction.<n>Specifically, at the transmitter, description and other condition signals are extracted from the source video, functioning as text and structural semantics, respectively.<n>At the receiver, the diffusion-based GenAI large models are utilized to fuse the semantics of the multiple modalities for reconstructing the video.
  arXiv  Detail & Related papers  (2025-02-19T15:59:07Z)
• Cross-Layer Encrypted Semantic Communication Framework for Panoramic Video Transmission [11.438045765196332]
  We propose a cross-layer encrypted semantic communication (CLESC) framework for panoramic video transmission. We propose an adaptive cross-layer transmission mechanism that dynamically adjusts CRC, channel coding, and retransmission schemes based on the importance of semantic information. Compared to traditional cross-layer transmission schemes, the CLESC framework can reduce bandwidth consumption by 85%.
  arXiv  Detail & Related papers  (2024-11-19T07:18:38Z)
• When Video Coding Meets Multimodal Large Language Models: A Unified Paradigm for Video Coding [118.72266141321647]
  Cross-Modality Video Coding (CMVC) is a pioneering approach to explore multimodality representation and video generative models in video coding.<n>During decoding, previously encoded components and video generation models are leveraged to create multiple encoding-decoding modes.<n>Experiments indicate that TT2V achieves effective semantic reconstruction, while IT2V exhibits competitive perceptual consistency.
  arXiv  Detail & Related papers  (2024-08-15T11:36:18Z)
• Synchronous Multi-modal Semantic Communication System with Packet-level Coding [20.397350999784276]
  We propose a Synchronous Multimodal Semantic Communication System (SyncSC) with Packet-Level Coding. To achieve semantic and time synchronization, 3D Morphable Mode (3DMM) coefficients and text are transmitted as semantics. To protect semantic packets under the erasure channel, we propose a packet-Level Forward Error Correction (FEC) method, called PacSC, that maintains a certain visual quality performance even at high packet loss rates.
  arXiv  Detail & Related papers  (2024-08-08T15:42:00Z)
• Neighbor Correspondence Matching for Flow-based Video Frame Synthesis [90.14161060260012]
  We introduce a neighbor correspondence matching (NCM) algorithm for flow-based frame synthesis. NCM is performed in a current-frame-agnostic fashion to establish multi-scale correspondences in the spatial-temporal neighborhoods of each pixel. coarse-scale module is designed to leverage neighbor correspondences to capture large motion, while the fine-scale module is more efficient to speed up the estimation process.
  arXiv  Detail & Related papers  (2022-07-14T09:17:00Z)
• Wireless Deep Video Semantic Transmission [14.071114007641313]
  We propose a new class of high-efficiency deep joint source-channel coding methods to achieve end-to-end video transmission over wireless channels. Our framework is collected under the name deep video semantic transmission (DVST)
  arXiv  Detail & Related papers  (2022-05-26T03:26:43Z)
• A Coding Framework and Benchmark towards Low-Bitrate Video Understanding [63.05385140193666]
  We propose a traditional-neural mixed coding framework that takes advantage of both traditional codecs and neural networks (NNs) The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved. We build a low-bitrate video understanding benchmark with three downstream tasks on eight datasets, demonstrating the notable superiority of our approach.
  arXiv  Detail & Related papers  (2022-02-06T16:29:15Z)
• Super-Resolving Compressed Video in Coding Chain [27.994055823226848]
  We present a mixed-resolution coding framework, which cooperates with a reference-based DCNN. In this novel coding chain, the reference-based DCNN learns the direct mapping from low-resolution (LR) compressed video to their high-resolution (HR) clean version at the decoder side.
  arXiv  Detail & Related papers  (2021-03-26T03:39:54Z)
• An Emerging Coding Paradigm VCM: A Scalable Coding Approach Beyond Feature and Signal [99.49099501559652]
  Video Coding for Machine (VCM) aims to bridge the gap between visual feature compression and classical video coding. We employ a conditional deep generation network to reconstruct video frames with the guidance of learned motion pattern. By learning to extract sparse motion pattern via a predictive model, the network elegantly leverages the feature representation to generate the appearance of to-be-coded frames.
  arXiv  Detail & Related papers  (2020-01-09T14:18:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.