U-Motion: Learned Point Cloud Video Compression with U-Structured Temporal Context Generation

• URL: http://arxiv.org/abs/2411.14501v4
• Date: Thu, 20 Mar 2025 00:37:20 GMT
• Title: U-Motion: Learned Point Cloud Video Compression with U-Structured Temporal Context Generation
• Authors: Tingyu Fan, Yueyu Hu, Ran Gong, Yao Wang,
• Abstract summary: Point cloud video (PCV) is a versatile 3D representation of dynamic scenes with emerging applications.<n>This paper introduces U-Motion, a learning-based compression scheme for both PCV geometry and attributes.
• Score: 10.435212618849544
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Point cloud video (PCV) is a versatile 3D representation of dynamic scenes with emerging applications. This paper introduces U-Motion, a learning-based compression scheme for both PCV geometry and attributes. We propose a U-Structured inter-frame prediction framework, U-Inter, which performs explicit motion estimation and compensation (ME/MC) at different scales with varying levels of detail. It integrates Top-Down (Fine-to-Coarse) Motion Propagation, Bottom-Up Motion Predictive Coding and Multi-scale Group Motion Compensation to enable accurate motion estimation and efficient motion compression at each scale. In addition, we design a multi-scale spatial-temporal predictive coding module to capture the cross-scale spatial redundancy remaining after U-Inter prediction. We conduct experiments following the MPEG Common Test Condition for dense dynamic point clouds and demonstrate that U-Motion can achieve significant gains over MPEG G-PCC-GesTM v3.0 and recently published learning-based methods for both geometry and attribute compression.

Related papers

• Content Adaptive based Motion Alignment Framework for Learned Video Compression [72.13599533975413]
  This paper proposes a content adaptive based motion alignment framework.<n>We first introduce a two-stage flow-guided deformable warping mechanism that refines motion compensation with coarse-to-fine offset prediction and mask modulation.<n>Second, we propose a multi-reference quality aware strategy that adjusts distortion weights based on reference quality, and applies it to hierarchical training to reduce error propagation.<n>Third, we integrate a training-free module that downsamples frames by motion magnitude and resolution to obtain smooth motion estimation.
  arXiv  Detail & Related papers  (2025-12-15T02:51:47Z)
• Bidirectional Feature-aligned Motion Transformation for Efficient Dynamic Point Cloud Compression [97.66080040613726]
  We propose a Bidirectional Feature-aligned Motion Transformation (Bi-FMT) framework that implicitly models motion in the feature space.<n>Bi-FMT aligns features across both past and future frames to produce temporally consistent latent representations.<n>We show Bi-FMT surpasses D-DPCC and AdaDPCC in both compression efficiency and runtime.
  arXiv  Detail & Related papers  (2025-09-18T03:51:06Z)
• D-FCGS: Feedforward Compression of Dynamic Gaussian Splatting for Free-Viewpoint Videos [12.24209693552492]
  Free-viewpoint video (FVV) enables immersive 3D experiences, but efficient compression of dynamic 3D representations remains a major challenge.<n>This paper presents Feedforward Compression of Dynamic Gaussian Splatting (D-FCGS), a novel feedforward framework for compressing temporally correlated Gaussian point cloud sequences.<n> Experiments show that it matches the rate-distortion performance of optimization-based methods, achieving over 40 times compression in under 2 seconds.
  arXiv  Detail & Related papers  (2025-07-08T10:39:32Z)
• Motion-Aware Concept Alignment for Consistent Video Editing [57.08108545219043]
  We introduce MoCA-Video (Motion-Aware Concept Alignment in Video), a training-free framework bridging the gap between image-domain semantic mixing and video.<n>Given a generated video and a user-provided reference image, MoCA-Video injects the semantic features of the reference image into a specific object within the video.<n>We evaluate MoCA's performance using the standard SSIM, image-level LPIPS, temporal LPIPS, and introduce a novel metric CASS (Conceptual Alignment Shift Score) to evaluate the consistency and effectiveness of the visual shifts between the source prompt and the modified video frames
  arXiv  Detail & Related papers  (2025-06-01T13:28:04Z)
• Uniformly Accelerated Motion Model for Inter Prediction [38.34487653360328]
  In natural videos, there are usually multiple moving objects with variable velocity, resulting in complex motion fields that are difficult to represent compactly. In Versatile Video Coding (VVC), existing inter prediction methods assume uniform speed motion between consecutive frames. We introduce a uniformly accelerated motion model (UAMM) to exploit motion-related elements (velocity, acceleration) of moving objects between the video frames.
  arXiv  Detail & Related papers  (2024-07-16T09:46:29Z)
• Multiscale Motion-Aware and Spatial-Temporal-Channel Contextual Coding Network for Learned Video Compression [24.228981098990726]
  We propose a motion-aware and spatial-temporal-channel contextual coding based video compression network (MASTC-VC) Our proposed MASTC-VC is surprior to previous state-of-the-art (SOTA) methods on three public benchmark datasets. Our method brings average 10.15% BD-rate savings against H.265/HEVC (HM-16.20) in PSNR metric and average 23.93% BD-rate savings against H.266/VVC (VTM-13.2) in MS-SSIM metric.
  arXiv  Detail & Related papers  (2023-10-19T13:32:38Z)
• Spatial-Temporal Transformer based Video Compression Framework [44.723459144708286]
  We propose a novel Spatial-Temporal Transformer based Video Compression (STT-VC) framework. It contains a Relaxed Deformable Transformer (RDT) with Uformer based offsets estimation for motion estimation and compensation, a Multi-Granularity Prediction (MGP) module based on multi-reference frames for prediction refinement, and a Spatial Feature Distribution prior based Transformer (SFD-T) for efficient temporal-spatial joint residual compression. Experimental results demonstrate that our method achieves the best result with 13.5% BD-Rate saving over VTM.
  arXiv  Detail & Related papers  (2023-09-21T09:23:13Z)
• Learning Dynamic Point Cloud Compression via Hierarchical Inter-frame Block Matching [35.80653765524654]
  3D dynamic point cloud (DPC) compression relies on mining its temporal context. This paper proposes a learning-based DPC compression framework via hierarchical block-matching-based inter-prediction module.
  arXiv  Detail & Related papers  (2023-05-09T11:44:13Z)
• You Can Ground Earlier than See: An Effective and Efficient Pipeline for Temporal Sentence Grounding in Compressed Videos [56.676761067861236]
  Given an untrimmed video, temporal sentence grounding aims to locate a target moment semantically according to a sentence query. Previous respectable works have made decent success, but they only focus on high-level visual features extracted from decoded frames. We propose a new setting, compressed-domain TSG, which directly utilizes compressed videos rather than fully-decompressed frames as the visual input.
  arXiv  Detail & Related papers  (2023-03-14T12:53:27Z)
• Learned Video Compression via Heterogeneous Deformable Compensation Network [78.72508633457392]
  We propose a learned video compression framework via heterogeneous deformable compensation strategy (HDCVC) to tackle the problems of unstable compression performance. More specifically, the proposed algorithm extracts features from the two adjacent frames to estimate content-Neighborhood heterogeneous deformable (HetDeform) kernel offsets. Experimental results indicate that HDCVC achieves superior performance than the recent state-of-the-art learned video compression approaches.
  arXiv  Detail & Related papers  (2022-07-11T02:31:31Z)
• D-DPCC: Deep Dynamic Point Cloud Compression via 3D Motion Prediction [18.897023700334458]
  This paper proposes a novel 3D sparse convolution-based Deep Dynamic Point Cloud Compression network. It compensates and compress the DPC geometry with 3D motion estimation and motion compensation in the feature space. The experimental result shows that the proposed D-DPCC framework achieves an average 76% BD-Rate (Bjontegaard Delta Rate) gains against state-of-the-art Video-based Point Cloud Compression (V-PCC) v13 in inter mode.
  arXiv  Detail & Related papers  (2022-05-02T18:10:45Z)
• Unsupervised Motion Representation Learning with Capsule Autoencoders [54.81628825371412]
  Motion Capsule Autoencoder (MCAE) models motion in a two-level hierarchy. MCAE is evaluated on a novel Trajectory20 motion dataset and various real-world skeleton-based human action datasets.
  arXiv  Detail & Related papers  (2021-10-01T16:52:03Z)
• EAN: Event Adaptive Network for Enhanced Action Recognition [66.81780707955852]
  We propose a unified action recognition framework to investigate the dynamic nature of video content. First, when extracting local cues, we generate the spatial-temporal kernels of dynamic-scale to adaptively fit the diverse events. Second, to accurately aggregate these cues into a global video representation, we propose to mine the interactions only among a few selected foreground objects by a Transformer.
  arXiv  Detail & Related papers  (2021-07-22T15:57:18Z)
• FVC: A New Framework towards Deep Video Compression in Feature Space [21.410266039564803]
  We propose a feature-space video coding network (FVC) by performing all major operations (i.e., motion estimation, motion compression, motion compensation and residual compression) in the feature space. The proposed framework achieves the state-of-the-art performance on four benchmark datasets including HEVC, UVG, VTL and MCL-JCV.
  arXiv  Detail & Related papers  (2021-05-20T08:55:32Z)
• Learning to Segment Rigid Motions from Two Frames [72.14906744113125]
  We propose a modular network, motivated by a geometric analysis of what independent object motions can be recovered from an egomotion field. It takes two consecutive frames as input and predicts segmentation masks for the background and multiple rigidly moving objects, which are then parameterized by 3D rigid transformations. Our method achieves state-of-the-art performance for rigid motion segmentation on KITTI and Sintel.
  arXiv  Detail & Related papers  (2021-01-11T04:20:30Z)
• Enhanced Quadratic Video Interpolation [56.54662568085176]
  We propose an enhanced quadratic video (EQVI) model to handle more complicated scenes and motion patterns. To further boost the performance, we devise a novel multi-scale fusion network (MS-Fusion) which can be regarded as a learnable augmentation process. The proposed EQVI model won the first place in the AIM 2020 Video Temporal Super-Resolution Challenge.
  arXiv  Detail & Related papers  (2020-09-10T02:31:50Z)
• Motion-Attentive Transition for Zero-Shot Video Object Segmentation [99.44383412488703]
  We present a Motion-Attentive Transition Network (MATNet) for zero-shot object segmentation. An asymmetric attention block, called Motion-Attentive Transition (MAT), is designed within a two-stream encoder. In this way, the encoder becomes deeply internative, allowing for closely hierarchical interactions between object motion and appearance.
  arXiv  Detail & Related papers  (2020-03-09T16:58:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.