TMP: Temporal Motion Propagation for Online Video Super-Resolution

• URL: http://arxiv.org/abs/2312.09909v2
• Date: Mon, 18 Dec 2023 02:36:23 GMT
• Title: TMP: Temporal Motion Propagation for Online Video Super-Resolution
• Abstract summary: Online video super-resolution (online-VSR) highly relies on an effective alignment module to aggregate temporal information. Most of the existing online-VSR methods estimate the motion fields of each frame separately to perform alignment. We propose an efficient Temporal Motion propagation (TMP) method, which leverages the continuity of motion field to achieve fast pixel-level alignment.
• Score: 20.56797955203068
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online video super-resolution (online-VSR) highly relies on an effective alignment module to aggregate temporal information, while the strict latency requirement makes accurate and efficient alignment very challenging. Though much progress has been achieved, most of the existing online-VSR methods estimate the motion fields of each frame separately to perform alignment, which is computationally redundant and ignores the fact that the motion fields of adjacent frames are correlated. In this work, we propose an efficient Temporal Motion Propagation (TMP) method, which leverages the continuity of motion field to achieve fast pixel-level alignment among consecutive frames. Specifically, we first propagate the offsets from previous frames to the current frame, and then refine them in the neighborhood, which significantly reduces the matching space and speeds up the offset estimation process. Furthermore, to enhance the robustness of alignment, we perform spatial-wise weighting on the warped features, where the positions with more precise offsets are assigned higher importance. Experiments on benchmark datasets demonstrate that the proposed TMP method achieves leading online-VSR accuracy as well as inference speed. The source code of TMP can be found at https://github.com/xtudbxk/TMP.

Related papers

• Event-Based Motion Estimation via Oriented Distance Fields [55.23416502666468]
  Event-based motion estimation is central to tasks that demand high temporal resolution and robustness to fast motion.<n>We propose Oriented Distance Field Motion Estimation (ODF Motion Estimation), which replaces this optimization with a single averaging step over a precomputed field of event distance vectors.<n>On public and self-collected datasets, ODF motion estimation reaches sub-pixel accuracy at the lowest latency among compared methods.
  arXiv  Detail & Related papers  (2026-08-25T08:27:42Z)
• Compressed-Domain-Aware Online Video Super-Resolution [15.33081890735916]
  In bandwidth-limited online video streaming, videos are usually downsampled and compressed.<n>Recent online video super-resolution (online VSR) approaches achieve promising results, but they are still compute-intensive.<n>We propose a compressed-domain-aware network (CDA-VSR) for online VSR to balance quality and efficiency.
  arXiv  Detail & Related papers  (2026-03-08T15:52:48Z)
• Error-Propagation-Free Learned Video Compression With Dual-Domain Progressive Temporal Alignment [92.57576987521107]
  We propose a novel unifiedtransform framework with dual-domain progressive temporal alignment and quality-conditioned mixture-of-expert (QCMoE)<n>QCMoE allows continuous and consistent rate control with appealing R-D performance.<n> Experimental results show that the proposed method achieves competitive R-D performance compared with the state-of-the-arts.
  arXiv  Detail & Related papers  (2025-12-11T09:14:51Z)
• Bidirectional Feature-aligned Motion Transformation for Efficient Dynamic Point Cloud Compression [97.66080040613726]
  We propose a Bidirectional Feature-aligned Motion Transformation (Bi-FMT) framework that implicitly models motion in the feature space.<n>Bi-FMT aligns features across both past and future frames to produce temporally consistent latent representations.<n>We show Bi-FMT surpasses D-DPCC and AdaDPCC in both compression efficiency and runtime.
  arXiv  Detail & Related papers  (2025-09-18T03:51:06Z)
• TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion [18.191333256398845]
  A hurdle for event-based video frame (VFI) is how to deal with non-linear motion.<n>We propose a novel continuous point tracking-based VFI framework, named TimeTracker.<n>Our method outperforms prior arts in both motion estimation and frame quality.
  arXiv  Detail & Related papers  (2025-05-06T02:12:19Z)
• MATE: Motion-Augmented Temporal Consistency for Event-based Point Tracking [58.719310295870024]
  This paper presents an event-based framework for tracking any point.<n>To resolve ambiguities caused by event sparsity, a motion-guidance module incorporates kinematic vectors into the local matching process.<n>The method improves the $Survival_50$ metric by 17.9% over event-only tracking of any point baseline.
  arXiv  Detail & Related papers  (2024-12-02T09:13:29Z)
• Collaborative Feedback Discriminative Propagation for Video Super-Resolution [66.61201445650323]
  Key success of video super-resolution (VSR) methods stems mainly from exploring spatial and temporal information. Inaccurate alignment usually leads to aligned features with significant artifacts. propagation modules only propagate the same timestep features forward or backward.
  arXiv  Detail & Related papers  (2024-04-06T22:08:20Z)
• StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences [31.210626775505407]
  Occlusions between consecutive frames have long posed a significant challenge in optical flow estimation. We present a Streamlined In-batch Multi-frame (SIM) pipeline tailored to video input, attaining a similar level of time efficiency to two-frame networks. StreamFlow not only excels in terms of performance on challenging KITTI and Sintel datasets, with particular improvement in occluded areas.
  arXiv  Detail & Related papers  (2023-11-28T07:53:51Z)
• Continuous Space-Time Video Super-Resolution Utilizing Long-Range Temporal Information [48.20843501171717]
  We propose a continuous ST-VSR (CSTVSR) method that can convert the given video to any frame rate and spatial resolution. We show that the proposed algorithm has good flexibility and achieves better performance on various datasets.
  arXiv  Detail & Related papers  (2023-02-26T08:02:39Z)
• Efficient Video Deblurring Guided by Motion Magnitude [37.25713728458234]
  We propose a novel framework that utilizes the motion magnitude prior (MMP) as guidance for efficient deep video deblurring. The MMP consists of both spatial and temporal blur level information, which can be further integrated into an efficient recurrent neural network (RNN) for video deblurring.
  arXiv  Detail & Related papers  (2022-07-27T08:57:48Z)
• Fast Online Video Super-Resolution with Deformable Attention Pyramid [172.16491820970646]
  Video super-resolution (VSR) has many applications that pose strict causal, real-time, and latency constraints, including video streaming and TV. We propose a recurrent VSR architecture based on a deformable attention pyramid (DAP)
  arXiv  Detail & Related papers  (2022-02-03T17:49:04Z)
• Optical-Flow-Reuse-Based Bidirectional Recurrent Network for Space-Time Video Super-Resolution [52.899234731501075]
  Space-time video super-resolution (ST-VSR) simultaneously increases the spatial resolution and frame rate for a given video. Existing methods typically suffer from difficulties in how to efficiently leverage information from a large range of neighboring frames. We propose a coarse-to-fine bidirectional recurrent neural network instead of using ConvLSTM to leverage knowledge between adjacent frames.
  arXiv  Detail & Related papers  (2021-10-13T15:21:30Z)
• Temporal Modulation Network for Controllable Space-Time Video Super-Resolution [66.06549492893947]
  Space-time video super-resolution aims to increase the spatial and temporal resolutions of low-resolution and low-frame-rate videos. Deformable convolution based methods have achieved promising STVSR performance, but they could only infer the intermediate frame pre-defined in the training stage. We propose a Temporal Modulation Network (TMNet) to interpolate arbitrary intermediate frame(s) with accurate high-resolution reconstruction.
  arXiv  Detail & Related papers  (2021-04-21T17:10:53Z)
• FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation [97.99012124785177]
  FLAVR is a flexible and efficient architecture that uses 3D space-time convolutions to enable end-to-end learning and inference for video framesupervised. We demonstrate that FLAVR can serve as a useful self- pretext task for action recognition, optical flow estimation, and motion magnification.
  arXiv  Detail & Related papers  (2020-12-15T18:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.