Real-World Video for Zoom Enhancement based on Spatio-Temporal Coupling
- URL: http://arxiv.org/abs/2306.13875v1
- Date: Sat, 24 Jun 2023 06:19:00 GMT
- Title: Real-World Video for Zoom Enhancement based on Spatio-Temporal Coupling
- Authors: Zhiling Guo, Yinqiang Zheng, Haoran Zhang, Xiaodan Shi, Zekun Cai,
Ryosuke Shibasaki, Jinyue Yan
- Abstract summary: We investigate the feasibility of applying realistic multi-frame clips to enhance zoom quality via paper-temporal information coupling.
The outperformed experimental results obtained in different zoom scenarios demonstrate the superiority of integrating real-world video and STCL into existing zoom models.
- Score: 44.2753331076938
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: In recent years, single-frame image super-resolution (SR) has become more
realistic by considering the zooming effect and using real-world short- and
long-focus image pairs. In this paper, we further investigate the feasibility
of applying realistic multi-frame clips to enhance zoom quality via
spatio-temporal information coupling. Specifically, we first built a real-world
video benchmark, VideoRAW, by a synchronized co-axis optical system. The
dataset contains paired short-focus raw and long-focus sRGB videos of different
dynamic scenes. Based on VideoRAW, we then presented a Spatio-Temporal Coupling
Loss, termed as STCL. The proposed STCL is intended for better utilization of
information from paired and adjacent frames to align and fuse features both
temporally and spatially at the feature level. The outperformed experimental
results obtained in different zoom scenarios demonstrate the superiority of
integrating real-world video dataset and STCL into existing SR models for zoom
quality enhancement, and reveal that the proposed method can serve as an
advanced and viable tool for video zoom.
Related papers
- EvLight++: Low-Light Video Enhancement with an Event Camera: A Large-Scale Real-World Dataset, Novel Method, and More [7.974102031202597]
EvLight++ is a novel event-guided low-light video enhancement approach designed for robust performance in real-world scenarios.
EvLight++ significantly outperforms both single image- and video-based methods by 1.37 dB and 3.71 dB, respectively.
arXiv Detail & Related papers (2024-08-29T04:30:31Z) - BVI-RLV: A Fully Registered Dataset and Benchmarks for Low-Light Video Enhancement [56.97766265018334]
This paper introduces a low-light video dataset, consisting of 40 scenes with various motion scenarios under two distinct low-lighting conditions.
We provide fully registered ground truth data captured in normal light using a programmable motorized dolly and refine it via an image-based approach for pixel-wise frame alignment across different light levels.
Our experimental results demonstrate the significance of fully registered video pairs for low-light video enhancement (LLVE) and the comprehensive evaluation shows that the models trained with our dataset outperform those trained with the existing datasets.
arXiv Detail & Related papers (2024-07-03T22:41:49Z) - RAP: Efficient Text-Video Retrieval with Sparse-and-Correlated Adapter [77.0205013713008]
Text-Video Retrieval (TVR) aims to align relevant video content with natural language queries.
To date, most state-of-the-art TVR methods learn image-to-video transfer learning based on large-scale pre-trained vision models.
We propose a sparse-andcorrelated AdaPter (RAP) to fine-tune the pre-trained model with a few parameterized layers.
arXiv Detail & Related papers (2024-05-29T19:23:53Z) - Inflation with Diffusion: Efficient Temporal Adaptation for
Text-to-Video Super-Resolution [19.748048455806305]
We propose an efficient diffusion-based text-to-video super-resolution (SR) tuning approach.
We investigate different tuning approaches based on our inflated architecture and report trade-offs between computational costs and super-resolution quality.
arXiv Detail & Related papers (2024-01-18T22:25:16Z) - Continuous Space-Time Video Super-Resolution Utilizing Long-Range
Temporal Information [48.20843501171717]
We propose a continuous ST-VSR (CSTVSR) method that can convert the given video to any frame rate and spatial resolution.
We show that the proposed algorithm has good flexibility and achieves better performance on various datasets.
arXiv Detail & Related papers (2023-02-26T08:02:39Z) - HSTR-Net: High Spatio-Temporal Resolution Video Generation For Wide Area
Surveillance [4.125187280299246]
This paper presents the usage of multiple video feeds for the generation of HSTR video.
The main purpose is to create an HSTR video from the fusion of HSLF and LSHF videos.
arXiv Detail & Related papers (2022-04-09T09:23:58Z) - STDAN: Deformable Attention Network for Space-Time Video
Super-Resolution [39.18399652834573]
We propose a deformable attention network called STDAN for STVSR.
First, we devise a long-short term feature (LSTFI) module, which is capable of abundant content from more neighboring input frames.
Second, we put forward a spatial-temporal deformable feature aggregation (STDFA) module, in which spatial and temporal contexts are adaptively captured and aggregated.
arXiv Detail & Related papers (2022-03-14T03:40:35Z) - ARVo: Learning All-Range Volumetric Correspondence for Video Deblurring [92.40655035360729]
Video deblurring models exploit consecutive frames to remove blurs from camera shakes and object motions.
We propose a novel implicit method to learn spatial correspondence among blurry frames in the feature space.
Our proposed method is evaluated on the widely-adopted DVD dataset, along with a newly collected High-Frame-Rate (1000 fps) dataset for Video Deblurring.
arXiv Detail & Related papers (2021-03-07T04:33:13Z) - Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video
Super-Resolution [95.26202278535543]
A simple solution is to split it into two sub-tasks: video frame (VFI) and video super-resolution (VSR)
temporalsynthesis and spatial super-resolution are intra-related in this task.
We propose a one-stage space-time video super-resolution framework, which directly synthesizes an HR slow-motion video from an LFR, LR video.
arXiv Detail & Related papers (2020-02-26T16:59:48Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.