Correcting Face Distortion in Wide-Angle Videos
- URL: http://arxiv.org/abs/2111.09950v1
- Date: Thu, 18 Nov 2021 21:28:17 GMT
- Title: Correcting Face Distortion in Wide-Angle Videos
- Authors: Wei-Sheng Lai, YiChang Shih, Chia-Kai Liang, Ming-Hsuan Yang
- Abstract summary: We present a video warping algorithm to correct these distortions.
Our key idea is to apply stereographic projection locally on the facial regions.
For performance evaluation, we develop a wide-angle video dataset with a wide range of focal lengths.
- Score: 85.88898349347149
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Video blogs and selfies are popular social media formats, which are often
captured by wide-angle cameras to show human subjects and expanded background.
Unfortunately, due to perspective projection, faces near corners and edges
exhibit apparent distortions that stretch and squish the facial features,
resulting in poor video quality. In this work, we present a video warping
algorithm to correct these distortions. Our key idea is to apply stereographic
projection locally on the facial regions. We formulate a mesh warp problem
using spatial-temporal energy minimization and minimize background deformation
using a line-preservation term to maintain the straight edges in the
background. To address temporal coherency, we constrain the temporal smoothness
on the warping meshes and facial trajectories through the latent variables. For
performance evaluation, we develop a wide-angle video dataset with a wide range
of focal lengths. The user study shows that 83.9% of users prefer our algorithm
over other alternatives based on perspective projection.
Related papers
- Kalman-Inspired Feature Propagation for Video Face Super-Resolution [78.84881180336744]
We introduce a novel framework to maintain a stable face prior to time.
The Kalman filtering principles offer our method a recurrent ability to use the information from previously restored frames to guide and regulate the restoration process of the current frame.
Experiments demonstrate the effectiveness of our method in capturing facial details consistently across video frames.
arXiv Detail & Related papers (2024-08-09T17:57:12Z) - Controllable One-Shot Face Video Synthesis With Semantic Aware Prior [10.968343822308812]
One-shot talking-head synthesis task aims to animate a source image to another pose and expression, which is dictated by a driving frame.
Recent methods rely on warping the appearance feature extracted from the source, by using motion fields estimated from the sparse keypoints, that are learned in an unsupervised manner.
We propose a novel method that leverages the rich face prior information.
arXiv Detail & Related papers (2023-04-27T19:17:13Z) - Parallax-Tolerant Unsupervised Deep Image Stitching [57.76737888499145]
We propose UDIS++, a parallax-tolerant unsupervised deep image stitching technique.
First, we propose a robust and flexible warp to model the image registration from global homography to local thin-plate spline motion.
To further eliminate the parallax artifacts, we propose to composite the stitched image seamlessly by unsupervised learning for seam-driven composition masks.
arXiv Detail & Related papers (2023-02-16T10:40:55Z) - Video2StyleGAN: Encoding Video in Latent Space for Manipulation [63.03250800510085]
We propose a novel network to encode face videos into the latent space of StyleGAN for semantic face video manipulation.
Our approach can significantly outperform existing single image methods, while achieving real-time (66 fps) speed.
arXiv Detail & Related papers (2022-06-27T06:48:15Z) - Parametric Reshaping of Portraits in Videos [24.428095383264456]
We present a robust and easy-to-use parametric method to reshape the portrait in a video to produce smooth retouched results.
Given an input portrait video, our method consists of two main stages: stabilized face reconstruction, and continuous video reshaping.
In the second stage, we first reshape the reconstructed 3D face using a parametric reshaping model reflecting the weight change of the face, and then utilize the reshaped 3D face to guide the warping of video frames.
arXiv Detail & Related papers (2022-05-05T09:55:16Z) - Unsupervised Action Localization Crop in Video Retargeting for 3D
ConvNets [0.0]
3D CNNs require a square-shaped video whose dimension is the original one. Random or center-cropping techniques in use may leave out the video's subject altogether.
We propose an unsupervised video cropping approach by shaping this as the spatial and video-to-video synthesis problem.
The synthesized video maintains 1:1 aspect ratio, smaller in size and is targeted at the video-subject throughout the duration.
arXiv Detail & Related papers (2021-11-14T19:27:13Z) - Consistent Depth of Moving Objects in Video [52.72092264848864]
We present a method to estimate depth of a dynamic scene, containing arbitrary moving objects, from an ordinary video captured with a moving camera.
We formulate this objective in a new test-time training framework where a depth-prediction CNN is trained in tandem with an auxiliary scene-flow prediction over the entire input video.
We demonstrate accurate and temporally coherent results on a variety of challenging videos containing diverse moving objects (pets, people, cars) as well as camera motion.
arXiv Detail & Related papers (2021-08-02T20:53:18Z) - Practical Wide-Angle Portraits Correction with Deep Structured Models [17.62752136436382]
This paper introduces the first deep learning based approach to remove perspective distortions from photos.
Given a wide-angle portrait as input, we build a cascaded network consisting of a LineNet, a ShapeNet, and a transition module.
For the quantitative evaluation, we introduce two novel metrics, line consistency and face congruence.
arXiv Detail & Related papers (2021-04-26T10:47:35Z) - Video Semantic Segmentation with Distortion-Aware Feature Correction [32.00672651803015]
Per-frame image segmentation is generally unacceptable in practice due to high computation cost.
We propose distortion-aware feature correction to alleviate the issue.
Our proposed method can significantly boost the accuracy of video semantic segmentation at a low price.
arXiv Detail & Related papers (2020-06-18T09:30:00Z) - Consistent Video Depth Estimation [57.712779457632024]
We present an algorithm for reconstructing dense, geometrically consistent depth for all pixels in a monocular video.
We leverage a conventional structure-from-motion reconstruction to establish geometric constraints on pixels in the video.
Our algorithm is able to handle challenging hand-held captured input videos with a moderate degree of dynamic motion.
arXiv Detail & Related papers (2020-04-30T17:59:26Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.