Animation from Blur: Multi-modal Blur Decomposition with Motion Guidance
- URL: http://arxiv.org/abs/2207.10123v1
- Date: Wed, 20 Jul 2022 18:05:53 GMT
- Title: Animation from Blur: Multi-modal Blur Decomposition with Motion Guidance
- Authors: Zhihang Zhong, Xiao Sun, Zhirong Wu, Yinqiang Zheng, Stephen Lin and
Imari Sato
- Abstract summary: We study the challenging problem of recovering detailed motion from a single motion-red image.
Existing solutions to this problem estimate a single image sequence without considering the motion ambiguity for each region.
In this paper, we explicitly account for such motion ambiguity, allowing us to generate multiple plausible solutions all in sharp detail.
- Score: 83.25826307000717
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We study the challenging problem of recovering detailed motion from a single
motion-blurred image. Existing solutions to this problem estimate a single
image sequence without considering the motion ambiguity for each region.
Therefore, the results tend to converge to the mean of the multi-modal
possibilities. In this paper, we explicitly account for such motion ambiguity,
allowing us to generate multiple plausible solutions all in sharp detail. The
key idea is to introduce a motion guidance representation, which is a compact
quantization of 2D optical flow with only four discrete motion directions.
Conditioned on the motion guidance, the blur decomposition is led to a
specific, unambiguous solution by using a novel two-stage decomposition
network. We propose a unified framework for blur decomposition, which supports
various interfaces for generating our motion guidance, including human input,
motion information from adjacent video frames, and learning from a video
dataset. Extensive experiments on synthesized datasets and real-world data show
that the proposed framework is qualitatively and quantitatively superior to
previous methods, and also offers the merit of producing physically plausible
and diverse solutions. Code is available at
https://github.com/zzh-tech/Animation-from-Blur.
Related papers
- Shape of Motion: 4D Reconstruction from a Single Video [51.04575075620677]
We introduce a method capable of reconstructing generic dynamic scenes, featuring explicit, full-sequence-long 3D motion.
We exploit the low-dimensional structure of 3D motion by representing scene motion with a compact set of SE3 motion bases.
Our method achieves state-of-the-art performance for both long-range 3D/2D motion estimation and novel view synthesis on dynamic scenes.
arXiv Detail & Related papers (2024-07-18T17:59:08Z) - DiverseMotion: Towards Diverse Human Motion Generation via Discrete
Diffusion [70.33381660741861]
We present DiverseMotion, a new approach for synthesizing high-quality human motions conditioned on textual descriptions.
We show that our DiverseMotion achieves the state-of-the-art motion quality and competitive motion diversity.
arXiv Detail & Related papers (2023-09-04T05:43:48Z) - A Constrained Deformable Convolutional Network for Efficient Single
Image Dynamic Scene Blind Deblurring with Spatially-Variant Motion Blur
Kernels Estimation [12.744989551644744]
We propose a novel constrained deformable convolutional network (CDCN) for efficient single image dynamic scene blind deblurring.
CDCN simultaneously achieves accurate spatially-variant motion blur kernels estimation and the high-quality image restoration.
arXiv Detail & Related papers (2022-08-23T03:28:21Z) - Single Stage Virtual Try-on via Deformable Attention Flows [51.70606454288168]
Virtual try-on aims to generate a photo-realistic fitting result given an in-shop garment and a reference person image.
We develop a novel Deformable Attention Flow (DAFlow) which applies the deformable attention scheme to multi-flow estimation.
Our proposed method achieves state-of-the-art performance both qualitatively and quantitatively.
arXiv Detail & Related papers (2022-07-19T10:01:31Z) - Exploring Motion and Appearance Information for Temporal Sentence
Grounding [52.01687915910648]
We propose a Motion-Appearance Reasoning Network (MARN) to solve temporal sentence grounding.
We develop separate motion and appearance branches to learn motion-guided and appearance-guided object relations.
Our proposed MARN significantly outperforms previous state-of-the-art methods by a large margin.
arXiv Detail & Related papers (2022-01-03T02:44:18Z) - Image Animation with Keypoint Mask [0.0]
Motion transfer is the task of synthesizing future video frames of a single source image according to the motion from a given driving video.
In this work, we extract the structure from a keypoint heatmap, without an explicit motion representation.
Then, the structures from the image and the video are extracted to warp the image according to the video, by a deep generator.
arXiv Detail & Related papers (2021-12-20T11:35:06Z) - Learning to Segment Rigid Motions from Two Frames [72.14906744113125]
We propose a modular network, motivated by a geometric analysis of what independent object motions can be recovered from an egomotion field.
It takes two consecutive frames as input and predicts segmentation masks for the background and multiple rigidly moving objects, which are then parameterized by 3D rigid transformations.
Our method achieves state-of-the-art performance for rigid motion segmentation on KITTI and Sintel.
arXiv Detail & Related papers (2021-01-11T04:20:30Z) - Convolutional Autoencoders for Human Motion Infilling [37.16099544563645]
Motion infilling aims to complete the missing gap in between, such that the filled in poses plausibly forecast the start sequence and naturally transition into the end sequence.
We show that a single model can be used to create natural transitions between different types of activities.
Our method is not only able to fill in entire missing frames, but it can also be used to complete gaps where partial poses are available.
arXiv Detail & Related papers (2020-10-22T08:45:38Z) - Multi-object Monocular SLAM for Dynamic Environments [12.537311048732017]
The term multibody, implies that we track the motion of the camera, as well as that of other dynamic participants in the scene.
Existing approaches solve restricted variants of the problem, but the solutions suffer relative scale ambiguity.
We propose a multi pose-graph optimization formulation, to resolve the relative and absolute scale factor ambiguities involved.
arXiv Detail & Related papers (2020-02-10T03:49:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.