GenFusion: Feed-forward Human Performance Capture via Progressive Canonical Space Updates
- URL: http://arxiv.org/abs/2603.28997v1
- Date: Mon, 30 Mar 2026 20:55:00 GMT
- Title: GenFusion: Feed-forward Human Performance Capture via Progressive Canonical Space Updates
- Abstract summary: We present a feed-forward human performance capture method that renders novel views of a performer from a monocular RGB stream.<n>We take advantage of the fact that more body parts become observable by maintaining a canonical space that is progressively updated with each incoming frame.<n>This canonical space accumulates appearance information over time and serves as a context bank when direct observations are missing in the current live frame.
- Score: 17.25826599799299
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a feed-forward human performance capture method that renders novel views of a performer from a monocular RGB stream. A key challenge in this setting is the lack of sufficient observations, especially for unseen regions. Assuming the subject moves continuously over time, we take advantage of the fact that more body parts become observable by maintaining a canonical space that is progressively updated with each incoming frame. This canonical space accumulates appearance information over time and serves as a context bank when direct observations are missing in the current live frame. To effectively utilize this context while respecting the deformation of the live state, we formulate the rendering process as probabilistic regression. This resolves conflicts between past and current observations, producing sharper reconstructions than deterministic regression approaches. Furthermore, it enables plausible synthesis even in regions with no prior observations. Experiments on in-domain (4D-Dress) and out-of-distribution (MVHumanNet) datasets demonstrate the effectiveness of our approach.
Related papers
- L2A: Learning to Accumulate Pose History for Accurate 3D Human Pose Estimation [15.020290085816583]
Existing 2D-3D lifting human pose estimation methods have achieved strong performance.<n>But the utilization of historical pose representations across network depth was overlooked.<n>We propose a history-aware framework that enables effective network cross-layer history feature utilization.
arXiv Detail & Related papers (2026-05-09T08:48:20Z) - VFMF: World Modeling by Forecasting Vision Foundation Model Features [67.09340259579761]
We introduce a generative forecaster that performs autoregressive flow matching in vision foundation models feature space.<n>We show that this latent information more effectively than previously used PCA-based alternatives, both for forecasting and other applications.<n>With matched architecture and compute, our method produces sharper and more accurate predictions than regression across all modalities.
arXiv Detail & Related papers (2025-12-12T02:10:05Z) - Kinaema: a recurrent sequence model for memory and pose in motion [25.677034515160567]
We introduce a new model, Kinaema, and agent, capable of integrating a stream of visual observations while moving in a potentially large scene.<n>Our model does not explicitly store an observation history, therefore does not have hard constraints on context length.<n>We show that our large-capacity recurrent model maintains a useful representation of the scene, navigates to goals observed before the actual episode start, and is computationally efficient.
arXiv Detail & Related papers (2025-10-23T06:34:53Z) - StateSpaceDiffuser: Bringing Long Context to Diffusion World Models [52.92249035412797]
We introduce StateSpaceDiffuser, where a diffusion model is enabled to perform long-context tasks by integrating features from a state-space model.<n>This design restores long-term memory while preserving the high-fidelity synthesis of diffusion models.
arXiv Detail & Related papers (2025-05-28T11:27:54Z) - A Study of Posterior Stability for Time-Series Latent Diffusion [59.41969496514184]
We first show that posterior collapse will reduce latent diffusion to a variational autoencoder (VAE), making it less expressive.
We then introduce a principled method: dependency measure, that quantifies the sensitivity of a recurrent decoder to input variables.
Building on our theoretical and empirical studies, we introduce a new framework that extends latent diffusion and has a stable posterior.
arXiv Detail & Related papers (2024-05-22T21:54:12Z) - Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMamba [11.400397931501338]
Event cameras draw inspiration from biological systems, boasting low latency and high dynamic range while consuming minimal power.<n>Most current approach to processing Event Cloud often involves converting it into frame-based representations.<n>We propose EventMamba, an efficient and effective framework based on Point Cloud representation.
arXiv Detail & Related papers (2024-05-09T21:47:46Z) - STARFlow: Spatial Temporal Feature Re-embedding with Attentive Learning for Real-world Scene Flow [5.476991379461233]
We propose global attentive flow embedding to match all-to-all point pairs in both Euclidean space.
We leverage novel domain adaptive losses to bridge the gap of motion inference from synthetic to real-world.
Our approach achieves state-of-the-art performance across various datasets, with particularly outstanding results on real-world LiDAR-scanned datasets.
arXiv Detail & Related papers (2024-03-11T04:56:10Z) - Streaming Factor Trajectory Learning for Temporal Tensor Decomposition [33.18423605559094]
We propose Streaming Factor Trajectory Learning for temporal tensor decomposition.
We use Gaussian processes (GPs) to model the trajectory of factors so as to flexibly estimate their temporal evolution.
We have shown the advantage of SFTL in both synthetic tasks and real-world applications.
arXiv Detail & Related papers (2023-10-25T21:58:52Z) - Reconstructing Spatiotemporal Data with C-VAEs [49.1574468325115]
Conditional continuous representation of moving regions is commonly used.
In this work, we explore the capabilities of Conditional Varitemporal Autoencoder (C-VAE) models to generate realistic representations of regions' evolution.
arXiv Detail & Related papers (2023-07-12T15:34:10Z) - Inverse Dynamics Pretraining Learns Good Representations for Multitask
Imitation [66.86987509942607]
We evaluate how such a paradigm should be done in imitation learning.
We consider a setting where the pretraining corpus consists of multitask demonstrations.
We argue that inverse dynamics modeling is well-suited to this setting.
arXiv Detail & Related papers (2023-05-26T14:40:46Z) - Multi-level Motion Attention for Human Motion Prediction [132.29963836262394]
We study the use of different types of attention, computed at joint, body part, and full pose levels.
Our experiments on Human3.6M, AMASS and 3DPW validate the benefits of our approach for both periodical and non-periodical actions.
arXiv Detail & Related papers (2021-06-17T08:08:11Z) - Crop Classification under Varying Cloud Cover with Neural Ordinary
Differential Equations [23.93148719731374]
State-of-the-art methods for crop classification rely on techniques that implicitly assume regular temporal spacing between observations.
We propose to use neural ordinary differential equations (NODEs) in combination with recurrent neural networks (RNNs) to classify crop types in irregularly spaced image sequences.
arXiv Detail & Related papers (2020-12-04T11:56:50Z) - Latent World Models For Intrinsically Motivated Exploration [140.21871701134626]
We present a self-supervised representation learning method for image-based observations.
We consider episodic and life-long uncertainties to guide the exploration of partially observable environments.
arXiv Detail & Related papers (2020-10-05T19:47:04Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.