HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models
with Minimal Feedback
- URL: http://arxiv.org/abs/2312.12227v1
- Date: Tue, 19 Dec 2023 15:13:08 GMT
- Title: HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models
with Minimal Feedback
- Authors: Gaoge Han, Shaoli Huang, Mingming Gong, Jinglei Tang
- Abstract summary: HuTuMotion is an innovative approach for generating natural human motions that navigates latent motion diffusion models by leveraging few-shot human feedback.
Our findings reveal that utilizing few-shot feedback can yield performance levels on par with those attained through extensive human feedback.
- Score: 46.744192144648764
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: We introduce HuTuMotion, an innovative approach for generating natural human
motions that navigates latent motion diffusion models by leveraging few-shot
human feedback. Unlike existing approaches that sample latent variables from a
standard normal prior distribution, our method adapts the prior distribution to
better suit the characteristics of the data, as indicated by human feedback,
thus enhancing the quality of motion generation. Furthermore, our findings
reveal that utilizing few-shot feedback can yield performance levels on par
with those attained through extensive human feedback. This discovery emphasizes
the potential and efficiency of incorporating few-shot human-guided
optimization within latent diffusion models for personalized and style-aware
human motion generation applications. The experimental results show the
significantly superior performance of our method over existing state-of-the-art
approaches.
Related papers
- Aligning Human Motion Generation with Human Perceptions [51.831338643012444]
We propose a data-driven approach to bridge the gap by introducing a large-scale human perceptual evaluation dataset, MotionPercept, and a human motion critic model, MotionCritic.
Our critic model offers a more accurate metric for assessing motion quality and could be readily integrated into the motion generation pipeline.
arXiv Detail & Related papers (2024-07-02T14:01:59Z) - Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases [76.9127853906115]
Bridging the gap between diffusion models and human preferences is crucial for their integration into practical generative.
We propose Temporal Diffusion Policy Optimization with critic active neuron Reset (TDPO-R), a policy gradient algorithm that exploits the temporal inductive bias of diffusion models.
Empirical results demonstrate the superior efficacy of our methods in mitigating reward overoptimization.
arXiv Detail & Related papers (2024-02-13T15:55:41Z) - GazeMoDiff: Gaze-guided Diffusion Model for Stochastic Human Motion
Prediction [11.997928273335129]
Existing methods have synthesised body motion only from observed past motion.
We present GazeMoDiff, a novel gaze-guided denoising model to generate human motions.
Our work makes a first important step towards gaze-guided human motion prediction.
arXiv Detail & Related papers (2023-12-19T12:10:12Z) - Motion Flow Matching for Human Motion Synthesis and Editing [75.13665467944314]
We propose emphMotion Flow Matching, a novel generative model for human motion generation featuring efficient sampling and effectiveness in motion editing applications.
Our method reduces the sampling complexity from thousand steps in previous diffusion models to just ten steps, while achieving comparable performance in text-to-motion and action-to-motion generation benchmarks.
arXiv Detail & Related papers (2023-12-14T12:57:35Z) - TransFusion: A Practical and Effective Transformer-based Diffusion Model
for 3D Human Motion Prediction [1.8923948104852863]
We propose TransFusion, an innovative and practical diffusion-based model for 3D human motion prediction.
Our model leverages Transformer as the backbone with long skip connections between shallow and deep layers.
In contrast to prior diffusion-based models that utilize extra modules like cross-attention and adaptive layer normalization, we treat all inputs, including conditions, as tokens to create a more lightweight model.
arXiv Detail & Related papers (2023-07-30T01:52:07Z) - FABRIC: Personalizing Diffusion Models with Iterative Feedback [0.0]
In an era where visual content generation is increasingly driven by machine learning, the integration of human feedback into generative models presents significant opportunities for enhancing user experience and output quality.
We propose FABRIC, a training-free approach applicable to a wide range of popular diffusion models, which exploits the self-attention layer present in the most widely used architectures to condition the diffusion process on a set of feedback images.
We show that generation results improve over multiple rounds of iterative feedback through exhaustive analysis, implicitly optimizing arbitrary user preferences.
arXiv Detail & Related papers (2023-07-19T17:39:39Z) - Executing your Commands via Motion Diffusion in Latent Space [51.64652463205012]
We propose a Motion Latent-based Diffusion model (MLD) to produce vivid motion sequences conforming to the given conditional inputs.
Our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks.
arXiv Detail & Related papers (2022-12-08T03:07:00Z) - HuMoR: 3D Human Motion Model for Robust Pose Estimation [100.55369985297797]
HuMoR is a 3D Human Motion Model for Robust Estimation of temporal pose and shape.
We introduce a conditional variational autoencoder, which learns a distribution of the change in pose at each step of a motion sequence.
We demonstrate that our model generalizes to diverse motions and body shapes after training on a large motion capture dataset.
arXiv Detail & Related papers (2021-05-10T21:04:55Z) - Multi-grained Trajectory Graph Convolutional Networks for
Habit-unrelated Human Motion Prediction [4.070072825448614]
A multigrained graph convolutional networks based lightweight framework is proposed for habit-unrelated human motion prediction.
A new motion generation method is proposed to generate the motion with left-handedness, to better model the motion with less bias to the human habit.
Experimental results on challenging datasets, including Humantemporal3.6M and CMU Mocap, show that the proposed model outperforms state-of-the-art with less than 0.12 times parameters.
arXiv Detail & Related papers (2020-12-23T09:41:50Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.