HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback

• URL: http://arxiv.org/abs/2312.12227v1
• Date: Tue, 19 Dec 2023 15:13:08 GMT
• Title: HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback
• Authors: Gaoge Han, Shaoli Huang, Mingming Gong, Jinglei Tang
• Abstract summary: HuTuMotion is an innovative approach for generating natural human motions that navigates latent motion diffusion models by leveraging few-shot human feedback. Our findings reveal that utilizing few-shot feedback can yield performance levels on par with those attained through extensive human feedback.
• Score: 46.744192144648764
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce HuTuMotion, an innovative approach for generating natural human motions that navigates latent motion diffusion models by leveraging few-shot human feedback. Unlike existing approaches that sample latent variables from a standard normal prior distribution, our method adapts the prior distribution to better suit the characteristics of the data, as indicated by human feedback, thus enhancing the quality of motion generation. Furthermore, our findings reveal that utilizing few-shot feedback can yield performance levels on par with those attained through extensive human feedback. This discovery emphasizes the potential and efficiency of incorporating few-shot human-guided optimization within latent diffusion models for personalized and style-aware human motion generation applications. The experimental results show the significantly superior performance of our method over existing state-of-the-art approaches.

Related papers

• Rethinking Diffusion for Text-Driven Human Motion Generation [8.153961351540834]
  Since 2023, Vector Quantization (VQ)-based discrete generation methods have dominated human motion generation. In this work, we investigate why current VQ-based methods perform well and explore the limitations of existing diffusion-based methods. Our approach introduces a human motion diffusion model enabled to perform bidirectional masked autoregression.
  arXiv  Detail & Related papers  (2024-11-25T16:59:42Z)
• MotionRL: Align Text-to-Motion Generation to Human Preferences with Multi-Reward Reinforcement Learning [99.09906827676748]
  We introduce MotionRL, the first approach to utilize Multi-Reward Reinforcement Learning (RL) for optimizing text-to-motion generation tasks. Our novel approach uses reinforcement learning to fine-tune the motion generator based on human preferences prior knowledge of the human perception model. In addition, MotionRL introduces a novel multi-objective optimization strategy to approximate optimality between text adherence, motion quality, and human preferences.
  arXiv  Detail & Related papers  (2024-10-09T03:27:14Z)
• Bayesian-Optimized One-Step Diffusion Model with Knowledge Distillation for Real-Time 3D Human Motion Prediction [2.402745776249116]
  We propose training a one-step multi-layer perceptron-based (MLP-based) diffusion model for motion prediction using knowledge distillation and Bayesian optimization. Our model can significantly improve the inference speed, achieving real-time prediction without noticeable degradation in performance.
  arXiv  Detail & Related papers  (2024-09-19T04:36:40Z)
• MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds [20.83684434910106]
  We present MoManifold, a novel human motion prior, which models plausible human motion in continuous high-dimensional motion space. Specifically, we propose novel decoupled joint acceleration to model human dynamics from existing limited motion data. Extensive experiments demonstrate that MoManifold outperforms existing SOTAs as a prior in several downstream tasks.
  arXiv  Detail & Related papers  (2024-09-01T15:00:16Z)
• Aligning Human Motion Generation with Human Perceptions [51.831338643012444]
  We propose a data-driven approach to bridge the gap by introducing a large-scale human perceptual evaluation dataset, MotionPercept, and a human motion critic model, MotionCritic. Our critic model offers a more accurate metric for assessing motion quality and could be readily integrated into the motion generation pipeline.
  arXiv  Detail & Related papers  (2024-07-02T14:01:59Z)
• Confronting Reward Overoptimization for Diffusion Models: A Perspective of Inductive and Primacy Biases [76.9127853906115]
  Bridging the gap between diffusion models and human preferences is crucial for their integration into practical generative. We propose Temporal Diffusion Policy Optimization with critic active neuron Reset (TDPO-R), a policy gradient algorithm that exploits the temporal inductive bias of diffusion models. Empirical results demonstrate the superior efficacy of our methods in mitigating reward overoptimization.
  arXiv  Detail & Related papers  (2024-02-13T15:55:41Z)
• Motion Flow Matching for Human Motion Synthesis and Editing [75.13665467944314]
  We propose emphMotion Flow Matching, a novel generative model for human motion generation featuring efficient sampling and effectiveness in motion editing applications. Our method reduces the sampling complexity from thousand steps in previous diffusion models to just ten steps, while achieving comparable performance in text-to-motion and action-to-motion generation benchmarks.
  arXiv  Detail & Related papers  (2023-12-14T12:57:35Z)
• FABRIC: Personalizing Diffusion Models with Iterative Feedback [0.0]
  In an era where visual content generation is increasingly driven by machine learning, the integration of human feedback into generative models presents significant opportunities for enhancing user experience and output quality. We propose FABRIC, a training-free approach applicable to a wide range of popular diffusion models, which exploits the self-attention layer present in the most widely used architectures to condition the diffusion process on a set of feedback images. We show that generation results improve over multiple rounds of iterative feedback through exhaustive analysis, implicitly optimizing arbitrary user preferences.
  arXiv  Detail & Related papers  (2023-07-19T17:39:39Z)
• Executing your Commands via Motion Diffusion in Latent Space [51.64652463205012]
  We propose a Motion Latent-based Diffusion model (MLD) to produce vivid motion sequences conforming to the given conditional inputs. Our MLD achieves significant improvements over the state-of-the-art methods among extensive human motion generation tasks.
  arXiv  Detail & Related papers  (2022-12-08T03:07:00Z)
• Multi-grained Trajectory Graph Convolutional Networks for Habit-unrelated Human Motion Prediction [4.070072825448614]
  A multigrained graph convolutional networks based lightweight framework is proposed for habit-unrelated human motion prediction. A new motion generation method is proposed to generate the motion with left-handedness, to better model the motion with less bias to the human habit. Experimental results on challenging datasets, including Humantemporal3.6M and CMU Mocap, show that the proposed model outperforms state-of-the-art with less than 0.12 times parameters.
  arXiv  Detail & Related papers  (2020-12-23T09:41:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.