Towards Highly-Constrained Human Motion Generation with Retrieval-Guided Diffusion Noise Optimization

• URL: http://arxiv.org/abs/2605.08054v1
• Date: Fri, 08 May 2026 17:43:29 GMT
• Title: Towards Highly-Constrained Human Motion Generation with Retrieval-Guided Diffusion Noise Optimization
• Abstract summary: We present a training-free diffusion noise optimization framework for generating motion that satisfies customized zero-temporal goal functions.<n>This framework is used to enable applications such as controllable character animation and behavior synthesis for virtual agents.
• Score: 32.04001045559129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Generating human motion that satisfies customized zero-shot goal functions, enabling applications such as controllable character animation and behavior synthesis for virtual agents, is a critical capability. While current approaches handle many unseen constraints, they fail on tasks with very challenging spatiotemporal restrictions, such as severe spatial obstacles or specified numbers of walking steps. To equip motion generators for these highly constrained tasks, we present a retrieval-guided method built on the training-free diffusion noise optimization framework. The key idea is to search within large motion datasets for guidance that can potentially satisfy difficult constraints. We introduce relational task parsing to group target constraints and identify the difficult ones to be handled by retrieved reference. A better initialization for diffusion noise is then obtained via a reward-guided mask that combines random noise with retrieved noise. By optimizing diffusion noise from this improved initialization, we successfully solve highly constrained generation tasks. By leveraging LLM for relational task parsing, the whole framework is further enabled to automatically reason for what to retrieve, improving the intelligence of moving agents under a training-free optimization scheme.

Related papers

• Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation [32.824608968455294]
  Diffusion-based text-to-motion models synthesize realistic human motions but often exhibit semantic drift from the input text.<n>We propose WInning Noise Retrieval and Optimization (WINRO), a framework that improves text-motion alignment by selecting and refining such tickets before diffusion sampling.<n>WINRO consistently improves text-motion fidelity across different base models, MDM and MotionLCM, on HumanML3D without retraining.
  arXiv  Detail & Related papers  (2026-07-07T22:32:10Z)
• DiRecT: Safe Diffusion-Based Planning via Receding-Horizon Denoising [4.249024052507976]
  We introduce DiRecT, a training-free algorithm for constrained sampling from diffusion models via optimal control (SOC)<n>Inspired by model predictive control, we derive a receding-horizon surrogate for the otherwise intractable constrained SOC formulation.<n>Experiments demonstrate that DiRecT substantially improves deployment safety and task performance over existing diffusion-based planning baselines.
  arXiv  Detail & Related papers  (2026-06-13T15:41:26Z)
• ANCHOR: Agentic Noise Creation Framework for Human Simulation and Denoising Recommendation [8.940102827747355]
  We present ANCHOR, an agent-based framework inspired by recent LLM-as-User research.<n>ANCHOR simulates user behaviors to generate realistic noise labels and enables supervised denoising through two stages: noise creation and noise recognition.
  arXiv  Detail & Related papers  (2026-06-04T02:46:31Z)
• Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments [68.02342920827135]
  We propose NoisyAgent, an agentic training framework that explicitly incorporates environmental imperfections into the agent learning process.<n>We identify two major sources of interaction noise in real-world scenarios: user noise, which captures ambiguity and variability in user interaction, and tool noise, which reflects failures and anomalies in tool execution.<n>Extensive experiments demonstrate that our approach consistently improves agent robustness under noisy and dynamic environments.
  arXiv  Detail & Related papers  (2026-05-26T16:02:00Z)
• CiMRAG: CiM-Aware Domain-Adaptive and Noise-Resilient Retrieval-Augmented Generation for Edge-Based LLMs [10.372308692379404]
  Retrieval-Augmented Generation (RAG) is a key method for personalization by retrieving relevant profile data and generating tailored responses.<n> Computing-in-Memory (CiM) architectures mitigate this bottleneck by eliminating data movement between memory and processing units via in-situ operations.<n>We propose Task-Oriented Noise-resilient Embedding Learning, a framework that improves noise and domain adaptability for RAG in noisy edge environments.
  arXiv  Detail & Related papers  (2026-01-27T20:30:59Z)
• Noise-Robust Tiny Object Localization with Flows [63.60972031108944]
  We propose a noise-robust localization framework leveraging normalizing flows for flexible error modeling and uncertainty-guided optimization.<n>Our method captures complex, non-Gaussian prediction distributions through flow-based error modeling, enabling robust learning under noisy supervision.<n>An uncertainty-aware gradient modulation mechanism further suppresses learning from high-uncertainty, noise-prone samples, mitigating overfitting while stabilizing training.
  arXiv  Detail & Related papers  (2026-01-02T09:16:55Z)
• Group-Theoretic Reinforcement Learning of Dynamical Decoupling Sequences [9.136190419305427]
  Dynamical decoupling seeks to mitigate phase decoherence in pulsebits by applying a sequence of effectively instantaneous electromagnetic pulses.<n>We demonstrate that our agent can learn sequences that dephas without requiring explicit knowledge of the underlying noise spectrum.
  arXiv  Detail & Related papers  (2025-12-15T20:48:08Z)
• Real-Time Iteration Scheme for Diffusion Policy [23.124189676943757]
  We introduce a novel approach inspired by the Real-Time Iteration (RTI) Scheme to accelerate inference.<n>We propose a scaling-based method to effectively handle discrete actions, such as grasping, in robotic manipulation.<n>The proposed scheme significantly reduces runtime computational costs without the need for distillation or policy redesign.
  arXiv  Detail & Related papers  (2025-08-07T13:49:00Z)
• One-step Noisy Label Mitigation [86.57572253460125]
  Mitigating the detrimental effects of noisy labels on the training process has become increasingly critical. We propose One-step Anti-Noise (OSA), a model-agnostic noisy label mitigation paradigm. We empirically demonstrate the superiority of OSA, highlighting its enhanced training robustness, improved task transferability, ease of deployment, and reduced computational costs.
  arXiv  Detail & Related papers  (2024-10-02T18:42:56Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• DeNoising-MOT: Towards Multiple Object Tracking with Severe Occlusions [52.63323657077447]
  We propose DNMOT, an end-to-end trainable DeNoising Transformer for multiple object tracking. Specifically, we augment the trajectory with noises during training and make our model learn the denoising process in an encoder-decoder architecture. We conduct extensive experiments on the MOT17, MOT20, and DanceTrack datasets, and the experimental results show that our method outperforms previous state-of-the-art methods by a clear margin.
  arXiv  Detail & Related papers  (2023-09-09T04:40:01Z)
• Improve Noise Tolerance of Robust Loss via Noise-Awareness [60.34670515595074]
  We propose a meta-learning method which is capable of adaptively learning a hyper parameter prediction function, called Noise-Aware-Robust-Loss-Adjuster (NARL-Adjuster for brevity) Four SOTA robust loss functions are attempted to be integrated with our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its noise tolerance and performance.
  arXiv  Detail & Related papers  (2023-01-18T04:54:58Z)
• Robust Semantic Communications with Masked VQ-VAE Enabled Codebook [56.63571713657059]
  We propose a framework for the robust end-to-end semantic communication systems to combat the semantic noise. To combat the semantic noise, the adversarial training with weight is developed to incorporate the samples with semantic noise in the training dataset. We develop a feature importance module (FIM) to suppress the noise-related and task-unrelated features.
  arXiv  Detail & Related papers  (2022-06-08T16:58:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.