Stylized Text-to-Motion Generation via Hypernetwork-Driven Low-Rank Adaptation

• URL: http://arxiv.org/abs/2605.13333v1
• Date: Wed, 13 May 2026 10:51:54 GMT
• Title: Stylized Text-to-Motion Generation via Hypernetwork-Driven Low-Rank Adaptation
• Abstract summary: Text-driven motion diffusion models are capable of generating realistic human motions, but text alone often struggles to express fine-level nuances of motion known as style.<n>Recent approaches have tackled this challenge by attaching a style injection mechanism to a pretrained text-driven diffusion model.<n>We propose a lightweight style conditioning framework that dynamically modulates a pretrained diffusion model through hypernetwork-generated LoRA parameters.<n> Experiments on the HumanML3D and 100STYLE datasets show state-of-the-art stylization results, while achieving improved stylization for unseen styles.
• Score: 15.45127792716575
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-driven motion diffusion models are capable of generating realistic human motions, but text alone often struggles to express fine-level nuances of motion, commonly referred to as style. Recent approaches have tackled this challenge by attaching a style injection mechanism to a pretrained text-driven diffusion model. Existing stylization methods, however, either require style-specific fine-tuning of existing models or rely on heavy ControlNet-based architectures, limiting efficiency and generalization to unseen styles. We propose a lightweight style conditioning framework that dynamically modulates a pretrained diffusion model through hypernetwork-generated LoRA parameters. A style reference motion is encoded into a global style embedding, which is mapped by a hypernetwork to low-rank updates applied at each denoising step of the diffusion model. By structuring the style latent space with a supervised contrastive loss, our framework reliably captures diverse stylistic attributes, improves generalization to unseen styles, and supports optimization-based guidance without requiring predefined style categories. Experiments on the HumanML3D and 100STYLE datasets show state-of-the-art stylization results, while achieving improved stylization for unseen styles.

Related papers

• Flexible Motion Generation from Language and Style References [8.863552570164048]
  We introduce FlexMoGen, a framework for flexible human motion synthesis conditioned on both natural language descriptions and motion style references.<n>A style example clip supplements the text by conveying nuanced motion characteristics directly, enabling the model to preserve high-level intent.<n>Experiments show that FlexMoGen achieves the best balance between content fidelity and style reflection.
  arXiv  Detail & Related papers  (2026-09-07T22:28:44Z)
• ClusterStyle: Modeling Intra-Style Diversity with Prototypical Clustering for Stylized Motion Generation [33.75564496181951]
  We propose a clustering-based framework, ClusterStyle, to address intra-style diversity.<n>We leverage a set of prototypes to model diverse style patterns across motions belonging to the same style category.<n>Our approach outperforms existing state-of-the-art models in stylized motion generation and motion style transfer.
  arXiv  Detail & Related papers  (2025-12-02T06:24:14Z)
• Dance Like a Chicken: Low-Rank Stylization for Human Motion Diffusion [28.94750481325469]
  We introduce LoRA-MDM, a framework for motion stylization that generalizes to complex actions while maintaining editability.<n>Our key insight is that adapting the generative prior to include the style, while preserving its overall distribution, is more effective than modifying each individual motion during generation.<n>LoRA-MDM learns to adapt the prior to include the reference style using only a few samples.
  arXiv  Detail & Related papers  (2025-03-25T11:23:34Z)
• SMooDi: Stylized Motion Diffusion Model [46.293854851116215]
  We introduce a novel Stylized Motion Diffusion model, dubbed SMooDi, to generate stylized motion driven by content texts and style sequences. Our proposed framework outperforms existing methods in stylized motion generation.
  arXiv  Detail & Related papers  (2024-07-17T17:59:42Z)
• ArtWeaver: Advanced Dynamic Style Integration via Diffusion Model [73.95608242322949]
  Stylized Text-to-Image Generation (STIG) aims to generate images from text prompts and style reference images. We present ArtWeaver, a novel framework that leverages pretrained Stable Diffusion to address challenges such as misinterpreted styles and inconsistent semantics.
  arXiv  Detail & Related papers  (2024-05-24T07:19:40Z)
• HiCAST: Highly Customized Arbitrary Style Transfer with Adapter Enhanced Diffusion Models [84.12784265734238]
  The goal of Arbitrary Style Transfer (AST) is injecting the artistic features of a style reference into a given image/video. We propose HiCAST, which is capable of explicitly customizing the stylization results according to various source of semantic clues. A novel learning objective is leveraged for video diffusion model training, which significantly improve cross-frame temporal consistency.
  arXiv  Detail & Related papers  (2024-01-11T12:26:23Z)
• ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer [57.6482608202409]
  Textual style transfer is the task of transforming stylistic properties of text while preserving meaning. We introduce a novel diffusion-based framework for general-purpose style transfer that can be flexibly adapted to arbitrary target styles. We validate the method on the Enron Email Corpus, with both human and automatic evaluations, and find that it outperforms strong baselines on formality, sentiment, and even authorship style transfer.
  arXiv  Detail & Related papers  (2023-08-29T17:36:02Z)
• MODIFY: Model-driven Face Stylization without Style Images [77.24793103549158]
  Existing face stylization methods always acquire the presence of the target (style) domain during the translation process. We propose a new method called MODel-drIven Face stYlization (MODIFY), which relies on the generative model to bypass the dependence of the target images. Experimental results on several different datasets validate the effectiveness of MODIFY for unsupervised face stylization.
  arXiv  Detail & Related papers  (2023-03-17T08:35:17Z)
• Learning Graph Neural Networks for Image Style Transfer [131.73237185888215]
  State-of-the-art parametric and non-parametric style transfer approaches are prone to either distorted local style patterns due to global statistics alignment, or unpleasing artifacts resulting from patch mismatching. In this paper, we study a novel semi-parametric neural style transfer framework that alleviates the deficiency of both parametric and non-parametric stylization.
  arXiv  Detail & Related papers  (2022-07-24T07:41:31Z)
• Adversarial Style Augmentation for Domain Generalized Urban-Scene Segmentation [120.96012935286913]
  We propose a novel adversarial style augmentation approach, which can generate hard stylized images during training. Experiments on two synthetic-to-real semantic segmentation benchmarks demonstrate that AdvStyle can significantly improve the model performance on unseen real domains.
  arXiv  Detail & Related papers  (2022-07-11T14:01:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.