HapticLDM: A Diffusion Model for Text-to-Vibrotactile Generation

• URL: http://arxiv.org/abs/2605.09971v1
• Date: Mon, 11 May 2026 04:26:51 GMT
• Title: HapticLDM: A Diffusion Model for Text-to-Vibrotactile Generation
• Abstract summary: HapticLDM is the first text-to-vibration generative model built upon Latent Diffusion Models (LDMs)<n>We introduce a text-processing strategy to curate high-quality data pairs for fine-grained dynamic modeling.<n>We conduct extensive evaluations, including A/B testing against the state-of-the-art baseline and a user study involving 30 participants.
• Score: 6.067124292824705
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Text-to-vibration generation converts natural language into haptic feedback, enabling vibration-effect designers to get scenarios-fitted vibrations more efficiently, which shows great potentials in application fields such as metaverse, games, and film to enrich the user experience in interactive scenarios. The core challenge in this field is how to generate accurate, consistent, and complete vibrations according to textual semantics. Very recent autoregressive (AR) approaches (e.g., HapticGen) exhibit limited capacity in fully capturing global dependencies, owing to the inherent sequential nature of their modeling and prevailing data constraints. In this paper, we proposed HapticLDM, the first text-to-vibration generative model built upon Latent Diffusion Models (LDMs). Firstly, with respect to the data, we introduced a text-processing strategy that emphasizes dynamic characteristics to curate high-quality data pairs for fine-grained dynamic modeling. Secondly, HapticLDM incorporates a global denoising mechanism that regulates coherent and stable variations in the temporal envelope. Furthermore, we conduct extensive evaluations, including A/B testing against the state-of-the-art baseline and a user study involving 30 participants. The results demonstrate that our model enhances realism and semantic alignment. Qualitative feedback further indicates that HapticLDM simplifies the haptic design workflow while generating diverse, subtle, and physically precise vibrations.

Related papers

• Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding [63.599342750566485]
  SPAR is a novel joint semantic-geometric encoding architecture that explicitly isolates transient dynamic noise prior to latent space aggregation.<n>Our end-to-end approach achieves exceptional novel view synthesis quality, yielding a PSNR of 22.15 dB and 23.33 dB given only 3 and 4 input views respectively.
  arXiv  Detail & Related papers  (2026-08-29T09:58:24Z)
• In-Context Model Predictive Generation: Open-Vocabulary Motion Synthesis from Language Models to Physics [109.99871326497333]
  In-Context Model Predictive Generation (ICMPG) is a framework that integrates language-model planning with inference-time physical feedback.<n>ICMPG reformulates motion synthesis as a Model Predictive Control (MPC)-like process with two modules.
  arXiv  Detail & Related papers  (2026-06-25T12:50:33Z)
• DC-Motion: Decoupling Semantics and Details via Discrete-Continuous Tokens for Human Motion Generation [11.57444637723047]
  We introduce DC-Motion, a factorized generative framework designed to explicitly decouple semantics and details via discrete-continuous tokens.<n>A masked AR model predicts the discrete structure from text, and a lightweight residual diffusion model recovers the continuous physical details.<n>On both HumanML3D and KIT-ML datasets, DC-Motion achieves state-of-the-art performance, delivering the best FID for motion realism and R-precision for text alignment.
  arXiv  Detail & Related papers  (2026-05-28T11:47:11Z)
• TextLDM: Language Modeling with Continuous Latent Diffusion [89.69255520673248]
  Diffusion Transformers (DiT) trained with flow matching in a VAE latent space have unified visual generation across images and videos.<n>We propose TextLDM, which transfers the visual latent diffusion recipe to text generation with minimal architectural modification.
  arXiv  Detail & Related papers  (2026-05-08T13:54:34Z)
• Continuous Latent Diffusion Language Model [48.974403879186916]
  Large language models have achieved remarkable success under the autoregressive paradigm.<n>Existing alternatives still struggle to jointly achieve generation efficiency, scalable representation learning, and effective global semantic modeling.<n>We propose Cola DLM, a hierarchical latent diffusion language model that frames text generation through hierarchical information decomposition.
  arXiv  Detail & Related papers  (2026-05-07T16:44:56Z)
• LaScA: Language-Conditioned Scalable Modelling of Affective Dynamics [3.3187704612685267]
  We propose a novel framework that uses Language Models (LMs) as semantic context conditioners over handcrafted affect descriptors.<n>Our approach begins with interpretable facial geometry and acoustic features derived from structured domain knowledge.<n>A pretrained LM processes these descriptions to generate semantic context embeddings that act as high-level priors over affective dynamics.
  arXiv  Detail & Related papers  (2026-04-08T15:18:39Z)
• Beyond Language Modeling: An Exploration of Multimodal Pretraining [125.34714978184638]
  We provide empirical clarity through controlled, from-scratch pretraining experiments.<n>We adopt the Transfusion framework, using next-token prediction for language and diffusion for vision.<n>We demonstrate that the MoE architecture harmonizes this scaling asymmetry by providing the high model capacity required by language.
  arXiv  Detail & Related papers  (2026-03-03T18:58:00Z)
• Aligning Agentic World Models via Knowledgeable Experience Learning [68.85843641222186]
  We introduce WorldMind, a framework that constructs a symbolic World Knowledge Repository by synthesizing environmental feedback.<n>WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.
  arXiv  Detail & Related papers  (2026-01-19T17:33:31Z)
• Exploring Representation-Aligned Latent Space for Better Generation [86.45670422239317]
  We introduce ReaLS, which integrates semantic priors to improve generation performance.<n>We show that fundamental DiT and SiT trained on ReaLS can achieve a 15% improvement in FID metric.<n>The enhanced semantic latent space enables more perceptual downstream tasks, such as segmentation and depth estimation.
  arXiv  Detail & Related papers  (2025-02-01T07:42:12Z)
• Steering Language Generation: Harnessing Contrastive Expert Guidance and Negative Prompting for Coherent and Diverse Synthetic Data Generation [0.0]
  Large Language Models (LLMs) hold immense potential to generate synthetic data of high quality and utility. We introduce contrastive expert guidance, where the difference between the logit distributions of fine-tuned and base language models is emphasised. We deem this dual-pronged approach to logit reshaping as STEER: Semantic Text Enhancement via Embedding Repositioning.
  arXiv  Detail & Related papers  (2023-08-15T08:49:14Z)
• Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation [117.3856882511919]
  We propose the Style-HAllucinated Dual consistEncy learning (SHADE) framework to handle domain shift. Our SHADE yields significant improvement and outperforms state-of-the-art methods by 5.07% and 8.35% on the average mIoU of three real-world datasets.
  arXiv  Detail & Related papers  (2022-04-06T02:49:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.