HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation

• URL: http://arxiv.org/abs/2608.19574v1
• Date: Thu, 20 Aug 2026 02:28:25 GMT
• Title: HiTac-WAM: A Hierarchical Tactile World Action Model for Contact-Rich Robot Manipulation
• Abstract summary: HiTac-WAM is a hierarchical tactile world action model that forecasts a sequence of future tactile states for each candidate action chunk before execution.<n>For planning, HiTac-WAM ranks candidate action chunks using tactile forecasts and task-progress estimates.<n>For execution, the selected tactile forecast is retained as a reference.
• Score: 16.161933557281447
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: World action models jointly predict future visual observations and actions, whereas existing tactile-aware variants typically represent future touch as an image or latent stream without modeling the physical dependencies that organize tactile states hierarchically. We present HiTac-WAM, a hierarchical tactile world action model that forecasts a sequence of future tactile states for each candidate action chunk before execution. The forecast factorizes into contact state, a 3D deformation field, and slip risk, organized as a directed hierarchy in which each downstream stage is conditioned on stop-gradient signals from preceding stages. A directed attention mask allows tactile queries to attend to the video-action context of each candidate while preventing video and action queries from attending to tactile tokens. For planning, HiTac-WAM ranks candidate action chunks using tactile forecasts and task-progress estimates. For execution, the selected tactile forecast is retained as a reference; persistent discrepancies between predicted and observed tactile states trigger corrective replanning. HiTac-WAM achieves a mean contact F1 of 0.921; under matched training budgets, the directed hierarchy reduces 3D displacement L2 error by 17.6% relative to the deformation-only predictor and improves slip AUPRC by 60.4% relative to the slip-only predictor. Across chip grasping, blackboard erasing, and USB insertion, selection guided by the hierarchical forecasts increases the average real-robot success rate from 31.1% to 61.1%, while the full system attains 72.2%.

Related papers

• World Models for Embodied Intelligence: From Plausible to Controllable to Actionable [100.13060753774657]
  World models connect perception and decision-making in embodied intelligence by maintaining hidden state, anticipating consequences, comparing interventions, and adapting when execution departs from expectations.<n>This raises a central question: which predictive capabilities improve behavior?<n>We introduce three progressively stronger capability levels: Plausible models preserve task-relevant temporal, geometric, or physical structure; Controllable models additionally predict how interventions alter that structure; and Actionable models translate predictions into measurable gains in planning, action, learning, evaluation, verification, recovery, or data selection.
  arXiv  Detail & Related papers  (2026-09-15T06:22:42Z)
• GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation [63.853929983870955]
  We introduce Genie Envisioner Act 2.0 (GE-Act 2.0), a world-action model whose trainable generative and action components are all from scratch on manipulation data.<n>It combines a control-oriented autoencoder (CoAE), a single-step visual planner (SVP), and an inverse dynamics model (IDM)<n>The components are then jointly trained with knowledge-aligned selective optimization (KASO), which reduces mismatched supervision by selecting only predicted futures judged behaviorally compatible with the recorded action.
  arXiv  Detail & Related papers  (2026-09-04T17:16:48Z)
• PredVLA: Predictive Sensorimotor Modeling for Sub-Million-Parameter Robot Manipulation [4.55011976634859]
  We present PredVLA, a language-conditioned predictive-coding policy with only 0.68 million trainable network parameters and no robot-data pretraining.<n>Its hierarchical recurrent dynamics predict visual features and proprioception, while observations influence latent state only through prediction-error-driven online inference.
  arXiv  Detail & Related papers  (2026-08-27T06:27:11Z)
• GaussianDream++: Efficient 3D Gaussian World Modeling for Robotic Manipulation [55.47130289185285]
  Vision-Language-Action (VLA) policies have advanced language-conditioned robotic manipulation.<n>We present textbfMethodname, a compact, policy-native extension that inserts textbfWorld State Tokens and textbfWorld Prediction Tokens directly into the VLA backbone.
  arXiv  Detail & Related papers  (2026-08-26T11:41:55Z)
• EMPIRE: Explicit Manipulation Planning as a Learnable Intermediate Representation for Egocentric Hand-Motion Forecasting [75.76075128622837]
  We propose a two-stage framework that introduces Explicit Manipulation Planning as an Intermediate Representation for Egocentric hand-motion forecasting.<n>A motion generator synthesizes future bimanual hand motions conditioned on frozen planner representations.<n>Our method achieves state-of-the-art forecasting accuracy, with an MPJPE of 84.53 mm and a finger-relative error of 38.97mm.
  arXiv  Detail & Related papers  (2026-08-23T14:58:31Z)
• DreamTrajectory: Trajectory-Guided Action Generation with World Model Alignment for Mobile Manipulation [33.046851310859495]
  We present DreamTrajectory, a trajectory-guided framework for language-conditioned mobile manipulation.<n>On MS-HAB, trajectory guidance raises average success from 32.3% to 47.5% and test-time refinement further to 54.8%.<n>On three real-world mobile manipulation tasks, the corresponding average success rates are 63.3%, 81.7%, and 90.0%.
  arXiv  Detail & Related papers  (2026-08-02T17:02:48Z)
• TacWAM: Anchor-Guided World Action Model with Mechanics-Aware Tactile Prediction [15.274487606802124]
  World Action Models (WAMs) combine future-state prediction with robot action generation, but existing approaches largely rely on visual futures.<n>We present TacWAM, a mechanics-aware tactile WAM that addresses this challenge in three steps.<n>We evaluate TacWAM on four real-world contact-rich manipulation tasks covering fragile grasping, sustained surface contact, and dynamic in-hand manipulation.
  arXiv  Detail & Related papers  (2026-07-30T15:47:01Z)
• FeelWorld: Visuo-Tactile World Model for Hierarchical Contact Prediction and Planning [17.259290794993895]
  We introduce FeelWorld, a visuo-tactile world model that predicts future visual latents and three tactile states.<n>FeelWorld organizes these states hierarchically as contact state, a 3D tactile latent that encodes force-related information, and slip state.<n>To prevent irrelevant tactile signals during free-space motion from degrading visual prediction, we introduce a contact-gated asymmetric attention mechanism.
  arXiv  Detail & Related papers  (2026-07-27T11:00:26Z)
• TouchWorld: A Predictive and Reactive Tactile Foundation Model for Dexterous Manipulation [14.063238637690597]
  Dexterous manipulation in everyday environments requires both anticipation and reaction.<n>We introduce TouchWorld, a predictive-and-reactive tactile foundation model for dexterous manipulation.
  arXiv  Detail & Related papers  (2026-07-08T11:28:13Z)
• Tactile-WAM: Touch-Aware World Action Model with Tactile Asymmetric Attention [53.84395207661823]
  We propose Tactile-WAM, a touch-aware WAM with a Tactile Asymmetric Attention Mechanism (TAAM)<n>On ManiFeel, Tactile-WAM improves the mean success rate by 38.9% overall and by 86% on contact-rich tasks.
  arXiv  Detail & Related papers  (2026-06-25T06:54:56Z)
• World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry [82.93104394404781]
  General-purpose world models promise scalable policy evaluation, optimization, and planning.<n>We propose World Action Verifier (WAV), a framework that enables world models to identify their own prediction errors and self-improve.
  arXiv  Detail & Related papers  (2026-04-02T12:48:36Z)
• Global Prior Meets Local Consistency: Dual-Memory Augmented Vision-Language-Action Model for Efficient Robotic Manipulation [95.89924101984566]
  We introduce OptimusVLA, a dual-memory VLA framework with Global Prior Memory (GPM) and Local Consistency Memory (LCM)<n>GPM replaces Gaussian noise with task-level priors retrieved from semantically similar trajectories.<n>LCM injects a learned consistency constraint that enforces temporal coherence and smoothness of trajectory.
  arXiv  Detail & Related papers  (2026-02-22T15:39:34Z)
• When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Models [54.08784776767683]
  We present Physical-Conditioned World Model Attack (PhysCond-WMA), the first white-box world model attack that perturbs physical-condition channels.<n>PhysCond-WMA induces semantic, logic, or decision-level distortion while preserving perceptual fidelity.
  arXiv  Detail & Related papers  (2026-02-21T07:22:37Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)
• Detect Anything via Next Point Prediction [51.55967987350882]
  Rex- Omni is a 3B-scale MLLM that achieves state-of-the-art object perception performance.<n>On benchmarks like COCO and LVIS, Rex- Omni attains performance comparable to or exceeding regression-based models.
  arXiv  Detail & Related papers  (2025-10-14T17:59:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.