EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation

• URL: http://arxiv.org/abs/2608.01221v1
• Date: Sun, 02 Aug 2026 13:11:22 GMT
• Title: EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation
• Abstract summary: We present EndoWAM, to our knowledge, the first WAM for generalizable robotic endoscopic navigation.<n>Future grounding predicts task-relevant target regions in future observations from intermediate denoising features of a video world model.<n>This design injects target-aware supervision into predictive dynamics modeling, improving robustness to visual degradation and viewpoint changes.
• Score: 12.580242508201827
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Autonomous endoscopic navigation can reduce clinicians' operational burden, yet robust control remains challenging due to tissue deformation, transient occlusions, and rapidly changing viewpoints. Existing learning-based policies typically predict actions from current observations without explicitly modeling future dynamics, limiting their robustness and reliability in safety-critical settings. World Action Models (WAMs) offer a promising alternative by coupling predictive visual dynamics with action generation, but extending them to robotic endoscopy remains challenging due to limited training data, restricted viewpoint diversity, deformable anatomy, and high inference latency. We present EndoWAM, which is, to our knowledge, the first WAM for generalizable robotic endoscopic navigation. EndoWAM introduces future grounding, which predicts task-relevant target regions in future observations from intermediate denoising features of a video world model. Specifically, EndoWAM couples a lightweight diffusion transformer for future target-region prediction with a discrete action expert through a shared predictive representation. This design injects target-aware supervision into predictive dynamics modeling, improving robustness to visual degradation and viewpoint changes while enabling real-time control in a single denoising pass. We further introduce EndoMotion, a robotic endoscopic motion dataset spanning three anatomically distinct procedures: ureteroscopy, esophagoscopy, and endoscopic retrograde cholangiopancreatography (ERCP). EndoWAM consistently outperforms all baselines and alternative grounding strategies, while demonstrating strong zero-shot generalization to unseen viewpoints, environments, and targets. These results establish EndoWAM as a predictive, target-grounded framework for accurate, generalizable, and long-horizon navigation in visually constrained endoscopic environments.

Related papers

• Freezing of Gait Prediction Under Spatial Occlusion: An IMU-Supervised Cross-Modal Distillation Approach [0.0]
  Parkinson's disease is a progressive neurodegenerative disorder characterised by gradual deterioration of movement control.<n> automated freezing-of-gait (FOG) detection supports the objective assessment of gait-related motor impairment.<n>Two common approaches are used for FOG prediction: (i) analysing video recordings of the patient's movements and (ii) analysing data collected using inertial measurement unit (IMU) wearable sensors attached to the patient's lower limbs.<n>Video-based approaches may suffer detection errors during continuous turning-in-place tasks because the lower limbs undergo substantial geometric self-occlusion, degrading pose-estimation accuracy.
  arXiv  Detail & Related papers  (2026-09-09T07:29:13Z)
• Supervised Cross-Modal Feature Alignment for Zero-Wearable Freezing of Gait Detection in Parkinsonism [0.0]
  Objective assessment of Freezing of Gait (FoG) in Parkinson's disease relies predominantly on wearable Inertial Measurement Units (IMUs)<n>We propose a supervised cross-modal subspace distillation framework to resolve these physical observation limits.<n>During optimisation, pre-trained kinematic data from IMU sensors and contextual clinical metadata act as oracles to guide a deployable visual architecture.
  arXiv  Detail & Related papers  (2026-09-08T06:40:50Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• SurgVista: Long-Horizon Surgical World Modeling with Plausible Instrument-Tissue Dynamics [49.771521708359955]
  SurgVista is a surgical world model that mitigates spatial interaction incoherence and temporal fidelity collapse.<n>It consistently outperforms state-of-the-art methods across visual quality, temporal consistency, and interaction fidelity.
  arXiv  Detail & Related papers  (2026-06-18T07:47:28Z)
• HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models [58.191567345416836]
  World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics.<n>HarmoWAM fully leverages a world model to unify predictive and reactive control, enabling both general transit and precise manipulation.<n>We construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics.
  arXiv  Detail & Related papers  (2026-05-11T17:59:56Z)
• LatentPilot: Scene-Aware Vision-and-Language Navigation by Dreaming Ahead with Latent Visual Reasoning [51.969318585152116]
  LatentPilot exploits future observations during training as a valuable data source to learn action-conditioned visual dynamics.<n>We propose a flywheel-style training mechanism that iteratively collects on-policy trajectories and retrains the model to better match the agent's behavior distribution.<n> Experiments on R2R-CE, RxR-CE, and R2R-PE benchmarks achieve new SOTA results, and real-robot tests across diverse environments demonstrate LatentPilot's superior understanding of environment-action dynamics in scene.
  arXiv  Detail & Related papers  (2026-03-31T02:21:59Z)
• Synthetic Cardiac MRI Image Generation using Deep Generative Models [0.4733942082447995]
  Synthetic cardiac MRI (CMRI) generation has emerged as a promising strategy to overcome the scarcity of annotated medical imaging data.<n>Recent advances in GANs, VAEs, diffusion probabilistic models, and flow-matching techniques aim to generate anatomically accurate images.<n>This review aims to compare existing CMRI generation approaches through the lenses of fidelity, utility, and privacy.
  arXiv  Detail & Related papers  (2026-03-25T19:37:05Z)
• FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Model [73.03346643967309]
  We argue that effective joint visuomotor predictive modeling requires both temporal continuity and visually-conditioned supervision decoupling.<n>FutureVLA is designed to extract joint visuomotor embeddings by first decoupling visual and motor information.<n>In the post-training stage, we employ a latent embeddings alignment strategy, enabling diverse downstream VLA models to internalize these temporal priors.
  arXiv  Detail & Related papers  (2026-03-11T12:39:55Z)
• DINO-CVA: A Multimodal Goal-Conditioned Vision-to-Action Model for Autonomous Catheter Navigation [0.33727511459109777]
  This work moves towards autonomous catheter navigation by introducing DINO-CVA, a multimodal goal-conditioned behavior cloning framework.<n>The proposed model fuses visual observations and joystick kinematics into a joint embedding space, enabling policies that are both vision-aware and kinematic-aware.<n>Results show that DINO-CVA achieves high accuracy in predicting actions, matching the performance of a kinematics-only baseline.
  arXiv  Detail & Related papers  (2025-10-19T22:59:32Z)
• EndoVLA: Dual-Phase Vision-Language-Action Model for Autonomous Tracking in Endoscopy [26.132684811981143]
  Vision-Language-Action (VLA) models integrate visual perception, language grounding, and motion planning within an end-to-end framework.<n>EndoVLA performs three core tasks: (1) polyp tracking, (2) delineation and following of abnormal mucosal regions, and (3) adherence to circular markers during circumferential cutting.
  arXiv  Detail & Related papers  (2025-05-21T07:35:00Z)
• EchoWorld: Learning Motion-Aware World Models for Echocardiography Probe Guidance [79.66329903007869]
  We present EchoWorld, a motion-aware world modeling framework for probe guidance.<n>It encodes anatomical knowledge and motion-induced visual dynamics.<n>It is trained on more than one million ultrasound images from over 200 routine scans.
  arXiv  Detail & Related papers  (2025-04-17T16:19:05Z)
• Towards Better Surgical Instrument Segmentation in Endoscopic Vision: Multi-Angle Feature Aggregation and Contour Supervision [22.253074722129053]
  We propose a general embeddable approach to improve current deep neural network (DNN) segmentation models. The proposed method is validated with ablation experiments on the novel Sinus-Surgery datasets collected from surgeons' operations.
  arXiv  Detail & Related papers  (2020-02-25T05:28:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.