AquaJEPA: Action-Conditioned Multimodal Predictive Representations for Underwater Robot Dynamics

• URL: http://arxiv.org/abs/2607.29393v1
• Date: Fri, 31 Jul 2026 13:11:11 GMT
• Title: AquaJEPA: Action-Conditioned Multimodal Predictive Representations for Underwater Robot Dynamics
• Abstract summary: AquaJEPA fuses an RGB camera, forward-looking sonar, and proprioception with explicit sensor validity.<n>It predicts a future latent target conditioned on eight-thruster commands.<n>It significantly outperforms three action-conditioned predictive baselines in paired final error.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Underwater robots combine complementary sensors whose reliability changes abruptly with water visibility, viewpoint, and vehicle motion. We introduce AquaJEPA, an action-conditioned joint-embedding predictive model that fuses an RGB camera, forward-looking sonar, and proprioception with explicit sensor validity. It predicts a future latent target conditioned on eight-thruster commands and supplies velocity and sonar-profile predictions to a shared receding-horizon planner. We study the method in Stonefish against reactive, state-only, ordinary multimodal, supervised dynamics, and recurrent world-model baselines. We further isolate the EMA target, action margin, masks, and modality dropout. A preregistered 120-environment replication comprises five independent replicates of a grid crossing three unseen obstacle maps, four water-visibility coefficients, and nominal versus shifted dynamics, while intermittently removing DVL observations. In 120 fresh paired environments with scheduled DVL loss, AquaJEPA reaches 74 goals, versus 68 for both state-only and the recurrent world model, and attains the lowest mean final error (0.906 m). Paired final-error reductions relative to ordinary multimodal prediction, supervised dynamics, and the recurrent world model are 0.273 m (95% CI: 0.190-0.356), 0.364 m (0.260-0.468), and 0.106 m (0.025-0.187), respectively. AquaJEPA therefore achieves the best aggregate closed-loop performance and significantly outperforms three action-conditioned predictive baselines in paired final error; its advantage over state-only remains statistically unresolved.

Related papers

• GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation [63.853929983870955]
  We introduce Genie Envisioner Act 2.0 (GE-Act 2.0), a world-action model whose trainable generative and action components are all from scratch on manipulation data.<n>It combines a control-oriented autoencoder (CoAE), a single-step visual planner (SVP), and an inverse dynamics model (IDM)<n>The components are then jointly trained with knowledge-aligned selective optimization (KASO), which reduces mismatched supervision by selecting only predicted futures judged behaviorally compatible with the recorded action.
  arXiv  Detail & Related papers  (2026-09-04T17:16:48Z)
• GigaBrain-WBC-0.5: A Behavior World Model for Robust Whole-Body Control with Environment Interaction [68.67772166423812]
  We present GigaBrain-WBC-0.5, the first Behavior World Model (BWM) for humanoid whole-body control.<n>Rather than a purely reactive tracker, we train a causal Transformer to jointly predict its next action, next state, and the distribution over its next latent behavior command.<n>An automatic terrain-annotation pipeline recovers full 3D contact geometry from retargeted motion, enabling terrain annotation at the scale of existing motion datasets.
  arXiv  Detail & Related papers  (2026-08-18T18:21:36Z)
• PhyLatent: Learning Dynamics-Relevant Representations for JEPA World Models [12.761667671058767]
  PhyLatent is a dynamics-relevant training objective for JointEmbedding Predictive Architecture (JEPA) world models.<n>We identify three failure modes in JEPA world models: physical invariance collapse, physical identifiability collapse, and counterfactual dynamics collapse.<n>On OGBench-Cube, PhyLatent reduces the three failure rates from 15.60%, 6.71%, and 8.41% to 7.53%, 0.95%, and 4.62%, respectively, and improves model predictive control (MPC) success from 70.0% to 78.1%.
  arXiv  Detail & Related papers  (2026-08-06T08:02:15Z)
• Qantara: Bridge-Flow Training for Multi-Paradigm JEPA Control [5.357451930622806]
  Joint-Embedding Predictive Architectures underpin a growing family of latent world models for control from raw pixels.<n>A single checkpoint that serves both would defer this choice to inference, when deployment constraints determine which path wins.<n>We present Qantara, an end-to-end JEPA whose joint training objective pairs a Brownian-bridge interpolant between consecutive clean latents.
  arXiv  Detail & Related papers  (2026-07-06T12:12:50Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• UWM-JEPA: Predictive World Models That Imagine in Belief Space [0.2864713389096699]
  We introduce the Unitary World Model JEPA, a JEPA world model with a density-matrix latent on a joint system-environment space and a learned unitary predictor.<n>The construction preserves the joint-state spectrum exactly during rollout, so the predictor itself cannot dissipate the represented uncertainty.<n>For JEPA world models to imagine under partial observability, latent geometry and predictor dynamics matter, not frozen context-encoding capacity alone.
  arXiv  Detail & Related papers  (2026-05-25T00:28:51Z)
• Instance-level Visual Active Tracking with Occlusion-Aware Planning [61.982298426203165]
  Visual Active Tracking (VAT) aims to control cameras to follow a target in 3D space.<n>VAT faces two key bottlenecks in real-world deployment: confusion from visually similar distractors and severe failure under occlusions.<n>We propose OA-VAT, a unified pipeline with three complementary modules.
  arXiv  Detail & Related papers  (2026-04-23T09:11:50Z)
• Edge-Efficient Two-Stream Multimodal Architecture for Non-Intrusive Bathroom Fall Detection [2.669350701173451]
  Falls in wet bathroom environments are a major safety risk for seniors living alone.<n>Recent work has shown that mmWave-only, vibration-only, and existing multimodal schemes can support privacy-preserving, non-intrusive fall detection.<n>We propose a two-stream architecture that encodes radar signals with a Motion--Mamba branch for long-range motion patterns and processes floor vibration.
  arXiv  Detail & Related papers  (2026-03-17T18:54:21Z)
• LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels [49.35636088613484]
  Joint Embedding Predictive Architectures (JEPAs) offer a compelling framework for learning world models in compact latent spaces.<n>We introduce LeModelWorld, the first JEPA that trains stably end-to-end from raw pixels using only two loss terms.<n>With 15M parameters trainable on a single GPU in a few hours, LeWM plans up to 48x faster than foundation-model-based world models.
  arXiv  Detail & Related papers  (2026-03-13T19:48:14Z)
• Adaptive Enhancement and Dual-Pooling Sequential Attention for Lightweight Underwater Object Detection with YOLOv10 [0.0]
  This manuscript introduces a streamlined yet robust framework for underwater object detection, grounded in the YOLOv10 architecture.<n>The proposed method integrates a Multi-Stage Adaptive Enhancement module to improve image quality and a Dual-Pooling Sequential Attention mechanism to strengthen multi-scale feature representation.
  arXiv  Detail & Related papers  (2026-03-04T07:39:57Z)
• When World Models Dream Wrong: Physical-Conditioned Adversarial Attacks against World Models [54.08784776767683]
  We present Physical-Conditioned World Model Attack (PhysCond-WMA), the first white-box world model attack that perturbs physical-condition channels.<n>PhysCond-WMA induces semantic, logic, or decision-level distortion while preserving perceptual fidelity.
  arXiv  Detail & Related papers  (2026-02-21T07:22:37Z)
• BadCLIP++: Stealthy and Persistent Backdoors in Multimodal Contrastive Learning [73.46118996284888]
  Research on backdoor attacks against multimodal contrastive learning models faces two key challenges: stealthiness and persistence.<n>We propose BadCLIP++, a unified framework that tackles both challenges.<n>For stealthiness, we introduce a semantic-fusion QR micro-trigger that embeds imperceptible patterns near task-relevant regions.<n>For persistence, we stabilize trigger embeddings via radius shrinkage and centroid alignment.
  arXiv  Detail & Related papers  (2026-02-19T08:31:16Z)
• Meta-Continual Mobility Forecasting for Proactive Handover Prediction [0.3906427348768226]
  Short-term mobility forecasting is a core requirement for proactive handover (HO) in cellular networks.<n>We propose a lightweight meta-continual forecasting framework that integrates a GRU-based predictor, Reptile meta-initialization for fast few-shot adaptation, and an EWMA residual detector that triggers compact online updates only when drift occurs.<n>The model is lightweight (128k parameters) and suitable for edge deployment in 5G and 6G systems.
  arXiv  Detail & Related papers  (2025-12-03T19:48:20Z)
• Agentic World Modeling for 6G: Near-Real-Time Generative State-Space Reasoning [70.56067503630486]
  We argue that sixth-generation (6G) intelligence is not fluent token prediction but calibrated the capacity to imagine and choose.<n>We show that WM-MS3M cuts mean absolute error (MAE) by 1.69% versus MS3M with 32% fewer parameters and similar latency, and achieves 35-80% lower root mean squared error (RMSE) than attention/hybrid baselines with 2.3-4.1x faster inference.
  arXiv  Detail & Related papers  (2025-11-04T17:22:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.