J-LAW: Joint Localization and Actionable World Modeling via Coupled Latent Factor Graphs

• URL: http://arxiv.org/abs/2606.28712v1
• Date: Sat, 27 Jun 2026 03:39:44 GMT
• Title: J-LAW: Joint Localization and Actionable World Modeling via Coupled Latent Factor Graphs
• Abstract summary: Action-conditioned world models learn compact latent dynamics for planning but ignore global metric consistency.<n>We propose J-LAW: a coupled factor graph that jointly optimize metric object poses, latent world states, and latent landmark embeddings.<n>We cast observation, action-conditioned prediction, metric odometry, pose--latent coupling, latent loop closure, and latent landmark observation as probabilistic factors in a single MAP objective.
• Score: 5.78717443304847
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Classical SLAM estimates metric poses and a geometric map but produces no actionable predictive model for planning. Action-conditioned world models learn compact latent dynamics for planning but ignore global metric consistency and accumulate drift under open-loop rollout. We argue these are two views of the same estimation problem and propose J-LAW (Joint Localization and Actionable World Modeling) in this letter: a coupled factor graph that jointly optimizes metric object poses, latent world states, and latent landmark embeddings. The bridge is a pose-conditioned latent encoder and a learned pose--latent coupling factor, so that better localization improves the world model and vice versa. We cast observation, action-conditioned prediction, metric odometry, pose--latent coupling, latent loop closure, and latent landmark observation as probabilistic factors in a single MAP objective. Real-data experiments on PushT and WildGS show that coupled graph correction substantially reduces latent prediction RMSE and endpoint drift relative to open-loop rollout, while latent loop closure improves global trajectory consistency. J-LAW yields a map that is simultaneously metric (poses) and actionable (latent landmarks for planning).

Related papers

• Does Latent Planning Survive Point Clouds? Action-Conditioned JEPA World Models for Geometric Observations [0.0]
  JEPA world models make latent-space planning a practical route to control.<n>With 0.3-15% of scene points moving, the slow-feature optimum of latent prediction compounds with the geometric shortcut of 3D self-supervision.<n>We lift three canonical JEPA designs to point clouds, frozen-encoder, distribution-prior, and action-sensitive, and re-sense the stable-worldmodel benchmark.
  arXiv  Detail & Related papers  (2026-08-29T20:41:41Z)
• Orthogonal JEPA: Factorized Predictive States for Latent World Models [95.20158869896393]
  We introduce method, a latent world-modeling framework based on predictive factorization.<n>We show that method can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure.<n>Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.
  arXiv  Detail & Related papers  (2026-08-20T13:59:57Z)
• DyG$^2$T: Modeling Object Dynamics with 3D Gaussian Temporal-Spatial Particle Graph Transformer [86.4618122245946]
  DyG$2$T is a dynamics modeling framework that infers object motion trajectories by spatially completing and temporally discriminating Key Point representations.<n>Experiments on both synthetic and real-world datasets demonstrate that DyG$2$T achieves accurate dynamics modeling and reasoning.
  arXiv  Detail & Related papers  (2026-08-19T03:44:28Z)
• VIScore: Diagnosing Planning-Relevant Quality in Latent World Models [21.894781659564817]
  Regulating the latent space to an isotropic Gaussian distribution provides a stable and information-maximized landscape for world model planning.<n>We study this by comparing SIGReg and VISReg, two regularization loss functions with the same distribution target but different properties.<n>Compared with SIGReg, VISReg has more flexibility in controlling the weights of center, scale, and shape regularization, and a larger batch size brings a finer distribution approximation.<n>We propose the Veracity-Influence-Sobriety score (VIScore), a metric that quantifies the reachability and capacity of a predictor given the encoded
  arXiv  Detail & Related papers  (2026-08-11T17:34:10Z)
• Pondering the Way: Spatial-perceiving World Action Model for Embodied Navigation [24.310993695214094]
  We propose SWAM (Spatial-perceiving World Action Model), a task-centric joint observation-action generation framework.<n>Given start and goal RGB observations, SWAM performs single-pass inference to simultaneously generate intermediate RGB-D sequences and corresponding action trajectories.<n>We show that SWAM significantly outperforms state-of-the-art two-stage planners in success rate, trajectory accuracy, and inference efficiency.
  arXiv  Detail & Related papers  (2026-06-29T07:43:47Z)
• Sensorimotor World Models: Perception for Action via Inverse Dynamics [59.121736411156384]
  We introduce a sensorimotor world model trained end-to-end with inverse dynamics regularization.<n>It prevents representation collapse and induces action-aligned representations.<n>It learns compact, interpretable latent spaces and enables competitive planning performance across simple 2D and 3D control tasks.
  arXiv  Detail & Related papers  (2026-06-18T11:25:16Z)
• Latent Geometry Beyond Search: Amortizing Planning in World Models [17.571800971505787]
  We show that a learned representation can simplify control, rather than merely enabling prediction.<n>We replace iterative planning with a lightweight Goal-Conditioned Inverse Dynamics Model (GC-IDM)<n>Our controller matches or exceeds CEM in seven of eight environment-protocol settings while reducing per-decision cost by 100-130x.
  arXiv  Detail & Related papers  (2026-05-09T06:36:23Z)
• MoCapAnything V2: End-to-End Motion Capture for Arbitrary Skeletons [56.68975315643491]
  We present the first fully end-to-end framework in which Videoto-Pose and Pose-to-Rotation are learnable and optimized jointly.<n>We show that our method reduces rotation error from 17 degrees to 10 degrees, and to 6.54 degrees on unseen skeletons, while achieving 20x faster inference than mesh-based pipelines.
  arXiv  Detail & Related papers  (2026-04-30T17:16:38Z)
• Lifting Embodied World Models for Planning and Control [59.09016913513998]
  We train a lightweight policy that maps high-level actions to sequences of low-level joint actions.<n>We instantiate this framework for a human-like embodiment, defining the high-level action space as a small set of 2D waypoints.<n>We show that the lifted world model substantially outperforms searching directly in low-level joint space.
  arXiv  Detail & Related papers  (2026-04-28T23:59:19Z)
• Policies over Poses: Reinforcement Learning based Distributed Pose-Graph Optimization for Multi-Robot SLAM [1.3750624267664158]
  We consider the sequential distributed pose-graph optimization (PGO) in multi-robot localization.<n>We cast PGO as a partially observable game defined on a Markov Neural Network (GNN), where each action refines a single edge's pose estimate.<n>We show that our learned actors reduce the global trajectory by an average of 37.5%, while enhancing efficiency by at least 6X.
  arXiv  Detail & Related papers  (2025-10-26T16:21:24Z)
• S$^2$Transformer: Scalable Structured Transformers for Global Station Weather Forecasting [67.93713728260646]
  Existing time series forecasting methods often ignore or unidirectionally model spatial correlation when conducting large-scale global station forecasting.<n>This contradicts the nature underlying observations of the global weather system limiting forecast performance.<n>We propose a novel Structured Spatial Attention in this paper.<n>It partitions the spatial graph into a set of subgraphs and instantiates Intra-subgraph Attention to learn local spatial correlation within each subgraph.<n>It aggregates nodes into subgraph representations for message passing among the subgraphs via Inter-subgraph Attention -- considering both spatial proximity and global correlation.
  arXiv  Detail & Related papers  (2025-09-10T05:33:28Z)
• OR-Net: Pointwise Relational Inference for Data Completion under Partial Observation [51.083573770706636]
  This work uses relational inference to fill in the incomplete data. We propose Omni-Relational Network (OR-Net) to model the pointwise relativity in two aspects.
  arXiv  Detail & Related papers  (2021-05-02T06:05:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.