Temporal Straightening for Latent Planning

• URL: http://arxiv.org/abs/2603.12231v1
• Date: Thu, 12 Mar 2026 17:49:47 GMT
• Title: Temporal Straightening for Latent Planning
• Abstract summary: We introduce temporal straightening to improve representation learning for latent planning.<n>We show that reducing curvature makes the Euclidean distance in latent space a better proxy for the geodesic distance.
• Score: 58.131390107122456
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Learning good representations is essential for latent planning with world models. While pretrained visual encoders produce strong semantic visual features, they are not tailored to planning and contain information irrelevant -- or even detrimental -- to planning. Inspired by the perceptual straightening hypothesis in human visual processing, we introduce temporal straightening to improve representation learning for latent planning. Using a curvature regularizer that encourages locally straightened latent trajectories, we jointly learn an encoder and a predictor. We show that reducing curvature this way makes the Euclidean distance in latent space a better proxy for the geodesic distance and improves the conditioning of the planning objective. We demonstrate empirically that temporal straightening makes gradient-based planning more stable and yields significantly higher success rates across a suite of goal-reaching tasks.

Related papers

• Latent Diffusion Planning for Imitation Learning [78.56207566743154]
  Latent Diffusion Planning (LDP) is a modular approach consisting of a planner and inverse dynamics model.<n>By separating planning from action prediction, LDP can benefit from the denser supervision signals of suboptimal and action-free data.<n>On simulated visual robotic manipulation tasks, LDP outperforms state-of-the-art imitation learning approaches.
  arXiv  Detail & Related papers  (2025-04-23T17:53:34Z)
• Planning Transformer: Long-Horizon Offline Reinforcement Learning with Planning Tokens [1.8416014644193066]
  We introduce Planning Tokens, which contain high-level, long time-scale information about the agent's future. We demonstrate that Planning Tokens improve the interpretability of the model's policy through the interpretable plan visualisations and attention map.
  arXiv  Detail & Related papers  (2024-09-14T19:30:53Z)
• PALMER: Perception-Action Loop with Memory for Long-Horizon Planning [1.5469452301122177]
  We introduce a general-purpose planning algorithm called PALMER. Palmer combines classical sampling-based planning algorithms with learning-based perceptual representations. This creates a tight feedback loop between representation learning, memory, reinforcement learning, and sampling-based planning.
  arXiv  Detail & Related papers  (2022-12-08T22:11:49Z)
• Trajectory-Constrained Deep Latent Visual Attention for Improved Local Planning in Presence of Heterogeneous Terrain [35.12388111707609]
  We present a reward-predictive, model-based deep learning method featuring trajectory-constrained visual attention for use in mapless, local visual navigation tasks. Our method learns to place visual attention at locations in latent image space which follow trajectories caused by vehicle control actions to enhance predictive accuracy during planning. We validated our model in visual navigation tasks of planning low turbulence, collision-free trajectories in off-road settings and hill climbing with locking differentials in the presence of slippery terrain.
  arXiv  Detail & Related papers  (2021-12-09T03:38:28Z)
• Model-Based Reinforcement Learning via Latent-Space Collocation [110.04005442935828]
  We argue that it is easier to solve long-horizon tasks by planning sequences of states rather than just actions. We adapt the idea of collocation, which has shown good results on long-horizon tasks in optimal control literature, to the image-based setting by utilizing learned latent state space models.
  arXiv  Detail & Related papers  (2021-06-24T17:59:18Z)
• Long-Horizon Visual Planning with Goal-Conditioned Hierarchical Predictors [124.30562402952319]
  The ability to predict and plan into the future is fundamental for agents acting in the world. Current learning approaches for visual prediction and planning fail on long-horizon tasks. We propose a framework for visual prediction and planning that is able to overcome both of these limitations.
  arXiv  Detail & Related papers  (2020-06-23T17:58:56Z)
• The Importance of Prior Knowledge in Precise Multimodal Prediction [71.74884391209955]
  Roads have well defined geometries, topologies, and traffic rules. In this paper we propose to incorporate structured priors as a loss function. We demonstrate the effectiveness of our approach on real-world self-driving datasets.
  arXiv  Detail & Related papers  (2020-06-04T03:56:11Z)
• Plan2Vec: Unsupervised Representation Learning by Latent Plans [106.37274654231659]
  We introduce plan2vec, an unsupervised representation learning approach that is inspired by reinforcement learning. Plan2vec constructs a weighted graph on an image dataset using near-neighbor distances, and then extrapolates this local metric to a global embedding by distilling path-integral over planned path. We demonstrate the effectiveness of plan2vec on one simulated and two challenging real-world image datasets.
  arXiv  Detail & Related papers  (2020-05-07T17:52:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.