DLAM: Distributional Latent Actions with Temporal Constraints

• URL: http://arxiv.org/abs/2607.27138v1
• Date: Wed, 29 Jul 2026 17:09:48 GMT
• Title: DLAM: Distributional Latent Actions with Temporal Constraints
• Abstract summary: We introduce DLAM, a distributional latent-action model that represents each transition as a diagonal Gaussian.<n>For downstream policy learning, we freeze the encoder and train a flow-matching policy to jointly generate mean transition sequences and robot actions.<n>On held-out transitions, DLAM learns more temporally consistent latent dynamics than existing latent-action baselines and achieves stronger direct and cumulative reconstruction on held-out videos.
• Score: 25.743937994244902
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-language-action (VLA) models remain constrained by scarce action-labeled robot data, whereas action-free videos offer abundant observations of physical change. Latent action models can extract such priors, but reconstruction-trained codes may predict future observations without the structure required for joint generation with robot actions. Existing structured methods add temporal constraints but retain deterministic transition points, so residual errors in locally inferred transitions may propagate and compound under recursive composition. We introduce DLAM, a distributional latent-action model that represents each transition as a diagonal Gaussian. Reconstruction conditioned on the reference frame grounds the mean in observed visual change, while normalized composition and reversal over equal-gap triplets constrain both the mean and dimension-wise variance. Variance composition uses a lightweight shared-correlation coefficient to account for dependence between adjacent transitions that share an intermediate frame, whereas reversal negates the mean and preserves the variance. For downstream policy learning, we freeze the encoder and train a flow-matching policy to jointly generate mean transition sequences and robot actions. On held-out transitions, DLAM learns more temporally consistent latent dynamics than existing latent-action baselines and achieves stronger direct and cumulative reconstruction on held-out videos. Under the same controlled $π_0$ transfer protocol, it also improves policy performance on MetaWorld MT50, LIBERO, and real-world manipulation tasks. Controlled ablations show that normalized mean constraints account for most of the reconstruction gain, while learned variance and correlation-aware composition provide complementary improvements in downstream control.

Related papers

• Balancing Emotional Alignment and Semantic Consistency in Image Generation via Reinforcement Learning with Valence-Arousal Anchoring [55.57242007016976]
  Continuous emotion control in text-to-image generation requires a model to improve affective alignment without changing the objects, layout, or scene described by the prompt.<n>Existing supervised emotion-injection methods often optimize feature-space proxies and may therefore exhibit emotion-semantic drift.<n>We address this problem with a flow-matching image-generation framework that combines continuous valence-arousal conditioning, Group Relative Policy Optimization, and a neutral semantic anchor.
  arXiv  Detail & Related papers  (2026-09-11T13:24:35Z)
• Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models [6.732084106610505]
  World Action Models (WAMs) augment robot policies by predicting how task-relevant scene states may evolve under interaction.<n>Latent transitions are commonly realized with Transformer-based predictors whose inductive structure is centered on token interaction rather than temporal evolution.<n>We introduce the Latent Evolution Operator Network (LEON), which models latent evolution in a learned observable space through context-modulated operator-based propagation and additive forcing.
  arXiv  Detail & Related papers  (2026-08-27T15:43:44Z)
• IADD-TR: Intervention-Aware Dynamics Decoupling with Targeted Regularization for Model-Based Reinforcement Learning [57.16513298507272]
  We propose IADD-TR, a unified framework combining Intervention-Aware Dynamics Decoupling (IADD) and Targeted Regularization (TR)<n>IADD factorizes transitions into an action-intervention stage and an action-free natural evolution stage, using a zero-action anchor to resolve the non-uniqueness of this two-stage factorization for robust generalization.<n>For policy learning, we derive TR from the efficient influence function of a replay-state policy-gradient functional.
  arXiv  Detail & Related papers  (2026-08-11T08:20:02Z)
• SegDiff: Segmented Trajectory Diffusion for Consistent and Adaptive Robot Manipulation [65.81476183180527]
  We introduce SegDiff, a closed-loop visuomotor policy that integrates the strengths of both paradigms.<n>SegDiff decomposes demonstrations into motion segments between keyposes and learns to predict the continuous trajectory from the current state to the next keypose.<n>SegDiff demonstrates significant performance gains over existing approaches across various simulated and real-world scenarios.
  arXiv  Detail & Related papers  (2026-07-13T02:48:03Z)
• Latent Actions from Factorized Transition Effects under Agent Ambiguity [15.872376579590437]
  We introduce Observed Transition Factorization (OTF), which decomposes each transition into a sparse set of observed transition primitives.<n>We also propose OTF-LAM, which abstracts motion primitives into action-like latents within the standard inverse-forward dynamics framework, and OTF-LAM-Dino, a decoder-free variant that predicts future states in a frozen DINOv2 representation space.
  arXiv  Detail & Related papers  (2026-06-29T16:45:04Z)
• ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Models [31.2829405167292]
  ALAM (Algebraic Latent Action Model) turns temporal relations in action-free video into structural supervision.<n>ALAM learns latent transitions grounded by reconstruction while being regularized by composition and reversal consistency.<n>For downstream VLA learning, we freeze the pretrained encoder and use its latent transition sequences as auxiliary generative targets.
  arXiv  Detail & Related papers  (2026-05-11T16:37:07Z)
• AnchorRefine: Synergy-Manipulation Based on Trajectory Anchor and Residual Refinement for Vision-Language-Action Models [60.04879435087352]
  Vision-language-action (VLA) policies generate actions within a single unified space.<n>We propose AnchorRefine, a hierarchical framework that factorizes VLA action modeling into trajectory anchor and residual refinement.<n> Experiments on LIBERO, CALVIN, and real-robot tasks demonstrate that AnchorRefine consistently improves both regression-based and diffusion-based VLA backbones.
  arXiv  Detail & Related papers  (2026-04-20T04:25:24Z)
• Structural Instability of Feature Composition [2.5754366051855837]
  We study the compositional-collapse threshold under a spherical dictionary model.<n>We show that, in the high-bias regime, ReLU converts microscopic correlation-induced variance fluctuations into a systematic drift.<n>We validate the predicted scaling trends on structured semantic features extracted from CLEVR.
  arXiv  Detail & Related papers  (2026-04-18T05:53:31Z)
• Error-Propagation-Free Learned Video Compression With Dual-Domain Progressive Temporal Alignment [92.57576987521107]
  We propose a novel unifiedtransform framework with dual-domain progressive temporal alignment and quality-conditioned mixture-of-expert (QCMoE)<n>QCMoE allows continuous and consistent rate control with appealing R-D performance.<n> Experimental results show that the proposed method achieves competitive R-D performance compared with the state-of-the-arts.
  arXiv  Detail & Related papers  (2025-12-11T09:14:51Z)
• On the Joint Minimization of Regularization Loss Functions in Deep Variational Bayesian Methods for Attribute-Controlled Symbolic Music Generation [47.38557855930304]
  Explicit latent variable models provide a flexible yet powerful framework for data synthesis.<n>We show that existing approaches struggle to jointly minimize both regularization objectives.<n>We show that suitable attribute transformations can help achieve both controllability and regularization of the target latent dimensions.
  arXiv  Detail & Related papers  (2025-11-10T14:09:25Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Time-series Generation by Contrastive Imitation [87.51882102248395]
  We study a generative framework that seeks to combine the strengths of both: Motivated by a moment-matching objective to mitigate compounding error, we optimize a local (but forward-looking) transition policy. At inference, the learned policy serves as the generator for iterative sampling, and the learned energy serves as a trajectory-level measure for evaluating sample quality.
  arXiv  Detail & Related papers  (2023-11-02T16:45:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.