SLAP: The Semantic Least Action Principle for Variational Video-Language Modeling

• URL: http://arxiv.org/abs/2605.30750v1
• Date: Fri, 29 May 2026 02:28:33 GMT
• Title: SLAP: The Semantic Least Action Principle for Variational Video-Language Modeling
• Abstract summary: generative hallucination (e.g., latent diffusion) or autoregressive extrapolation often fail to maintain semantic consistency over long horizons.<n>We propose a paradigm shift from probabilistic generation to variational mechanics with the textbfSemantic Least Action Principle (SLAP).
• Score: 13.404415593709986
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the era of Large Video-Language Models (LVLMs), the computational necessity of sparse frame sampling creates a fundamental ``temporal gap'', rendering models blind to critical causal transitions. Existing solutions relying on generative hallucination (e.g., latent diffusion) or autoregressive extrapolation often fail to maintain semantic consistency over long horizons, suffering from object vanishing and energetic instability. We propose a paradigm shift from probabilistic generation to variational mechanics with the \textbf{Semantic Least Action Principle (SLAP)}. Drawing a rigorous isomorphism between classical mechanics and semantic dynamics, we model the latent video trajectory as a path on a Riemannian manifold governed by a Semantic Lagrangian. By formulating the interpolation task as a Boundary Value Problem (BVP) solved via the discrete Euler-Lagrange equations, SLAP naturally enforces object persistence without pixel-level rendering. Extensive experiments show the effectiveness of our proposed SLAP.

Related papers

• Stable by Construction: Variational Latent Markov Operators for Long-Horizon PDE Prediction [12.736876538921626]
  We introduce latent Markov dynamics in which physical states are represented by latent distributions and evolved through probabilistic transitions.<n>We instantiate the framework as the Variational Autoencoding Markov Operator (VAMO), which combines spatially resolved latent fields, structured Gaussian perturbations, and a neural-operator transition.
  arXiv  Detail & Related papers  (2026-09-15T04:28:59Z)
• Prototype-Anchored Generalized Manifold Regression for Unknown-Domain Object Detection [58.25418970608328]
  We study Single-Domain Generalized Object Detection (Single-DGOD), which aims to transfer a detector trained on a single source domain to multiple unseen domains.<n>Existing methods mainly rely on simulation-driven strategies, such as data augmentation or textual prompts, to enlarge the training distribution.<n>We propose Manifold Regression with Visual-Text Dual Chain-of-Thought (MR-DCoT), which formulates unknown-domain generalization as a manifold regression problem.
  arXiv  Detail & Related papers  (2026-07-08T09:27:15Z)
• LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations [9.35652906982887]
  We propose Least Action World Models (LaWM) to operationalize the Principle of Least Action in learned visual latent space.<n>LaWM encodes observations into learned generalized coordinates, learns a latent discrete Lagrangian over consecutive latent states.<n>Because the transition is induced by a discrete variational principle, LaWM provides a structure-preserving bias for long-horizon visual prediction.
  arXiv  Detail & Related papers  (2026-05-08T07:03:13Z)
• Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models [54.16797570104461]
  Diffusion-based language models (dLLMs) have emerged as a promising alternative to autoregressive language models.<n>We investigate non-autoregressive decoding in dLLMs by systematically analyzing its inference dynamics along the temporal axis.
  arXiv  Detail & Related papers  (2026-04-12T10:26:41Z)
• Neural Implicit Action Fields: From Discrete Waypoints to Continuous Functions for Vision-Language-Action Models [35.98187913967879]
  We propose a paradigm shift that reformulates action prediction from discrete waypoints to continuous action function regression.<n>Our approach achieves state-of-the-art results on CALVIN and LIBERO benchmarks across diverse backbones.
  arXiv  Detail & Related papers  (2026-03-02T11:48:24Z)
• IDLM: Inverse-distilled Diffusion Language Models [70.5793829229702]
  We extend Inverse Distillation, a technique originally developed to accelerate continuous diffusion models, to the discrete setting.<n>From a theoretical perspective, the inverse distillation objective lacks uniqueness guarantees, which may lead to suboptimal solutions.<n>We show that our method, Inverse-distilled Diffusion Language Models (IDLM), reduces the number of inference steps by 4x-64x.
  arXiv  Detail & Related papers  (2026-02-22T06:47:04Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion [52.315729095824906]
  MLLM Semantic-Corrected Ping-Pong-Ahead Diffusion (PPAD) is a novel framework that introduces a Multimodal Large Language Model (MLLM) as a semantic observer during inference.<n>It performs real-time analysis on intermediate generations, identifies latent semantic inconsistencies, and translates feedback into controllable signals that actively guide the remaining denoising steps.<n>Extensive experiments demonstrate PPAD's significant improvements.
  arXiv  Detail & Related papers  (2025-05-26T14:42:35Z)
• Generalized Interpolating Discrete Diffusion [65.74168524007484]
  Masked diffusion is a popular choice due to its simplicity and effectiveness.<n>We generalize a new family of general interpolating discrete diffusion (GIDD) which offers greater flexibility in the design of the noising processes.<n>Exploiting GIDD's flexibility, we explore a hybrid approach combining masking and uniform noise, leading to improved sample quality.
  arXiv  Detail & Related papers  (2025-03-06T14:30:55Z)
• A Stochastic Dynamical Theory of LLM Self-Adversariality: Modeling Severity Drift as a Critical Process [0.0]
  This paper introduces a continuous-time dynamical framework for understanding how large language models (LLMs) may self-amplify latent biases or toxicity through their own chain-of-thought reasoning.<n>The model posits an instantaneous "severity" variable $x(t) in [0,1]$ evolving under a differential equation (SDE) with a drift term $mu(x)$ and diffusion $sigma(x)$.
  arXiv  Detail & Related papers  (2025-01-28T08:08:25Z)
• FFHFlow: Diverse and Uncertainty-Aware Dexterous Grasp Generation via Flow Variational Inference [36.02645364048733]
  We propose FFHFlow, a flow-based variational framework that generates diverse, robust multi-finger grasps.<n>By exploiting the invertibility and exact likelihoods of flows, FFHFlow introspects shape uncertainty in partial observations.<n>We also integrate a discriminative grasp evaluator with the flow likelihoods, formulating an uncertainty-aware ranking strategy.
  arXiv  Detail & Related papers  (2024-07-21T13:33:08Z)
• Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors [11.01598521921903]
  DDVI is a black-box variational inference algorithm for latent variable models.<n>We introduce an expressive class of diffusion-based variational posteriors.<n>We train these posteriors with a novel regularized evidence lower bound.
  arXiv  Detail & Related papers  (2024-01-05T10:27:44Z)
• A Variational Inference Approach to Inverse Problems with Gamma Hyperpriors [60.489902135153415]
  This paper introduces a variational iterative alternating scheme for hierarchical inverse problems with gamma hyperpriors. The proposed variational inference approach yields accurate reconstruction, provides meaningful uncertainty quantification, and is easy to implement.
  arXiv  Detail & Related papers  (2021-11-26T06:33:29Z)
• Addressing the Topological Defects of Disentanglement via Distributed Operators [10.29148285032989]
  A popular approach to disentanglement consists in learning to map each of these factors to distinct subspaces of a model's latent representation. Here, we show that for a broad family of transformations acting on images, this approach introduces topological defects. Motivated by classical results from group representation theory, we study an alternative, more flexible approach to disentanglement.
  arXiv  Detail & Related papers  (2021-02-10T18:34:55Z)
• Discrete Variational Attention Models for Language Generation [51.88612022940496]
  We propose a discrete variational attention model with categorical distribution over the attention mechanism owing to the discrete nature in languages. Thanks to the property of discreteness, the training of our proposed approach does not suffer from posterior collapse.
  arXiv  Detail & Related papers  (2020-04-21T05:49:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.