Your VLM Already Knows When: Training-Free Temporal Grounding by Asking Yes or No

• URL: http://arxiv.org/abs/2608.08315v1
• Date: Sat, 08 Aug 2026 19:57:47 GMT
• Title: Your VLM Already Knows When: Training-Free Temporal Grounding by Asking Yes or No
• Abstract summary: Multimodal LLMs that recognise events reliably still fail to say when they happen.<n>We show that this failure lives in the task interface, not in perception.
• Score: 2.531680975633497
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal LLMs that recognise events reliably still fail to say when they happen. Prompted for timestamps, strong VLMs reach as little as $3.8\%$ R@0.5 on Charades-STA, and $77$ to $80\%$ of their wrong predictions carry low output entropy: the models are confidently wrong, and entropy-based error detection stays below a random classifier. We show that this failure lives in the task interface, not in perception. Holding the weights fixed, replacing timestamp regression with a coarse-to-fine scan of binary questions, whose first-token probabilities are consumed only as a ranking, raises R@0.5 by $28$ to $50$ points across four frozen backbones. The residual failures decompose into two measurable axes: a perception axis that moves with the backbone, and a geometry axis that is analytically predictable from the ratio of the output-window and event widths. FV-Action, the training-free method built on this analysis, reaches $56.8\%$ R@0.5 on Charades-STA, above the same backbone's native grounding pipeline and the strongest training-free result on this benchmark; it surpasses every TVG-trained model evaluated zero-shot on TACoS, and improves over direct prediction on ActivityNet Captions and QVHighlights, with no temporal supervision at any stage.

Related papers

• ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training [66.72708893922605]
  Post-training of large multimodal models should add new capabilities while preserving those from pre-training.<n>SFT gives explicit target supervision that learns a task from near-zero accuracy, but its off-policy targets move the model far enough to cause forgetting.<n>We introduce ReDraft, which obtains both from the model's own failures.
  arXiv  Detail & Related papers  (2026-09-15T04:59:03Z)
• Lévy Attention: Single-Pass Predictive Uncertainty for Continuous-Time Attention [0.0]
  Lévy Attention is a cross-attention operator whose output is integral against an inhomogeneous random measure.<n>On t-PatchGNN operator swap costs at most 5.6% accuracy against a matched control and nothing on the sparsest dataset.<n>One pass ranks 3,383 unseen patients by trust in 1.4 seconds.
  arXiv  Detail & Related papers  (2026-08-19T17:50:16Z)
• Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors [51.56484100374058]
  Autoregressive models accumulate error over long rollouts, yet at deployment there is no ground truth to measure it against.<n>We train a single conditional latent diffusion model that steps a dynamical system forward or backward in time via a direction flag.<n>We show that this bidirectionality supplies a measurement-free test-time error signal.
  arXiv  Detail & Related papers  (2026-08-01T13:49:46Z)
• When Can You Correct Distribution Drift in Temporal Graph Generation? A Sharpening--Drift Tension and an Impossibility for Observation-Based Correction [16.716278644368]
  Generative models of temporal graphs are trained on one stretch of an evolving network and deployed on the next.<n>We show this degradation is derivable, general, and not fixable from observations.<n>We prove that any corrector measurable with respect to past observations leaves at least the conditional variance of the statistic it tracks.
  arXiv  Detail & Related papers  (2026-07-27T16:59:57Z)
• Black-Box Assisted Regression: Phase Transitions and Minimax Optimality [13.030603690976202]
  Foundation models are often used as fixed black-box predictors for downstream tasks with limited labeled data.<n>We give a finite-sample minimax characterization showing a phase transition at $_c(n) asymp n-/(2+d)$, with leading risk $min2, n-2/(2+d)$.<n>We analyze a Safe Residual Estimator: it learns a correction around $f_0$, initializes the residual head at zero so the initial predictor equals $f_0$, and uses holdout selection
  arXiv  Detail & Related papers  (2026-06-24T12:11:59Z)
• Capacity-Constrained Online Convex Optimization with Delayed Feedback [58.59385794080679]
  We study delayed online convex optimization (OCO) under a hard capacity constraint.<n>We find that for first-order feedback, capacity $C = (log T)$ suffices to recover standard delayed OCO rates up to logarithmic factors.<n>For bandit feedback, the regret rates are modulated by powers of $(1 + _textmax/C)$, where $_textmax$ is the maximum number of pending observations at any time.
  arXiv  Detail & Related papers  (2026-06-10T06:37:06Z)
• Causal Label Recovery in Payment Networks [0.0]
  Fraud detection models in payment networks train on chargeback labels that are systematically biased.<n>A companion paper [arXiv05:26.27557] proved that these four impairments impose a minimax lower bound on detection performance.<n>We formalize the observation pipeline as a sequential missing-data problem with three stages and a corruption layer, and construct the Sequential Triply Robust estimator.
  arXiv  Detail & Related papers  (2026-05-28T02:43:34Z)
• Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect [0.0]
  We investigate whether small language models can detect and report on perturbations their own internal activations.<n>We first show that the binary detection paradigm used in prior work is confounded in small models, as steering biases the model toward affirmative responses regardless of the question content.<n>We then introduce emphIntrospection Fine-Tuning (IFT), supervised fine-tuning on sentence-localization examples constructed from the model's own perturbed forward passes.
  arXiv  Detail & Related papers  (2026-05-08T05:30:26Z)
• Learning Shrinks the Hard Tail: Training-Dependent Inference Scaling in a Solvable Linear Model [2.7074235008521246]
  We analyze neural scaling laws in a solvable model of last-layer fine-tuning where targets have intrinsic, instance-heterogeneous difficulty.<n>We show that learning shrinks the hard tail'' of the error distribution.
  arXiv  Detail & Related papers  (2026-01-07T10:00:17Z)
• ZIP-RC: Optimizing Test-Time Compute via Zero-Overhead Joint Reward-Cost Prediction [57.799425838564]
  We present ZIP-RC, an adaptive inference method that equips models with zero-overhead inference-time predictions of reward and cost.<n> ZIP-RC improves accuracy by up to 12% over majority voting at equal or lower average cost.
  arXiv  Detail & Related papers  (2025-12-01T09:44:31Z)
• Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers [90.50039419576807]
  Reinforcement Learning with Verifiable Rewards (RLVR) trains policies against automated verifiers to avoid costly human labeling.<n>To reduce vulnerability to verifier hacking, many RLVR systems collapse rewards to binary $0,1$ during training.<n>This choice carries a cost: it introduces textitfalse negatives (rejecting correct answers, FNs) and textitfalse positives (accepting incorrect ones, FPs)
  arXiv  Detail & Related papers  (2025-10-01T13:56:44Z)
• From Continual Learning to SGD and Back: Better Rates for Continual Linear Models [50.11453013647086]
  We analyze the forgetting, i.e., loss on previously seen tasks, after $k$ iterations.<n>We develop novel last-iterate upper bounds in the realizable least squares setup.<n>We prove for the first time that randomization alone, with no task repetition, can prevent catastrophic in sufficiently long task sequences.
  arXiv  Detail & Related papers  (2025-04-06T18:39:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.