GridProbe: Posterior-Probing for Adaptive Test-Time Compute in Long-Video VLMs

• URL: http://arxiv.org/abs/2605.10762v1
• Date: Mon, 11 May 2026 15:57:46 GMT
• Title: GridProbe: Posterior-Probing for Adaptive Test-Time Compute in Long-Video VLMs
• Abstract summary: GridProbe is an efficient training-free posterior-probing inference paradigm.<n>It scores evidence in answer space using a frozen VLM's own reasoning.<n>It selects question-relevant frames adaptively, resulting in sub-quadratic attention cost with little to no accuracy loss.
• Score: 3.9266376632068485
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Long-video understanding in VLMs is bottlenecked by a single monolithic forward pass over thousands of frames at quadratic attention cost. A common mitigation is to first select a small subset of informative frames before the forward pass; common for training-free selectors via auxiliary encoder-space similarities. Such signals are capped by contrastive pretraining, which usually fails on reasoning-heavy queries (negation, cross-frame counting, holistic summarization). We propose GridProbe, an efficient training-free posterior-probing inference paradigm that scores evidence in answer space using a frozen VLM's own reasoning and then selects question-relevant frames adaptively, resulting in sub-quadratic attention cost with little to no accuracy loss. We arrange frames on a $K{\times}K$ grid and run lightweight row R and column C probes, where each probe reads its peak posterior as a query-conditioned confidence. The outer product of R and C yields an interpretable importance map whose skewness and kurtosis drive Shape-Adaptive Selection, a closed-form rule that reliably replaces the fixed frame budget $M$ with a per-question $M_{\mathrm{eff}}$. We show empirically that $M_{\mathrm{eff}}$ tracks intrinsic question difficulty without ever seeing the answer, a sign of test-time adaptive compute. On Video-MME-v2, GridProbe matches the monolithic baseline within $1.6$ pp Avg Acc at $3.36\times$ TFLOPs reduction, while on LongVideoBench it Pareto-dominates the baseline ($+0.9$ pp at $0.35\times$ compute). Because the selector and QA models can be decoupled, pairing a small 2B selector with a stronger 4B or 8B QA is strictly Pareto-dominant over the 2B monolithic baseline (up to $+4.0$ pp at $0.52\times$ compute, on average), with no retraining. Finally, the interpretability of the importance maps opens future avenues for behavioral diagnostics, grounding, and frame-selection distillation.

Related papers

• Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs [16.31176289941024]
  We show that Video LLMs improve smoothly or saturate as visual budgets grow.<n>We represent each frozen model--item pair by its response trajectory under controlled visual budgets.<n>We derive matched-grid measures of configuration complementarity, harmful transitions, and text overwrite.
  arXiv  Detail & Related papers  (2026-08-07T09:25:20Z)
• Coverage-Driven Adaptive Keyframe Selection for Video Understanding [12.078063042214573]
  We propose CSES, a training-free semantic selector that adaptively determines the numbers of frames to score and select.<n> CSES estimates the prominence of the frame-query to guide active acquisition and adapt the temporal coverage of each input.<n>It then formulates selection relevance as a coverage problem that jointly accounts for semantic relevance, temporal redundancy, and visual redundancy.
  arXiv  Detail & Related papers  (2026-08-01T15:24:06Z)
• Conditioning Tree-Based Diffusions and Flows for Probabilistic Tabular Regression [0.0]
  Tree-based diffusion models fit flexible conditional predictive distributions for regression without a neural density estimator.<n>We show these defaults are the binding constraint: what a gradient-boosted ensemble actually solves is a supervised regression problem.<n>We present DiffGBM, which makes them explicit along two axes.
  arXiv  Detail & Related papers  (2026-07-30T22:08:02Z)
• Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understanding [17.716106078756066]
  Long-video understanding commonly compresses videos into a small set of frames or visual tokens for answer generation.<n>Our key idea is to organize selected frames into query-relevant cross-frame evidence before generation.<n>Across four benchmarks and two Video-MLLM backbones, GenEvA consistently improves matched-frame baselines.
  arXiv  Detail & Related papers  (2026-07-30T16:55:00Z)
• PCA: Persistence-Aware Compression and Aggregation for Fast Video Large Language Models [63.31550146180189]
  This paper introduces a training-free $mathbfP$ersistence-Aware $mathbfC$ompression and $mathbfA$ggregation (PCA) method to preserve high-sampling raw visual information before the encoding stage.<n>PCA consistently outperforms existing state-of-the-art approaches in both efficiency and accuracy.
  arXiv  Detail & Related papers  (2026-07-22T05:17:38Z)
• Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors [69.75895255395702]
  Cross-modal attention at validation-selected extraction layers in MLLMs already provides query-relevant frame evidence without requiring autoregressive generation.<n>A lightweight MLLM selector, even with only 2B parameters, can extract frame-level evidence by converting selected-layer attention into relevance scores through query-conditioned aggregation.<n>Under a 32-frame budget, our selector improves over uniform sampling by up to 6.4 points on Video-MME and outperforms prior training-based selectors under matched frame budgets.
  arXiv  Detail & Related papers  (2026-07-17T07:04:31Z)
• Compander-Aligned Query Geometry for Quantized Zeroth-Order Optimization [12.92201130466754]
  CAQ-ZO forms one-grid-step Rademacher stencils $z pm r$ in $z = (x)$, maps endpoints back through $-1$, and updates in $z$.<n>Our theory proves the grid-spanarity, decomposes endpoint-rounding estimator residuals, and gives station bounds in which generic off-grid queries retain a $2/2$ residual channel while CAQ-ZO makes the query-time residual exactly zero.
  arXiv  Detail & Related papers  (2026-05-11T14:55:20Z)
• Total Generalized Variation regularization closes the gap between neural-eld and classical methods in seismic travel-time tomography [0.0]
  Traveltime tomography forces a tradeoff between mesh resolution and stability in which the regularizer choice dominates what can be recovered.<n>We introduce MIMIR, a differentiable framework that represents the 2D velocity field as a Fourier-feature neural network.
  arXiv  Detail & Related papers  (2026-05-11T04:13:52Z)
• Scaling Federated Linear Contextual Bandits via Sketching [49.12000877146222]
  This paper proposes Federated Sketch Contextual Linear Bandits (FSCLB)<n>Experiments on both synthetic and real-world datasets show that FSCLB significantly reduces computational and communication costs by over 90 %.
  arXiv  Detail & Related papers  (2026-05-01T08:22:06Z)
• CRPS-Optimal Binning for Conformal Regression [0.0]
  We propose a method for non-parametric conditional distribution estimation based on partitioning co-sorted observations into contiguous bins.<n>We form two complementary predictive objects: the Venn prediction band and a conformal prediction set based on CRPS as the nonconformity score.
  arXiv  Detail & Related papers  (2026-03-23T14:07:09Z)
• Adaptive Greedy Frame Selection for Long Video Understanding [11.923839324117674]
  Large vision--language models (VLMs) are increasingly applied to long-video question answering.<n>Inference is often bottlenecked by the number of input frames and resulting visual tokens.<n>We propose a question-adaptive greedy frame selection method that jointly optimize query relevance and semantic representativeness under a fixed frame budget.
  arXiv  Detail & Related papers  (2026-03-20T17:55:32Z)
• Post-Training with Policy Gradients: Optimality and the Base Model Barrier [27.674563695368665]
  We study post-training linear autoregressive models with outcome and process rewards.<n>We prove that variants of policy gradient (PG) can achieve likelihood $1 - varepsilon$ with an essentially minimax optimal number of reward queries.
  arXiv  Detail & Related papers  (2026-03-07T00:25:53Z)
• DISCO: Diversifying Sample Condensation for Efficient Model Evaluation [59.01400190971061]
  Costly evaluation reduces inclusivity, slows the cycle of innovation, and worsens environmental impact.<n>We argue that promoting diversity among samples is not essential; what matters is to select samples thatmaximise diversity in model responses.<n>Our method, $textbfDiversifying Sample Condensation (DISCO)$, selects the top-k samples with the greatest model disagreements.
  arXiv  Detail & Related papers  (2025-10-09T08:53:59Z)
• Fast Rates for Bandit PAC Multiclass Classification [73.17969992976501]
  We study multiclass PAC learning with bandit feedback, where inputs are classified into one of $K$ possible labels and feedback is limited to whether or not the predicted labels are correct. Our main contribution is in designing a novel learning algorithm for the agnostic $(varepsilon,delta)$PAC version of the problem.
  arXiv  Detail & Related papers  (2024-06-18T08:54:04Z)
• FABind: Fast and Accurate Protein-Ligand Binding [127.7790493202716]
  $mathbfFABind$ is an end-to-end model that combines pocket prediction and docking to achieve accurate and fast protein-ligand binding. Our proposed model demonstrates strong advantages in terms of effectiveness and efficiency compared to existing methods.
  arXiv  Detail & Related papers  (2023-10-10T16:39:47Z)
• Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models [96.76758318732308]
  We show that the recently proposed Deep Equilibrium Model (DEQ) can be naturally adapted to this form of computation. Our Landmark DEQ (LDEQ) achieves state-of-the-art performance on the WFLW facial landmark dataset.
  arXiv  Detail & Related papers  (2023-04-02T19:08:02Z)
• Revisiting Smoothed Online Learning [70.09792747315323]
  We investigate the problem of smoothed online learning, in which the online learner suffers both a hitting cost and a switching cost. To bound the competitive ratio, we assume the hitting cost is known to the learner in each round, and investigate the greedy algorithm which simply minimizes the weighted sum of the hitting cost and the switching cost.
  arXiv  Detail & Related papers  (2021-02-13T14:15:55Z)
• Optimal Robust Linear Regression in Nearly Linear Time [97.11565882347772]
  We study the problem of high-dimensional robust linear regression where a learner is given access to $n$ samples from the generative model $Y = langle X,w* rangle + epsilon$ We propose estimators for this problem under two settings: (i) $X$ is L4-L2 hypercontractive, $mathbbE [XXtop]$ has bounded condition number and $epsilon$ has bounded variance and (ii) $X$ is sub-Gaussian with identity second moment and $epsilon$ is
  arXiv  Detail & Related papers  (2020-07-16T06:44:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.