MotionBlind: Probing the Illusion of Motion Understanding in Video-LLMs

• URL: http://arxiv.org/abs/2609.09528v1
• Date: Tue, 08 Sep 2026 23:15:20 GMT
• Title: MotionBlind: Probing the Illusion of Motion Understanding in Video-LLMs
• Abstract summary: Video large language models (Video-LLMs) are increasingly used as the perceptual front end of world models.<n>We introduce MotionBlind, a contrastive benchmark of self-recorded video for physically grounded motion.<n>A Video-LLM can watch two clips of the same person in the same room, name every object in both, and still fail to say which clip moves faster.
• Score: 4.006324106431453
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Video large language models (Video-LLMs) are increasingly used as the perceptual front end of world models, a role that assumes they can read motion: how fast something moves, which way it travels, how hard it is pushed. We show they cannot. A Video-LLM can watch two clips of the same person in the same room, name every object in both, and still fail to say which clip moves faster. We introduce MotionBlind, a contrastive benchmark of self-recorded video for physically grounded motion(speed, magnitude, and direction), the variables a world model must predict. Each instance is a pair of near-identical clips that differ only in motion. Each clip carries two complementary yes/no questions, giving four items per instance, and a model earns credit only if all four are correct. We report Instance Accuracy(IAcc), which has a 6.25% chance floor. Single-frame, appearance, and language-only shortcuts all collapse to it. MotionBlind complements the recent TimeBlind benchmark. We run a controlled study of six open and two frontier Video-LLMs, varying whether the video is present, whether frames are shown in the correct temporal order, and how frames are sampled (1 to 24 frames, four selection strategies). Open models sit near the 6.25% floor, and scale does not help. Removing the video drops every model to zero IAcc, and shuffling frames collapses IAcc to chance, so the task genuinely needs video in order. Neither more frames nor smarter frame selection closes the gap, because these change which frames are seen, not whether motion is read. Only Gemini3.1 Pro clears the benchmark overall, and even it fails on speed. A frontend that cannot tell two speeds of the same action apart is not yet a trustworthy source of supervision, reward, or evaluation for a world model.

Related papers

• TEMPO: Learning Temporal Context for Dynamic Robot Manipulation [12.057683796484232]
  We propose TEMPO, which augments a pretrained VLA with two temporal inputs to resolve motion ambiguity and state aliasing.<n>It improves Bottle Handover success from 44% to 74% and is the only method that solves state aliasing.<n>We further release TEMPO-Bench, a benchmark of over 50k annotated frames for evaluating motion-aware robot perception.
  arXiv  Detail & Related papers  (2026-09-15T08:56:52Z)
• ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow [24.623235518370695]
  We present ShadowDancer, a novel approach to any-action, frame-level control of interactive video world models.<n>Existing interfaces either encode an action loosely, leaving how it unfolds for the model to improvise, or encode it exactly through structured signals.<n>ShadowDancer addresses this with two key innovations: shadow pairs, video pairs that replay the same dynamics under independently resampled appearance, and cross-shadow prediction, which learns actions by predicting one shadow from the other.
  arXiv  Detail & Related papers  (2026-07-30T15:28:43Z)
• Frames2LoRA: Parametric Video Internalization for Vision-Language Models [52.46510577817688]
  Frames2LoRA is a method for parametric video internalization.<n>A perceiver hypernetwork reads the intermediate representations produced layer-by-layer as a frozen VLM encodes a video.<n>It generates a Low-Rank Adaptation (LoRA) adapter in a single forward pass.
  arXiv  Detail & Related papers  (2026-06-03T02:07:35Z)
• What-If World: A Causal Benchmark for General World Models in Embodied Scenarios [23.527918480081013]
  Video generation models are increasingly used as world simulators for tasks like driving and robotic manipulation.<n>What matters in these settings is not whether a single video looks right, but whether the model's output changes when its input changes.<n>We test this by giving a model two prompts describing the same scene with one physical detail varied, and checking whether the two videos diverge the way physics predicts.
  arXiv  Detail & Related papers  (2026-05-26T19:02:26Z)
• DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction [73.7305982336243]
  DuoMo is a generative method that recovers human motion in world-space coordinates from unconstrained videos with noisy or incomplete observations.<n>Our approach addresses the problem by factorizing motion learning into two diffusion models.<n>Together, the two models can reconstruct motion across diverse scenes and trajectories, even from highly noisy or incomplete observations.
  arXiv  Detail & Related papers  (2026-03-03T18:54:17Z)
• In-2-4D: Inbetweening from Two Single-View Images to 4D Generation [63.68181731564576]
  We propose a new problem, Inbetween-2-4D, for generative 4D (i.e., 3D + motion) in interpolate two single-view images.<n>In contrast to video/4D generation from only text or a single image, our interpolative task can leverage more precise motion control to better constrain the generation.
  arXiv  Detail & Related papers  (2025-04-11T09:01:09Z)
• Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts [67.5094490054134]
  We propose a practical framework, named Follow-Your-Click, to achieve image animation with a simple user click. Our framework has simpler yet precise user control and better generation performance than previous methods.
  arXiv  Detail & Related papers  (2024-03-13T05:44:37Z)
• Masked Motion Encoding for Self-Supervised Video Representation Learning [84.24773072241945]
  We present Masked Motion MME, a new pre-training paradigm that reconstructs both appearance and motion information to explore temporal clues. Motivated by the fact that human is able to recognize an action by tracking objects' position changes and shape changes, we propose to reconstruct a motion trajectory that represents these two kinds of change in the masked regions. Pre-trained with our MME paradigm, the model is able to anticipate long-term and fine-grained motion details.
  arXiv  Detail & Related papers  (2022-10-12T11:19:55Z)
• Revealing Single Frame Bias for Video-and-Language Learning [115.01000652123882]
  We show that a single-frame trained model can achieve better performance than existing methods that use multiple frames for training. This result reveals the existence of a strong "static appearance bias" in popular video-and-language datasets. We propose two new retrieval tasks based on existing fine-grained action recognition datasets that encourage temporal modeling.
  arXiv  Detail & Related papers  (2022-06-07T16:28:30Z)
• No frame left behind: Full Video Action Recognition [26.37329995193377]
  We propose full video action recognition and consider all video frames. We first cluster all frame activations along the temporal dimension. We then temporally aggregate the frames in the clusters into a smaller number of representations.
  arXiv  Detail & Related papers  (2021-03-29T07:44:28Z)
• Layered Neural Rendering for Retiming People in Video [108.85428504808318]
  We present a method for retiming people in an ordinary, natural video. We can temporally align different motions, change the speed of certain actions, or "erase" selected people from the video altogether. A key property of our model is that it not only disentangles the direct motions of each person in the input video, but also correlates each person automatically with the scene changes they generate.
  arXiv  Detail & Related papers  (2020-09-16T17:48:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.