NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics

• URL: http://arxiv.org/abs/2605.08452v1
• Date: Fri, 08 May 2026 20:17:44 GMT
• Title: NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics
• Abstract summary: This work aims to provide a fundamental understanding of how vision-language models perceive the physical world, and utilize physical laws.<n>We propose NICE and FACT, a dual-diagnostic paradigm that explicitly decomposes quantitative reasoning for kinematic physics.<n>NICE studies our novel neighborhood-informed calibration method and novel metrics to evaluate and calibrate confidence reliability.
• Score: 65.02899948986969
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ability to derive precise spatial and physical insights is a cornerstone of vision-language models (VLMs), yet their poor performances in related spatial intelligence tasks such as physical reasoning remain a fundamental barrier. The community critically lacks a scientific analysis revealing whether VLMs faithfully reach answers or plausibly make guesses. This work aims to provide a fundamental understanding of how VLMs perceive the physical world, and utilize physical laws, while assessing the reliability of model confidence. We propose NICE and FACT, a dual-diagnostic paradigm that explicitly decomposes quantitative reasoning for kinematic physics: FACT diagnoses visual fidelity, physical law comprehension, and temporal grounding. NICE studies our novel neighborhood-informed calibration method and novel metrics to evaluate and calibrate confidence reliability. Evaluated across 6 latest state-of-the-art VLMs, we uncover that models fail to identify visual preconditions or utilize necessary physical laws to reach answers. This work highlights and establishes a standardized diagnostic paradigm to guide the development of faithful, physically-grounded VLMs.

Related papers

• PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models [38.21000830724267]
  We propose PhysQuantAgent, a framework for real-world object mass estimation using Vision-Language Models (VLMs)<n>We introduce three visual prompting methods that enhance the input image with object detection, scale estimation, and cross-sectional image generation to help the model comprehend the size and internal structure of the target object.<n> Experiments show that visual prompting significantly improves mass estimation accuracy on real-world data, suggesting the efficacy of integrating spatial reasoning with VLM knowledge for physical inference.
  arXiv  Detail & Related papers  (2026-03-17T02:35:36Z)
• QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models [14.860588888047708]
  QuantiPhy is the first benchmark designed to quantitatively measure a VLM's physical reasoning ability.<n>Our experiments on state-of-the-art VLMs reveal a consistent gap between their qualitative plausibility and actual numerical correctness.
  arXiv  Detail & Related papers  (2025-12-22T16:18:00Z)
• LikePhys: Evaluating Intuitive Physics Understanding in Video Diffusion Models via Likelihood Preference [57.086932851733145]
  We introduce LikePhys, a training-free method that evaluates intuitive physics in video diffusion models.<n>We benchmark intuitive physics understanding in current video diffusion models.<n> Empirical results show that, despite current models struggling with complex and chaotic dynamics, there is a clear trend of improvement in physics understanding as model capacity and inference settings scale.
  arXiv  Detail & Related papers  (2025-10-13T15:19:07Z)
• TRAVL: A Recipe for Making Video-Language Models Better Judges of Physics Implausibility [70.24211591214528]
  Video generative models produce sequences that violate intuitive physical laws, such as objects floating, teleporting, or morphing.<n>Existing Video-Language Models (VLMs) struggle to identify physics violations, exposing fundamental limitations in their temporal and causal reasoning.<n>We introduce TRAVL, a fine-tuning recipe that combines a balanced training dataset with a trajectory-aware attention module to improve motion encoding.<n>We propose ImplausiBench, a benchmark of 300 videos (150 real, 150 generated) that removes linguistic biases and isolates visual-temporal understanding.
  arXiv  Detail & Related papers  (2025-10-08T21:03:46Z)
• Does Physics Knowledge Emerge in Frontier Models? [19.035965618393096]
  Leading Vision-Language Models (VLMs) show strong results in visual perception and general reasoning.<n>But their ability to understand and predict physical dynamics remains unclear.<n>We benchmark six frontier VLMs on three physical simulation datasets.
  arXiv  Detail & Related papers  (2025-10-03T22:30:06Z)
• Mimicking the Physicist's Eye:A VLM-centric Approach for Physics Formula Discovery [98.58830663687911]
  VIPERR-aq1 is a multimodal model that performs Visual Induction for Equation Reasoning.<n>It integrates visual perception, trajectory data, and symbolic reasoning to emulate the scientific discovery process.<n>It consistently outperforms state-of-the-art VLM baselines in accuracy and interpretability.
  arXiv  Detail & Related papers  (2025-08-24T14:34:21Z)
• VLM4D: Towards Spatiotemporal Awareness in Vision Language Models [66.833085504228]
  We introduce V4DLM, the first benchmark specifically designed to evaluate visual language models (VLMs)<n>Our benchmark comprises diverse real-world and synthetic videos accompanied by carefully curated question-answer pairs.<n>We identify significant performance gaps compared to human baselines, highlighting fundamental deficiencies in existing models.
  arXiv  Detail & Related papers  (2025-08-04T06:06:06Z)
• Can Theoretical Physics Research Benefit from Language Agents? [50.57057488167844]
  Large Language Models (LLMs) are rapidly advancing across diverse domains, yet their application in theoretical physics research is not yet mature.<n>This position paper argues that LLM agents can potentially help accelerate theoretical, computational, and applied physics when properly integrated with domain knowledge and toolbox.<n>We envision future physics-specialized LLMs that could handle multimodal data, propose testable hypotheses, and design experiments.
  arXiv  Detail & Related papers  (2025-06-06T16:20:06Z)
• PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding [21.91860938879665]
  We show that Vision-Language Models (VLMs) excel in common-sense reasoning, but struggle with understanding the physical world.<n>We introduce PhysAgent, a framework that combines the generalization strengths of VLMs with the specialized expertise of vision models.<n>Our results show that enhancing VLMs' physical world understanding capabilities can help embodied agents such as MOKA.
  arXiv  Detail & Related papers  (2025-01-27T18:59:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.