PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models

• URL: http://arxiv.org/abs/2603.16958v1
• Date: Tue, 17 Mar 2026 02:35:36 GMT
• Title: PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models
• Abstract summary: We propose PhysQuantAgent, a framework for real-world object mass estimation using Vision-Language Models (VLMs)<n>We introduce three visual prompting methods that enhance the input image with object detection, scale estimation, and cross-sectional image generation to help the model comprehend the size and internal structure of the target object.<n> Experiments show that visual prompting significantly improves mass estimation accuracy on real-world data, suggesting the efficacy of integrating spatial reasoning with VLM knowledge for physical inference.
• Score: 38.21000830724267
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Vision-Language Models (VLMs) are increasingly applied to robotic perception and manipulation, yet their ability to infer physical properties required for manipulation remains limited. In particular, estimating the mass of real-world objects is essential for determining appropriate grasp force and ensuring safe interaction. However, current VLMs lack reliable mass reasoning capabilities, and most existing benchmarks do not explicitly evaluate physical quantity estimation under realistic sensing conditions. In this work, we propose PhysQuantAgent, a framework for real-world object mass estimation using VLMs, together with VisPhysQuant, a new benchmark dataset for evaluation. VisPhysQuant consists of RGB-D videos of real objects captured from multiple viewpoints, annotated with precise mass measurements. To improve estimation accuracy, we introduce three visual prompting methods that enhance the input image with object detection, scale estimation, and cross-sectional image generation to help the model comprehend the size and internal structure of the target object. Experiments show that visual prompting significantly improves mass estimation accuracy on real-world data, suggesting the efficacy of integrating spatial reasoning with VLM knowledge for physical inference.

Related papers

• Automatically Improving Simulation Physics for Articulated Objects [0.0]
  We introduce interaction-readiness, which characterizes whether an object can be reliably simulated under manipulation.<n>We present a simulator-in-the-loop approach for generating interaction-ready articulated objects from incomplete 3D assets.
  arXiv  Detail & Related papers  (2026-05-18T21:34:33Z)
• NICE FACT: Diagnosing and Calibrating VLMs in Quantitative Reasoning for Kinematic Physics [65.02899948986969]
  This work aims to provide a fundamental understanding of how vision-language models perceive the physical world, and utilize physical laws.<n>We propose NICE and FACT, a dual-diagnostic paradigm that explicitly decomposes quantitative reasoning for kinematic physics.<n>NICE studies our novel neighborhood-informed calibration method and novel metrics to evaluate and calibrate confidence reliability.
  arXiv  Detail & Related papers  (2026-05-08T20:17:44Z)
• Mirage2Matter: A Physically Grounded Gaussian World Model from Video [87.9732484393686]
  We present Simulate Anything, a graphics-driven world modeling and simulation framework.<n>Our approach reconstructs real-world environments into a photorealistic scene representation using 3D Gaussian Splatting (3DGS)<n>We then leverage generative models to recover a physically realistic representation and integrate it into a simulation environment via a precision calibration target.
  arXiv  Detail & Related papers  (2026-01-24T07:43:57Z)
• QuantiPhy: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models [14.860588888047708]
  QuantiPhy is the first benchmark designed to quantitatively measure a VLM's physical reasoning ability.<n>Our experiments on state-of-the-art VLMs reveal a consistent gap between their qualitative plausibility and actual numerical correctness.
  arXiv  Detail & Related papers  (2025-12-22T16:18:00Z)
• Do-Undo: Generating and Reversing Physical Actions in Vision-Language Models [57.71440995598757]
  We introduce the Do-Undo task and benchmark to address a critical gap in vision-language models.<n>Do-Undo requires models to simulate the outcome of a physical action and then accurately reverse it, reflecting true cause-and-effect in the visual world.
  arXiv  Detail & Related papers  (2025-12-15T18:03:42Z)
• PhysPose: Refining 6D Object Poses with Physical Constraints [26.47867548297063]
  We introduce PhysPose, a novel approach that integrates physical reasoning into pose estimation.<n>By leveraging scene geometry, PhysPose refines pose estimates to ensure physical plausibility.<n>We demonstrate its impact in robotics by significantly improving success rates in a challenging pick-and-place task.
  arXiv  Detail & Related papers  (2025-03-30T20:52:17Z)
• UniQuadric: A SLAM Backend for Unknown Rigid Object 3D Tracking and Light-Weight Modeling [7.626461564400769]
  We propose a novel SLAM backend that unifies ego-motion tracking, rigid object motion tracking, and modeling. Our system showcases the potential application of object perception in complex dynamic scenes.
  arXiv  Detail & Related papers  (2023-09-29T07:50:09Z)
• Physically Grounded Vision-Language Models for Robotic Manipulation [59.143640049407104]
  We propose PhysObjects, an object-centric dataset of 39.6K crowd-sourced and 417K automated physical concept annotations. We show that fine-tuning a vision-language model on PhysObjects improves its understanding of physical object concepts. We incorporate this physically grounded VLM in an interactive framework with a large language model-based robotic planner.
  arXiv  Detail & Related papers  (2023-09-05T20:21:03Z)
• Information-Theoretic Odometry Learning [83.36195426897768]
  We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
  arXiv  Detail & Related papers  (2022-03-11T02:37:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.