PhysScene: A Scene Graph Dataset for Scientific Visual Reasoning in Physics Experiments

• URL: http://arxiv.org/abs/2606.09368v1
• Date: Mon, 08 Jun 2026 11:40:48 GMT
• Title: PhysScene: A Scene Graph Dataset for Scientific Visual Reasoning in Physics Experiments
• Abstract summary: Scene Graphs (SGs) provide structured representations of visual scenes by modeling objects and their pairwise relationships.<n>PhysScene is the first SG dataset tailored to physics experiments.<n>PhysScene encompasses specialized instruments, structured experimental setups, and functional relations intrinsic to experimental environments.
• Score: 37.632499627458714
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Scene Graphs (SGs) provide structured representations of visual scenes by modeling objects and their pairwise relationships. Despite recent progress, existing datasets primarily focus on generic natural contexts, leaving domain-specific and function-oriented scenes largely underexplored. This limitation restricts the evaluation of relational reasoning in scientific experimental scenes, thereby hindering the development of intelligent monitoring, analysis, and related applications in such scenes. To address this gap, we introduce PhysScene, the first SG dataset tailored to physics experiments. PhysScene encompasses specialized instruments, structured experimental setups, and functional relations intrinsic to experimental environments, enabling reasoning that extends beyond spatial co-occurrence to logical dependencies. Rather than pursuing large data scale, PhysScene focuses on strong semantic constraints and high relation density in experimental scenes, posing new challenges for existing scene parsing algorithms while offering opportunities for further improvements. Extensive analyses and experiments show that PhysScene complements existing benchmarks and establishes a valuable testbed for advancing scientific visual reasoning. The dataset is publicly available at https://github.com/ZMH-SDUST/PhysScene.

Related papers

• Modeling Scientific Experiment Scenes: Dataset and Model [35.54149729801777]
  Cross-Modal Dual-Path Generator (CM-DPG) is a model for robust open-vocabulary Scene Graph Generation (SGG)<n>Model enhances object-level semantic representations through joint visual-textual encoding.<n>Experiments on PhysScene and VG150 show CM-DPG achieves competitive performance across multiple evaluation settings.
  arXiv  Detail & Related papers  (2026-08-03T21:20:15Z)
• The Perception-Physics Paradox: Probing Scientific Alignment with TC-Bench [30.054788656949636]
  We introduce scientific alignment as an implicit objective for representation learning in scientific domains.<n>We study a principled, testable aspect of scientific alignment through structural isomorphism.<n>We release TC-Bench, a global, reproducible benchmark dataset with an automated construction pipeline for tropical cyclone research.
  arXiv  Detail & Related papers  (2026-05-23T23:51:19Z)
• PhyScensis: Physics-Augmented LLM Agents for Complex Physical Scene Arrangement [89.35154754765502]
  PhyScensis is an agent-based framework powered by a physics engine to produce physically plausible scene configurations.<n>Our framework preserves strong controllability over fine-grained textual descriptions and numerical parameters.<n> Experimental results show that our method outperforms prior approaches in scene complexity, visual quality, and physical accuracy.
  arXiv  Detail & Related papers  (2026-02-16T17:55:25Z)
• PhysLab: A Benchmark Dataset for Multi-Granularity Visual Parsing of Physics Experiments [16.800647681719045]
  We introduce PhysLab, the first video dataset that captures students conducting complex physics experiments.<n>The dataset includes four representative experiments that feature diverse scientific instruments and rich human-object interaction (HOI) patterns.<n>PhysLab comprises 620 long-form videos and provides multilevel annotations that support a variety of vision tasks.
  arXiv  Detail & Related papers  (2025-06-07T02:28:24Z)
• PhysGaia: A Physics-Aware Dataset of Multi-Body Interactions for Dynamic Novel View Synthesis [62.283499219361595]
  PhysGaia is a physics-aware dataset specifically designed for Dynamic Novel View Synthesis (DyNVS)<n>Our dataset provides complex dynamic scenarios with rich interactions among multiple objects.<n>PhysGaia will significantly advance research in dynamic view synthesis, physics-based scene understanding, and deep learning models integrated with physical simulation.
  arXiv  Detail & Related papers  (2025-06-03T12:19:18Z)
• LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery [141.39722070734737]
  We propose to enhance the knowledge-driven, abstract reasoning abilities of Large Language Models with the computational strength of simulations. We introduce Scientific Generative Agent (SGA), a bilevel optimization framework. We conduct experiments to demonstrate our framework's efficacy in law discovery and molecular design.
  arXiv  Detail & Related papers  (2024-05-16T03:04:10Z)
• X-VoE: Measuring eXplanatory Violation of Expectation in Physical Events [75.94926117990435]
  This study introduces X-VoE, a benchmark dataset to assess AI agents' grasp of intuitive physics. X-VoE establishes a higher bar for the explanatory capacities of intuitive physics models. We present an explanation-based learning system that captures physics dynamics and infers occluded object states.
  arXiv  Detail & Related papers  (2023-08-21T03:28:23Z)
• Learning dynamics from partial observations with structured neural ODEs [5.757156314867639]
  We propose a flexible framework to incorporate a broad spectrum of physical insight into neural ODE-based system identification. We demonstrate the performance of the proposed approach on numerical simulations and on an experimental dataset from a robotic exoskeleton.
  arXiv  Detail & Related papers  (2022-05-25T07:54:10Z)
• Visual Grounding of Learned Physical Models [66.04898704928517]
  Humans intuitively recognize objects' physical properties and predict their motion, even when the objects are engaged in complicated interactions. We present a neural model that simultaneously reasons about physics and makes future predictions based on visual and dynamics priors. Experiments show that our model can infer the physical properties within a few observations, which allows the model to quickly adapt to unseen scenarios and make accurate predictions into the future.
  arXiv  Detail & Related papers  (2020-04-28T17:06:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.