FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning

• URL: http://arxiv.org/abs/2604.03696v1
• Date: Sat, 04 Apr 2026 11:53:12 GMT
• Title: FunFact: Building Probabilistic Functional 3D Scene Graphs via Factor-Graph Reasoning
• Abstract summary: We introduce FunFact, a framework for constructing probabilistic open-vocabulary functional 3D scene graphs.<n>FunFact first builds an object- and part-centric 3D map and uses foundation models to propose semantically plausible functional relations.<n>To benchmark this setting, we introduce FunThor, a synthetic dataset based on AI2-THOR with part-level geometry and rule-based functional annotations.
• Score: 48.3789545323541
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent work in 3D scene understanding is moving beyond purely spatial analysis toward functional scene understanding. However, existing methods often consider functional relationships between object pairs in isolation, failing to capture the scene-wide interdependence that humans use to resolve ambiguity. We introduce FunFact, a framework for constructing probabilistic open-vocabulary functional 3D scene graphs from posed RGB-D images. FunFact first builds an object- and part-centric 3D map and uses foundation models to propose semantically plausible functional relations. These candidates are converted into factor graph variables and constrained by both LLM-derived common-sense priors and geometric priors. This formulation enables joint probabilistic inference over all functional edges and their marginals, yielding substantially better calibrated confidence scores. To benchmark this setting, we introduce FunThor, a synthetic dataset based on AI2-THOR with part-level geometry and rule-based functional annotations. Experiments on SceneFun3D, FunGraph3D, and FunThor show that FunFact improves node and relation discovery recall and significantly reduces calibration error for ambiguous relations, highlighting the benefits of holistic probabilistic modeling for functional scene understanding. See our project page at https://funfact-scenegraph.github.io/

Related papers

• Functional-SLAM: Interaction-Aware Mapping with Online Functional Scene Graphs [52.091507529866725]
  Functional-SLAM is a framework that maintains a functional scene graph as an online SLAM state.<n>It efficiently constructs stable functional maps online, substantially improving runtime over offline methods.<n>Compared with peer SLAM systems, it further improves pose estimation accuracy through functional-topology-assisted loop closure.
  arXiv  Detail & Related papers  (2026-09-07T13:46:56Z)
• Functionalization via Structure Completion and Motion Rectification [44.37223222305385]
  We introduce object functionalization, a novel task aimed at transforming visually plausible but non-functional 3D models into functional and physically operable ones.<n>We develop a neural Graph Functionalizer (GraFu) to complete an incomplete graph representing a non-functional 3D object.<n>The completed graph then drives a geometry realization stage that instantiates predicted connectors and structural elements in 3D.
  arXiv  Detail & Related papers  (2026-05-18T08:05:07Z)
• Hierarchical and Holistic Open-Vocabulary Functional 3D Scene Graphs for Indoor Spaces [116.57196064763924]
  We extend the benchmark coverage by introducing dense tabletop objects and explicit multi-level functional relationships.<n>This work introduces critical challenges involving small-scale, dense, and similar instances.<n>We propose an open-vocabulary pipeline based on 2D visual grounding and 3D graph optimization.
  arXiv  Detail & Related papers  (2026-05-15T09:14:50Z)
• ReLaGS: Relational Language Gaussian Splatting [20.136674901612334]
  We present a novel framework that constructs a hierarchical language-distilled Gaussian scene and its 3D semantic scene graph without scene-specific training.<n>On top of this hierarchy, we build an open-vocabulary 3D scene graph with Vision Language derived annotations and Graph Neural Network-based relational reasoning.<n>Our approach enables efficient and scalable open-vocabulary 3D reasoning by jointly modeling hierarchical semantics and inter/intra-object relationships.
  arXiv  Detail & Related papers  (2026-03-18T11:18:23Z)
• ArtiSG: Functional 3D Scene Graph Construction via Human-demonstrated Articulated Objects Manipulation [51.54082859171464]
  ArtiSG is a framework that constructs functional 3D scene graphs by encoding human demonstrations into structured robotic memory.<n>We show that ArtiSG significantly outperforms baselines in functional element recall and articulation estimation precision.
  arXiv  Detail & Related papers  (2025-12-31T13:10:40Z)
• Language-guided 3D scene synthesis for fine-grained functionality understanding [64.148891566272]
  We introduce SynthFun3D, the first method for task-based 3D scene synthesis.<n>It generates a 3D indoor environment using a furniture asset database with part-level annotation.<n>It reasons about the action to automatically identify and retrieve the 3D mask of the correct functional element.
  arXiv  Detail & Related papers  (2025-11-28T14:40:03Z)
• Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces [113.91791599146786]
  We introduce the task of predicting functional 3D scene graphs for real-world indoor environments from posed RGB-D images.<n>Unlike traditional 3D scene graphs that focus on spatial relationships of objects, functional 3D scene graphs capture objects, interactive elements, and their functional relationships.<n>We evaluate our approach on an extended SceneFun3D dataset and a newly collected dataset, FunGraph3D, both annotated with functional 3D scene graphs.
  arXiv  Detail & Related papers  (2025-03-24T22:53:19Z)
• FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction [1.8124328823188356]
  We aim to develop a representation that enables robots to directly interact with their environment.<n>We focus on detecting and storing objects at a finer resolution, focusing on affordance-relevant parts.<n>We leverage currently available 3D resources to generate 2D data and train a detector, which is then used to augment the standard 3D scene graph generation pipeline.
  arXiv  Detail & Related papers  (2025-03-10T23:13:35Z)
• 3DGraphLLM: Combining Semantic Graphs and Large Language Models for 3D Scene Understanding [0.5755004576310334]
  A 3D scene graph represents a compact scene model by capturing both the objects present and the semantic relationships between them.<n>We propose 3DGraphLLM, a method for constructing a learnable representation of a 3D scene graph that explicitly incorporates semantic relationships.
  arXiv  Detail & Related papers  (2024-12-24T14:21:58Z)
• GREAT: Geometry-Intention Collaborative Inference for Open-Vocabulary 3D Object Affordance Grounding [53.42728468191711]
  Open-Vocabulary 3D object affordance grounding aims to anticipate action possibilities'' regions on 3D objects with arbitrary instructions.<n>We propose GREAT (GeometRy-intEntion collAboraTive inference) for Open-Vocabulary 3D Object Affordance Grounding.
  arXiv  Detail & Related papers  (2024-11-29T11:23:15Z)
• Open-Vocabulary Octree-Graph for 3D Scene Understanding [54.11828083068082]
  Octree-Graph is a novel scene representation for open-vocabulary 3D scene understanding. An adaptive-octree structure is developed that stores semantics and depicts the occupancy of an object adjustably according to its shape.
  arXiv  Detail & Related papers  (2024-11-25T10:14:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.