From Pixels to Predicates Structuring urban perception with scene graphs

• URL: http://arxiv.org/abs/2512.19221v1
• Date: Mon, 22 Dec 2025 10:02:53 GMT
• Title: From Pixels to Predicates Structuring urban perception with scene graphs
• Abstract summary: This study proposes a three stage pipeline that transforms street view imagery (SVI) into structured representations for predicting six perceptual indicators.<n>In the first stage, each image is parsed using an open-set Panoptic Scene Graph model (OpenPSG) to extract object predicate triplets object.<n>In the second stage, compact scene-level embeddings are learned through a heterogeneous graph autoencoder (GraphMAE)<n>In the third stage, a neural network predicts perception scores from these embeddings.
• Score: 4.7912190748710515
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Perception research is increasingly modelled using streetscapes, yet many approaches still rely on pixel features or object co-occurrence statistics, overlooking the explicit relations that shape human perception. This study proposes a three stage pipeline that transforms street view imagery (SVI) into structured representations for predicting six perceptual indicators. In the first stage, each image is parsed using an open-set Panoptic Scene Graph model (OpenPSG) to extract object predicate object triplets. In the second stage, compact scene-level embeddings are learned through a heterogeneous graph autoencoder (GraphMAE). In the third stage, a neural network predicts perception scores from these embeddings. We evaluate the proposed approach against image-only baselines in terms of accuracy, precision, and cross-city generalization. Results indicate that (i) our approach improves perception prediction accuracy by an average of 26% over baseline models, and (ii) maintains strong generalization performance in cross-city prediction tasks. Additionally, the structured representation clarifies which relational patterns contribute to lower perception scores in urban scenes, such as graffiti on wall and car parked on sidewalk. Overall, this study demonstrates that graph-based structure provides expressive, generalizable, and interpretable signals for modelling urban perception, advancing human-centric and context-aware urban analytics.

Related papers

• Modeling Subjective Urban Perception with Human Gaze [60.73742038237224]
  We introduce Place Pulse-Gaze, an urban perception dataset that augments street view images with synchronized eye-tracking recordings and individual perception labels.<n>We propose a Gaze-Guided Urban Perception Framework to study how gaze behavior contributes to the modeling of subjective urban perception.
  arXiv  Detail & Related papers  (2026-05-01T16:25:13Z)
• A Scene Graph Backed Approach to Open Set Semantic Mapping [1.8266902367595235]
  Open Set Semantic Mapping and 3D Scene Graphs (3DSSGs) are established paradigms in robotic perception.<n>We propose a mapping architecture where the 3DSSG serves as the foundational backend, acting as the primary knowledge representation.<n>This ensures that the map remains topologically consistent and computationally efficient, even during extended operations in large-scale settings.
  arXiv  Detail & Related papers  (2026-02-03T17:41:51Z)
• Statistical Confidence Rescoring for Robust 3D Scene Graph Generation from Multi-View Images [56.134885746889026]
  semantic scene graph estimation methods utilize ground truth 3D annotations to accurately predict target objects, predicates, and relationships.<n>We overcome the noisy reconstructed pseudo point-based geometry from predicted depth maps and reduce the amount of background noise present in multi-view image features.<n>Our method outperforms current methods purely using multi-view images as the initial input.
  arXiv  Detail & Related papers  (2025-08-05T21:25:50Z)
• RadOcc: Learning Cross-Modality Occupancy Knowledge through Rendering Assisted Distillation [50.35403070279804]
  3D occupancy prediction is an emerging task that aims to estimate the occupancy states and semantics of 3D scenes using multi-view images. We propose RadOcc, a Rendering assisted distillation paradigm for 3D Occupancy prediction.
  arXiv  Detail & Related papers  (2023-12-19T03:39:56Z)
• Graph-level Representation Learning with Joint-Embedding Predictive Architectures [43.89120279424267]
  Joint-Embedding Predictive Architectures (JEPAs) have emerged as a novel and powerful technique for self-supervised representation learning.<n>We show that graph-level representations can be effectively modeled using this paradigm by proposing a Graph Joint-Embedding Predictive Architecture (Graph-JEPA)<n>In particular, we employ masked modeling and focus on predicting the latent representations of masked subgraphs starting from the latent representation of a context subgraph.
  arXiv  Detail & Related papers  (2023-09-27T20:42:02Z)
• Incremental 3D Semantic Scene Graph Prediction from RGB Sequences [86.77318031029404]
  We propose a real-time framework that incrementally builds a consistent 3D semantic scene graph of a scene given an RGB image sequence. Our method consists of a novel incremental entity estimation pipeline and a scene graph prediction network. The proposed network estimates 3D semantic scene graphs with iterative message passing using multi-view and geometric features extracted from the scene entities.
  arXiv  Detail & Related papers  (2023-05-04T11:32:16Z)
• Image Aesthetics Assessment Using Graph Attention Network [17.277954886018353]
  We present a two-stage framework based on graph neural networks for image aesthetics assessment. First, we propose a feature-graph representation in which the input image is modelled as a graph, maintaining its original aspect ratio and resolution. Second, we propose a graph neural network architecture that takes this feature-graph and captures the semantic relationship between the different regions of the input image using visual attention.
  arXiv  Detail & Related papers  (2022-06-26T12:52:46Z)
• Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images [79.70127290464514]
  We decompose the task into two stages, i.e. person localization and pose estimation. And we propose three task-specific graph neural networks for effective message passing. Our approach achieves state-of-the-art performance on CMU Panoptic and Shelf datasets.
  arXiv  Detail & Related papers  (2021-09-13T11:44:07Z)
• Exploiting Edge-Oriented Reasoning for 3D Point-based Scene Graph Analysis [21.920148546359016]
  We propose a 3D point-based scene graph generation framework to bridge perception and reasoning. Within the reasoning stage, an EDGE-oriented Graph Convolutional Network is created to exploit multi-dimensional edge features. Our experimental results show promising edge-oriented reasoning effects on scene graph generation studies.
  arXiv  Detail & Related papers  (2021-03-09T17:09:46Z)
• Structure-Augmented Text Representation Learning for Efficient Knowledge Graph Completion [53.31911669146451]
  Human-curated knowledge graphs provide critical supportive information to various natural language processing tasks. These graphs are usually incomplete, urging auto-completion of them. graph embedding approaches, e.g., TransE, learn structured knowledge via representing graph elements into dense embeddings. textual encoding approaches, e.g., KG-BERT, resort to graph triple's text and triple-level contextualized representations.
  arXiv  Detail & Related papers  (2020-04-30T13:50:34Z)
• Structured Landmark Detection via Topology-Adapting Deep Graph Learning [75.20602712947016]
  We present a new topology-adapting deep graph learning approach for accurate anatomical facial and medical landmark detection. The proposed method constructs graph signals leveraging both local image features and global shape features. Experiments are conducted on three public facial image datasets (WFLW, 300W, and COFW-68) as well as three real-world X-ray medical datasets (Cephalometric (public), Hand and Pelvis)
  arXiv  Detail & Related papers  (2020-04-17T11:55:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.