Organization of Valence and Arousal in Vision-Language Representations of Built Environments: Insights from the EMOIS Dataset

• URL: http://arxiv.org/abs/2609.06870v1
• Date: Sun, 06 Sep 2026 23:12:30 GMT
• Title: Organization of Valence and Arousal in Vision-Language Representations of Built Environments: Insights from the EMOIS Dataset
• Abstract summary: We introduce the Emotional Impression of Spaces dataset, comprising 1,544 real-world built-environment images.<n>Using Contrastive Language--Image Pre-training (CLIP) representations, we perform predictive and geometric analyses.<n>We show how learned representations can support qualitative interpretation of predicted affective values.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual perception of built environments contributes to the affective impressions that people form in everyday life. However, how these impressions are represented within vision foundation models remains largely unexplored. To support the systematic investigation of this subject, we introduce the Emotional Impression of Spaces (EMOIS) dataset, comprising 1,544 real-world built-environment images. Each image is annotated with image-evoked valence and arousal ratings collected from Japanese adults by conducting a large-scale web-based survey, with approximately 120 ratings per image. Using Contrastive Language--Image Pre-training (CLIP) representations, we perform predictive and geometric analyses to systematically investigate how valence and arousal are encoded and organized within the representation space. These analyses reveal that valence exhibited stronger and more coherent organization than arousal. Cross-dataset analyses with the Open Affective Standardized Image Set (OASIS), a benchmark dataset of general affective photographs, reveal differences in affective organization between the two datasets. Regression analyses demonstrate high predictive performance for valence and arousal within EMOIS, with mean coefficients of determination of 0.865 and 0.807, respectively, across repeated internal hold-out evaluations. Finally, we present an example-based interface illustrating how learned representations can support qualitative interpretation of predicted affective values. These findings can help elucidate affective representations of built environments and establish EMOIS as a densely annotated resource for future affective computing research in this domain.

Related papers

• Beyond Semantics: Modeling Factual and Affective Perceptual Experiences from Vision-Language Data [37.91467682358732]
  We introduce **PercepT** (**Percep**tion topic **T**ransformer), a two-stage architecture that tackles P-Topics modeling.<n>PercepT captures semantically meaningful perception experiences and significantly outperforms existing methods.
  arXiv  Detail & Related papers  (2026-06-02T08:54:59Z)
• Joint Imaging-ROI Representation Learning via Cross-View Contrastive Alignment for Brain Disorder Classification [13.749675574071873]
  We propose a unified cross-view contrastive framework for joint imaging-ROI representation learning.<n>Our method learns subject-level global (imaging) and local (ROI-graph) embeddings and aligns them in a shared latent space.<n>Experiments on the ADHD-200 and ABIDE datasets demonstrate that joint learning consistently improves classification performance.
  arXiv  Detail & Related papers  (2026-03-10T22:14:53Z)
• Visual Affect Analysis: Predicting Emotions of Image Viewers with Vision-Language Models [2.2023261946811563]
  Vision-language models (VLMs) show promise as tools for inferring affect from visual stimuli at scale.<n>We benchmarked nine VLMs, ranging from state-of-the-art proprietary models to open-source models, on three psycho-metrically validated affective image datasets.
  arXiv  Detail & Related papers  (2026-01-27T17:56:21Z)
• SITE: towards Spatial Intelligence Thorough Evaluation [121.1493852562597]
  Spatial intelligence (SI) represents a cognitive ability encompassing the visualization, manipulation, and reasoning about spatial relationships.<n>We introduce SITE, a benchmark dataset towards SI Thorough Evaluation.<n>Our approach to curating the benchmark combines a bottom-up survey about 31 existing datasets and a top-down strategy drawing upon three classification systems in cognitive science.
  arXiv  Detail & Related papers  (2025-05-08T17:45:44Z)
• Generalized Visual Relation Detection with Diffusion Models [94.62313788626128]
  Visual relation detection (VRD) aims to identify relationships (or interactions) between object pairs in an image.<n>We propose to model visual relations as continuous embeddings, and design diffusion models to achieve generalized VRD in a conditional generative manner.<n>Our Diff-VRD is able to generate visual relations beyond the pre-defined category labels of datasets.
  arXiv  Detail & Related papers  (2025-04-16T14:03:24Z)
• Balancing the Scales: Enhancing Fairness in Facial Expression Recognition with Latent Alignment [5.784550537553534]
  This workleverages representation learning based on latent spaces to mitigate bias in facial expression recognition systems. It also enhances a deep learning model's fairness and overall accuracy.
  arXiv  Detail & Related papers  (2024-10-25T10:03:10Z)
• When Does Perceptual Alignment Benefit Vision Representations? [76.32336818860965]
  We investigate how aligning vision model representations to human perceptual judgments impacts their usability. We find that aligning models to perceptual judgments yields representations that improve upon the original backbones across many downstream tasks. Our results suggest that injecting an inductive bias about human perceptual knowledge into vision models can contribute to better representations.
  arXiv  Detail & Related papers  (2024-10-14T17:59:58Z)
• Exploring Affordance and Situated Meaning in Image Captions: A Multimodal Analysis [1.124958340749622]
  We annotate images from the Flickr30k dataset with five perceptual properties: Affordance, Perceptual Salience, Object Number, Cue Gazeing, and Ecological Niche Association (ENA) Our findings reveal that images with Gibsonian affordance show a higher frequency of captions containing 'holding-verbs' and 'container-nouns' compared to images displaying telic affordance.
  arXiv  Detail & Related papers  (2023-05-24T01:30:50Z)
• Perceptual Grouping in Contrastive Vision-Language Models [59.1542019031645]
  We show how vision-language models are able to understand where objects reside within an image and group together visually related parts of the imagery. We propose a minimal set of modifications that results in models that uniquely learn both semantic and spatial information.
  arXiv  Detail & Related papers  (2022-10-18T17:01:35Z)
• DORA: Exploring Outlier Representations in Deep Neural Networks [0.0]
  We present DORA, the first data-agnostic framework for analyzing the representational space of Deep Neural Networks (DNNs) Central to our framework is the proposed Extreme-Activation (EA) distance measure, which assesses similarities between representations. We validate the EA metric quantitatively, demonstrating its effectiveness both in controlled scenarios and real-world applications.
  arXiv  Detail & Related papers  (2022-06-09T14:25:14Z)
• Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework [83.21732533130846]
  The paper focuses on large in-the-wild databases, i.e., Aff-Wild and Aff-Wild2. It presents the design of two classes of deep neural networks trained with these databases. A novel multi-task and holistic framework is presented which is able to jointly learn and effectively generalize and perform affect recognition.
  arXiv  Detail & Related papers  (2021-03-29T17:36:20Z)
• Deep Co-Attention Network for Multi-View Subspace Learning [73.3450258002607]
  We propose a deep co-attention network for multi-view subspace learning. It aims to extract both the common information and the complementary information in an adversarial setting. In particular, it uses a novel cross reconstruction loss and leverages the label information to guide the construction of the latent representation.
  arXiv  Detail & Related papers  (2021-02-15T18:46:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.