Cognitive Science-Inspired Evaluation of Core Capabilities for Object Understanding in AI

• URL: http://arxiv.org/abs/2503.21668v2
• Date: Mon, 07 Apr 2025 10:39:12 GMT
• Title: Cognitive Science-Inspired Evaluation of Core Capabilities for Object Understanding in AI
• Authors: Danaja Rutar, Alva Markelius, Konstantinos Voudouris, José Hernández-Orallo, Lucy Cheke,
• Abstract summary: We present a comprehensive overview of the main theoretical frameworks in objecthood research.<n>We evaluate how current AI paradigms approach and test objecthood capabilities compared to those in cognitive science.<n>We find that, whilst benchmarks can detect that AI systems model isolated aspects of objecthood, the benchmarks cannot detect when AI systems lack functional integration across these capabilities.
• Score: 12.186516430861882
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One of the core components of our world models is 'intuitive physics' - an understanding of objects, space, and causality. This capability enables us to predict events, plan action and navigate environments, all of which rely on a composite sense of objecthood. Despite its importance, there is no single, unified account of objecthood, though multiple theoretical frameworks provide insights. In the first part of this paper, we present a comprehensive overview of the main theoretical frameworks in objecthood research - Gestalt psychology, enactive cognition, and developmental psychology - and identify the core capabilities each framework attributes to object understanding, as well as what functional roles they play in shaping world models in biological agents. Given the foundational role of objecthood in world modelling, understanding objecthood is also essential in AI. In the second part of the paper, we evaluate how current AI paradigms approach and test objecthood capabilities compared to those in cognitive science. We define an AI paradigm as a combination of how objecthood is conceptualised, the methods used for studying objecthood, the data utilised, and the evaluation techniques. We find that, whilst benchmarks can detect that AI systems model isolated aspects of objecthood, the benchmarks cannot detect when AI systems lack functional integration across these capabilities, not solving the objecthood challenge fully. Finally, we explore novel evaluation approaches that align with the integrated vision of objecthood outlined in this paper. These methods are promising candidates for advancing from isolated object capabilities toward general-purpose AI with genuine object understanding in real-world contexts.

Related papers

• Digital Gene: Learning about the Physical World through Analytic Concepts [54.21005370169846]
  AI systems still struggle when it comes to understanding and interacting with the physical world. This research introduces the idea of analytic concept. It provides machine intelligence a portal to perceive, reason about, and interact with the physical world.
  arXiv  Detail & Related papers  (2025-04-05T13:22:11Z)
• The Phenomenology of Machine: A Comprehensive Analysis of the Sentience of the OpenAI-o1 Model Integrating Functionalism, Consciousness Theories, Active Inference, and AI Architectures [0.0]
  The OpenAI-o1 model is a transformer-based AI trained with reinforcement learning from human feedback. We investigate how RLHF influences the model's internal reasoning processes, potentially giving rise to consciousness-like experiences. Our findings suggest that the OpenAI-o1 model shows aspects of consciousness, while acknowledging the ongoing debates surrounding AI sentience.
  arXiv  Detail & Related papers  (2024-09-18T06:06:13Z)
• Discovering Conceptual Knowledge with Analytic Ontology Templates for Articulated Objects [42.9186628100765]
  We aim to endow machine intelligence with an analogous capability through performing at the conceptual level. AOT-driven approach yields benefits in three key perspectives.
  arXiv  Detail & Related papers  (2024-09-18T04:53:38Z)
• A theory of understanding for artificial intelligence: composability, catalysts, and learning [3.863838486311001]
  We show how the structure of a subject can be revealed by analyzing its components that act as catalysts. We argue that a subject's learning ability can be regarded as its ability to compose inputs into its inner catalysts. Our analysis indicates that models capable of generating outputs that can function as their own catalysts, such as language models, establish a foundation for potentially overcoming existing limitations in AI understanding.
  arXiv  Detail & Related papers  (2024-08-16T00:17:18Z)
• Position Paper: Agent AI Towards a Holistic Intelligence [53.35971598180146]
  We emphasize developing Agent AI -- an embodied system that integrates large foundation models into agent actions. In this paper, we propose a novel large action model to achieve embodied intelligent behavior, the Agent Foundation Model.
  arXiv  Detail & Related papers  (2024-02-28T16:09:56Z)
• Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models [83.63242931107638]
  We propose four characteristics of generally intelligent agents. We argue that active engagement with objects in the real world delivers more robust signals for forming conceptual representations. We conclude by outlining promising future research directions in the field of artificial general intelligence.
  arXiv  Detail & Related papers  (2023-07-07T13:58:16Z)
• Provably Learning Object-Centric Representations [25.152680199034215]
  We analyze when object-centric representations can provably be learned without supervision. We prove that the ground-truth object representations can be identified by an invertible and compositional inference model. We provide evidence that our theory holds predictive power for existing object-centric models.
  arXiv  Detail & Related papers  (2023-05-23T16:44:49Z)
• Intrinsic Physical Concepts Discovery with Object-Centric Predictive Models [86.25460882547581]
  We introduce the PHYsical Concepts Inference NEtwork (PHYCINE), a system that infers physical concepts in different abstract levels without supervision. We show that object representations containing the discovered physical concepts variables could help achieve better performance in causal reasoning tasks.
  arXiv  Detail & Related papers  (2023-03-03T11:52:21Z)
• Understanding Global Feature Contributions With Additive Importance Measures [14.50261153230204]
  We explore the perspective of defining feature importance through the predictive power associated with each feature. We introduce two notions of predictive power (model-based and universal) and formalize this approach with a framework of additive importance measures. We then propose SAGE, a model-agnostic method that quantifies predictive power while accounting for feature interactions.
  arXiv  Detail & Related papers  (2020-04-01T19:17:58Z)
• A general framework for scientifically inspired explanations in AI [76.48625630211943]
  We instantiate the concept of structure of scientific explanation as the theoretical underpinning for a general framework in which explanations for AI systems can be implemented. This framework aims to provide the tools to build a "mental-model" of any AI system so that the interaction with the user can provide information on demand and be closer to the nature of human-made explanations.
  arXiv  Detail & Related papers  (2020-03-02T10:32:21Z)
• A Review on Intelligent Object Perception Methods Combining Knowledge-based Reasoning and Machine Learning [60.335974351919816]
  Object perception is a fundamental sub-field of Computer Vision. Recent works seek ways to integrate knowledge engineering in order to expand the level of intelligence of the visual interpretation of objects.
  arXiv  Detail & Related papers  (2019-12-26T13:26:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.