Bringing Back Rule Induction to Fluid Intelligence Research? An Initial Validation of the ARC-AGI Benchmark in Humans

• URL: http://arxiv.org/abs/2607.11263v3
• Date: Wed, 15 Jul 2026 12:09:51 GMT
• Title: Bringing Back Rule Induction to Fluid Intelligence Research? An Initial Validation of the ARC-AGI Benchmark in Humans
• Abstract summary: Two competing perspectives on fluid intelligence (gf) measures propose that performance is constrained by working memory capacity or by the ability to induce novel relations.<n>The ARC-AGI benchmark predominantly requires rule induction and was proposed as a measure of gf for both humans and artificial systems.<n>We investigated the psychometric characteristics and nomological network of ARC-AGI in a first study with 100 participants.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Two competing perspectives on fluid intelligence (gf) measures propose that performance is primarily constrained either by working memory capacity or by the ability to induce novel relations. The first perspective is currently dominant in measurement, as evident from the use of a limited set of recurring rules, whereas the second perspective is reflected in many definitions but rarely present in measurement. The ARC-AGI benchmark predominantly requires rule induction and was proposed as a measure of gf for both humans and artificial systems. However, its psychometric properties have not yet been examined in human samples. We therefore investigated the psychometric characteristics and nomological network of ARC-AGI in a first study with 100 participants. A compilation of ARC-AGI items showed good psychometric properties and correlated substantially with figural fluid intelligence as measured by a figural reasoning test ($ρ$ = .63). Associations with figural originality were weak. These findings provide initial support for the validity of ARC-AGI as a measure of human fluid intelligence. Future research should include more rule induction tasks as well as additional multivariate covariates. This study is unusual by studying a task in humans that was initially designed for machines. We suggest systematically embedding AI benchmarks into the nomological network of human cognitive abilities to enable more systematic evaluation and interdisciplinary cooperation.

Related papers

• Uneven Evolution of Cognition Across Generations of Generative AI Models [13.772329509422201]
  The pursuit of artificial general intelligence requires robust methods for evaluating the cognitive capabilities of models beyond narrow task performance.<n>Here, we introduce a psychometric framework to assess the cognitive profiles of generative AI, comparing them to human norms and tracking their evolution across generations.
  arXiv  Detail & Related papers  (2026-05-07T18:16:49Z)
• Probing the "Psyche'' of Large Reasoning Models: Understanding Through a Human Lens [28.277723085755806]
  Large reasoning models (LRMs) have garnered significant attention from researchers owing to their exceptional capability in addressing complex tasks.<n>This paper introduces a comprehensive taxonomy to characterize atomic reasoning steps and probe the psyche'' of LRM intelligence.<n>It comprises five groups and seventeen categories derived from human mental processes, thereby grounding the understanding of LRMs in an interdisciplinary perspective.
  arXiv  Detail & Related papers  (2025-11-30T04:49:44Z)
• A Definition of AGI [208.25193480759026]
  The lack of a concrete definition for Artificial General Intelligence obscures the gap between today's specialized AI and human-level cognition.<n>This paper introduces a quantifiable framework to address this, defining AGI as matching the cognitive versatility and proficiency of a well-educated adult.
  arXiv  Detail & Related papers  (2025-10-21T01:28:35Z)
• The next question after Turing's question: Introducing the Grow-AI test [51.56484100374058]
  This study aims to extend the framework for assessing artificial intelligence, called GROW-AI.<n>GROW-AI is designed to answer the question "Can machines grow up?" -- a natural successor to the Turing Test.<n>The originality of the work lies in the conceptual transposition of the process of "growing" from the human world to that of artificial intelligence.
  arXiv  Detail & Related papers  (2025-08-22T10:19:42Z)
• Stop Evaluating AI with Human Tests, Develop Principled, AI-specific Tests instead [2.809966405091883]
  We argue that interpreting benchmark performance as measurements of human-like traits, lacks sufficient theoretical and empirical justification.<n>We call for the development of principled, AI-specific evaluation frameworks tailored to AI systems.
  arXiv  Detail & Related papers  (2025-07-30T18:14:35Z)
• ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems [0.03431023404301193]
  ARC-AGI-2 preserves the input-output pair task format of its predecessor, ensuring continuity for researchers.<n>It incorporates a newly curated and expanded set of tasks specifically designed to assess abstract reasoning and problem-solving abilities.<n> ARC-AGI-2 aims to serve as a next-generation tool for rigorously measuring progress towards more general and human-like AI capabilities.
  arXiv  Detail & Related papers  (2025-05-17T04:34:48Z)
• Evaluating General-Purpose AI with Psychometrics [43.85432514910491]
  We discuss the need for a comprehensive and accurate evaluation of general-purpose AI systems such as large language models. Current evaluation methodology, mostly based on benchmarks of specific tasks, falls short of adequately assessing these versatile AI systems. To tackle these challenges, we suggest transitioning from task-oriented evaluation to construct-oriented evaluation.
  arXiv  Detail & Related papers  (2023-10-25T05:38:38Z)
• Self-Emotion-Mediated Exploration in Artificial Intelligence Mirrors: Findings from Cognitive Psychology [0.08739101659113153]
  This study proposes a learning framework for artificial agents to obtain an intrinsic exploratory drive.<n>Data analysis scores dictate pride or surprise, in accordance with psychological studies on humans.<n>Results: Causal relationships between states and exploration are demonstrated by the majority of agents.
  arXiv  Detail & Related papers  (2023-02-13T18:20:44Z)
• ACP++: Action Co-occurrence Priors for Human-Object Interaction Detection [102.9428507180728]
  A common problem in the task of human-object interaction (HOI) detection is that numerous HOI classes have only a small number of labeled examples. We observe that there exist natural correlations and anti-correlations among human-object interactions. We present techniques to learn these priors and leverage them for more effective training, especially on rare classes.
  arXiv  Detail & Related papers  (2021-09-09T06:02:50Z)
• ACRE: Abstract Causal REasoning Beyond Covariation [90.99059920286484]
  We introduce the Abstract Causal REasoning dataset for systematic evaluation of current vision systems in causal induction. Motivated by the stream of research on causal discovery in Blicket experiments, we query a visual reasoning system with the following four types of questions in either an independent scenario or an interventional scenario. We notice that pure neural models tend towards an associative strategy under their chance-level performance, whereas neuro-symbolic combinations struggle in backward-blocking reasoning.
  arXiv  Detail & Related papers  (2021-03-26T02:42:38Z)
• Detecting Human-Object Interactions with Action Co-occurrence Priors [108.31956827512376]
  A common problem in human-object interaction (HOI) detection task is that numerous HOI classes have only a small number of labeled examples. We observe that there exist natural correlations and anti-correlations among human-object interactions. We present techniques to learn these priors and leverage them for more effective training, especially in rare classes.
  arXiv  Detail & Related papers  (2020-07-17T02:47:45Z)
• Machine Common Sense [77.34726150561087]
  Machine common sense remains a broad, potentially unbounded problem in artificial intelligence (AI) This article deals with the aspects of modeling commonsense reasoning focusing on such domain as interpersonal interactions.
  arXiv  Detail & Related papers  (2020-06-15T13:59:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.