On the use of foundation models in cognitive science

• URL: http://arxiv.org/abs/2608.07812v1
• Date: Fri, 07 Aug 2026 23:22:18 GMT
• Title: On the use of foundation models in cognitive science
• Abstract summary: Using Foundation Models as candidate cognitive models poses significant methodological and conceptual challenges.<n>We articulate a four-stage inferential framework for evaluating FMs as cognitive and developmental models.
• Score: 10.941781046160221
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A host of recent studies have evaluated the cognitive and developmental alignment of Foundation Models (FMs). These investigations include evaluations of their correspondence to adult performance across a range of cognitive domains, as well as whether aspects of model training track children's cognitive development. However, using FMs as candidate cognitive models poses significant methodological and conceptual challenges. A key question underlies this effort: under what conditions does behavioral alignment justify treating FMs as explanatory models of cognition? In this paper, we articulate a four-stage inferential framework for evaluating FMs as cognitive and developmental models: adapting human experimental tasks to model-compatible formats, specifying linking hypotheses that map model outputs to human measures, evaluating behavioral correspondence, and comparing across candidate models or manipulations. We clarify the role of linking hypotheses in mapping model outputs to human behavioral measures, identify challenges that constrain alignment claims, and propose principles for theory-driven and comparative evaluation. Throughout, we argue that behavioral fit alone is insufficient. Alignment becomes scientifically meaningful only when embedded within explicit theoretical commitments, theory-diagnostic tasks, and systematic contrastive evaluation across candidate models.

Related papers

• Cognitive Profiling of LRMs' Reasoning Traces Using Bloom's Taxonomy [51.08110837378928]
  We introduce a framework for automatic annotation of reasoning steps through the lens of Bloom's taxonomy.<n>We perform a large-scale analysis across models and datasets, revealing both similarities and differences in thinking patterns across models and tasks.<n>Our findings establish a fine-grained framework for analyzing thinking patterns in LRMs and provide actionable insights for enhancing reasoning quality.
  arXiv  Detail & Related papers  (2026-08-24T12:55:51Z)
• Evaluating Pragmatic Reasoning in Large Language Models: Evidence from Scalar Diversity [2.538209532048867]
  This study compares direct probability measurement and metalinguistic prompting across multiple models and experimental settings.<n>The results show that neither evaluation method consistently outperforms the other and that pragmatic behavior varies substantially across model families, prompting strategies, and task structures.
  arXiv  Detail & Related papers  (2026-05-09T16:28:50Z)
• Taming the Centaur(s) with LAPITHS: a framework for a theoretically grounded interpretation of AI performances [0.0]
  We introduce a framework called LAPITHS (Language model Analysis through Paradigm grounded Interpretations of Theses about Human likenesS)<n>We use it to show that several major claims advanced by models such as CENTAUR, proposed as an artificial Unified Model of Cognition, are not theoretically or empirically justified.
  arXiv  Detail & Related papers  (2026-04-30T14:29:50Z)
• Cognitive Foundations for Reasoning and Their Manifestation in LLMs [63.12951576410617]
  Large language models (LLMs) solve complex problems yet fail on simpler variants, suggesting they achieve correct outputs through mechanisms fundamentally different from human reasoning.<n>We synthesize cognitive science research into a taxonomy of 28 cognitive elements spanning reasoning invariants, meta-cognitive controls, representations for organizing reasoning & knowledge, and transformation operations.<n>We develop test-time reasoning guidance that automatically scaffold successful structures, improving performance by up to 66.7% on complex problems.
  arXiv  Detail & Related papers  (2025-11-20T18:59:00Z)
• Machine Psychophysics: Cognitive Control in Vision-Language Models [0.0]
  We evaluate 108 vision-language models on three classic conflict tasks and their more demanding "squared" variants across 2,220 trials.<n>Results indicate that some form of human-like executive function have emerged in current multi-modal foundational models.
  arXiv  Detail & Related papers  (2025-05-25T04:23:28Z)
• On the Reasoning Capacity of AI Models and How to Quantify It [0.0]
  Large Language Models (LLMs) have intensified the debate surrounding the fundamental nature of their reasoning capabilities.<n>While achieving high performance on benchmarks such as GPQA and MMLU, these models exhibit limitations in more complex reasoning tasks.<n>We propose a novel phenomenological approach that goes beyond traditional accuracy metrics to probe the underlying mechanisms of model behavior.
  arXiv  Detail & Related papers  (2025-01-23T16:58:18Z)
• The potential -- and the pitfalls -- of using pre-trained language models as cognitive science theories [2.6549754445378344]
  We discuss challenges to the use of PLMs as cognitive science theories.<n>We review assumptions used by researchers to map measures of PLM performance to measures of human performance.<n>We end by enumerating criteria for using PLMs as credible accounts of cognition and cognitive development.
  arXiv  Detail & Related papers  (2025-01-22T05:24:23Z)
• Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach [61.04606493712002]
  Susceptibility to misinformation describes the degree of belief in unverifiable claims that is not observable. Existing susceptibility studies heavily rely on self-reported beliefs. We propose a computational approach to model users' latent susceptibility levels.
  arXiv  Detail & Related papers  (2023-11-16T07:22:56Z)
• Rethinking Model Evaluation as Narrowing the Socio-Technical Gap [47.632123167141245]
  We argue that model evaluation practices must take on a critical task to cope with the challenges and responsibilities brought by this homogenization.<n>We urge the community to develop evaluation methods based on real-world contexts and human requirements.
  arXiv  Detail & Related papers  (2023-06-01T00:01:43Z)
• Gender Biases in Automatic Evaluation Metrics for Image Captioning [87.15170977240643]
  We conduct a systematic study of gender biases in model-based evaluation metrics for image captioning tasks. We demonstrate the negative consequences of using these biased metrics, including the inability to differentiate between biased and unbiased generations. We present a simple and effective way to mitigate the metric bias without hurting the correlations with human judgments.
  arXiv  Detail & Related papers  (2023-05-24T04:27:40Z)
• Are Neural Topic Models Broken? [81.15470302729638]
  We study the relationship between automated and human evaluation of topic models. We find that neural topic models fare worse in both respects compared to an established classical method.
  arXiv  Detail & Related papers  (2022-10-28T14:38:50Z)
• Neural Causal Models for Counterfactual Identification and Estimation [62.30444687707919]
  We study the evaluation of counterfactual statements through neural models. First, we show that neural causal models (NCMs) are expressive enough. Second, we develop an algorithm for simultaneously identifying and estimating counterfactual distributions.
  arXiv  Detail & Related papers  (2022-09-30T18:29:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.