Do Machines Fail Like Humans? A Human-Centred Out-of-Distribution Spectrum for Mapping Error Alignment

• URL: http://arxiv.org/abs/2603.07462v1
• Date: Sun, 08 Mar 2026 04:51:39 GMT
• Title: Do Machines Fail Like Humans? A Human-Centred Out-of-Distribution Spectrum for Mapping Error Alignment
• Abstract summary: We propose a human-centred framework that redefines the degree of OOD as a spectrum of human perceptual difficulty.<n>We apply this framework to object recognition and reveal unique, regime-dependent model-human alignment rankings and profiles across deep learning architectures.
• Score: 1.974820485832244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Determining whether AI systems process information similarly to humans is central to cognitive science and trustworthy AI. While modern AI models match human accuracy on standard tasks, such parity does not guarantee that their underlying decision-making strategies are aligned with human information processing. Assessing performance using i) error alignment metrics to compare how humans and models fail, and ii) using distorted, or otherwise more challenging, stimuli, provides a viable pathway toward a finer characterization of model-human alignment. However, existing out-of-distribution (OOD) analyses for challenging stimuli are limited due to methodological choices: they define OOD shift relative to model training data or use arbitrary distortion-specific parameters with little correspondence to human perception, hindering principled comparisons. We propose a human-centred framework that redefines the degree of OOD as a spectrum of human perceptual difficulty. By quantifying how much a collection of stimuli deviates from an undistorted reference set based on human accuracy, we construct an OOD spectrum and identify four distinct regimes of perceptual challenge. This approach enables principled model-human comparisons at calibrated difficulty levels. We apply this framework to object recognition and reveal unique, regime-dependent model-human alignment rankings and profiles across deep learning architectures. Vision-language models are the most consistently human aligned across near- and far-OOD conditions, but CNNs are more aligned than ViTs for near-OOD and ViTs are more aligned than CNNs for far-OOD conditions. Our work demonstrates the critical importance of accounting for cross-condition differences such as perceptual difficulty for a principled assessment of model-human alignment.

Related papers

• Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot [18.610144165897264]
  A central question in computational vision is whether human-like visual representations are better explained by discriminative or generative learning.<n>We address this confound using Joint Energy-Based Models (JEMs), which interpolate continuously between discriminative and generative training within a fixed architecture.
  arXiv  Detail & Related papers  (2026-05-22T16:21:25Z)
• A Study of Failure Modes in Two-Stage Human-Object Interaction Detection [49.37675694881915]
  We present a study to better understand the failure modes of two-stage HOI models.<n>We decompose HOI detection into multiple interpretable perspectives to study different types of failure patterns.
  arXiv  Detail & Related papers  (2026-04-15T04:01:23Z)
• LVLM-Aided Alignment of Task-Specific Vision Models [49.96265491629163]
  Small task-specific vision models are crucial in high-stakes domains.<n>We introduce a novel and efficient method for aligning small task-specific vision models with human domain knowledge.<n>Our method demonstrates substantial improvement in aligning model behavior with human specifications.
  arXiv  Detail & Related papers  (2025-12-26T11:11:25Z)
• Accuracy Does Not Guarantee Human-Likeness in Monocular Depth Estimators [2.466518228012258]
  Deep neural networks (DNNs) have achieved superhuman accuracy on physical-based benchmarks.<n>Monocular depth estimation is a fundamental capability for real-world applications such as autonomous driving and robotics.<n>Research in object recognition has revealed a complex trade-off between model accuracy and human-like behavior.
  arXiv  Detail & Related papers  (2025-12-09T01:42:00Z)
• RoHOI: Robustness Benchmark for Human-Object Interaction Detection [84.78366452133514]
  Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support.<n>We introduce the first benchmark for HOI detection, evaluating model resilience under diverse challenges.<n>Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric.
  arXiv  Detail & Related papers  (2025-07-12T01:58:04Z)
• Empirically evaluating commonsense intelligence in large language models with large-scale human judgments [4.212429064310439]
  We propose a method for evaluating common sense in artificial intelligence.<n>We measure the correspondence between a model's judgment and that of a human population.<n>Our framework contributes to the growing call for adapting AI models to human collectivities that possess different, often incompatible, social stocks of knowledge.
  arXiv  Detail & Related papers  (2025-05-15T13:55:27Z)
• Human and AI Perceptual Differences in Image Classification Errors [13.045020949359621]
  This study first analyzes the statistical distributions of mistakes from the two sources and then explores how task difficulty level affects these distributions.<n>We find that even when AI learns an excellent model from the training data, one that outperforms humans in overall accuracy, these AI models have significant and consistent differences from human perception.
  arXiv  Detail & Related papers  (2023-04-18T05:09:07Z)
• Perceptual Attacks of No-Reference Image Quality Models with Human-in-the-Loop [113.75573175709573]
  We make one of the first attempts to examine the perceptual robustness of NR-IQA models. We test one knowledge-driven and three data-driven NR-IQA methods under four full-reference IQA models. We find that all four NR-IQA models are vulnerable to the proposed perceptual attack.
  arXiv  Detail & Related papers  (2022-10-03T13:47:16Z)
• Empirical Estimates on Hand Manipulation are Recoverable: A Step Towards Individualized and Explainable Robotic Support in Everyday Activities [80.37857025201036]
  Key challenge for robotic systems is to figure out the behavior of another agent. Processing correct inferences is especially challenging when (confounding) factors are not controlled experimentally. We propose equipping robots with the necessary tools to conduct observational studies on people.
  arXiv  Detail & Related papers  (2022-01-27T22:15:56Z)
• Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency [10.028543085687803]
  A central problem in cognitive science and behavioural neuroscience is to ascertain whether two or more decision makers (be they brains or algorithms) use the same strategy. We introduce trial-by-trial error consistency, a quantitative analysis for measuring whether two decision making systems systematically make errors on the same inputs.
  arXiv  Detail & Related papers  (2020-06-30T12:47:17Z)
• Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs [90.20235972293801]
  Aiming to understand how human (false-temporal)-belief-a core socio-cognitive ability unify-would affect human interactions with robots, this paper proposes to adopt a graphical model to the representation of object states, robot knowledge, and human (false-)beliefs. An inference algorithm is derived to fuse individual pg from all robots across multi-views into a joint pg, which affords more effective reasoning inference capability to overcome the errors originated from a single view.
  arXiv  Detail & Related papers  (2020-04-25T23:02:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.