Unlocking the power of partnership: How humans and machines can work together to improve face recognition

• URL: http://arxiv.org/abs/2510.02570v1
• Date: Thu, 02 Oct 2025 21:19:56 GMT
• Title: Unlocking the power of partnership: How humans and machines can work together to improve face recognition
• Authors: P. Jonathon Phillips, Geraldine Jeckeln, Carina A. Hahn, Amy N. Yates, Peter C. Fontana, Alice J. O'Toole,
• Abstract summary: We examined the benefits of human-human and human-machine collaborations.<n>We implemented "intelligent human-machine fusion" by selecting people with the potential to increase the accuracy of a high-performing machine.<n>The highest system-wide accuracy achievable with human-only partnerships was found by graph theory.
• Score: 0.6157382820537719
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Human review of consequential decisions by face recognition algorithms creates a "collaborative" human-machine system. Individual differences between people and machines, however, affect whether collaboration improves or degrades accuracy in any given case. We establish the circumstances under which combining human and machine face identification decisions improves accuracy. Using data from expert and non-expert face identifiers, we examined the benefits of human-human and human-machine collaborations. The benefits of collaboration increased as the difference in baseline accuracy between collaborators decreased-following the Proximal Accuracy Rule (PAR). This rule predicted collaborative (fusion) benefit across a wide range of baseline abilities, from people with no training to those with extensive training. Using the PAR, we established a critical fusion zone, where humans are less accurate than the machine, but fusing the two improves system accuracy. This zone was surprisingly large. We implemented "intelligent human-machine fusion" by selecting people with the potential to increase the accuracy of a high-performing machine. Intelligent fusion was more accurate than the machine operating alone and more accurate than combining all human and machine judgments. The highest system-wide accuracy achievable with human-only partnerships was found by graph theory. This fully human system approximated the average performance achieved by intelligent human-machine collaboration. However, intelligent human-machine collaboration more effectively minimized the impact of low-performing humans on system-wide accuracy. The results demonstrate a meaningful role for both humans and machines in assuring accurate face identification. This study offers an evidence-based road map for the intelligent use of AI in face identification.

Related papers

• A Comparison of Human and Machine Learning Errors in Face Recognition [8.064574155970963]
  We perform experiments in the area of face recognition and compare two automated face recognition systems against human annotators.<n>Our research uncovers important ways in which machine learning errors and human errors differ from each other, and suggests potential strategies in which human-machine collaboration can improve accuracy in face recognition.
  arXiv  Detail & Related papers  (2025-02-17T01:27:35Z)
• Human adaptation to adaptive machines converges to game-theoretic equilibria [13.234975857626752]
  We show how game theory can be used to predict and design outcomes of co-adaptive interactions between intelligent humans and machines. One algorithm steers the human-machine interaction to the machine's optimum, effectively controlling the human's actions.
  arXiv  Detail & Related papers  (2023-05-01T23:35:51Z)
• Human and AI Perceptual Differences in Image Classification Errors [13.045020949359621]
  This study first analyzes the statistical distributions of mistakes from the two sources and then explores how task difficulty level affects these distributions.<n>We find that even when AI learns an excellent model from the training data, one that outperforms humans in overall accuracy, these AI models have significant and consistent differences from human perception.
  arXiv  Detail & Related papers  (2023-04-18T05:09:07Z)
• Human-Algorithm Collaboration: Achieving Complementarity and Avoiding Unfairness [92.26039686430204]
  We show that even in carefully-designed systems, complementary performance can be elusive. First, we provide a theoretical framework for modeling simple human-algorithm systems. Next, we use this model to prove conditions where complementarity is impossible.
  arXiv  Detail & Related papers  (2022-02-17T18:44:41Z)
• Show Me What You Can Do: Capability Calibration on Reachable Workspace for Human-Robot Collaboration [83.4081612443128]
  We show that a short calibration using REMP can effectively bridge the gap between what a non-expert user thinks a robot can reach and the ground-truth. We show that this calibration procedure not only results in better user perception, but also promotes more efficient human-robot collaborations.
  arXiv  Detail & Related papers  (2021-03-06T09:14:30Z)
• Joint Mind Modeling for Explanation Generation in Complex Human-Robot Collaborative Tasks [83.37025218216888]
  We propose a novel explainable AI (XAI) framework for achieving human-like communication in human-robot collaborations. The robot builds a hierarchical mind model of the human user and generates explanations of its own mind as a form of communications. Results show that the generated explanations of our approach significantly improves the collaboration performance and user perception of the robot.
  arXiv  Detail & Related papers  (2020-07-24T23:35:03Z)
• Learning to Complement Humans [67.38348247794949]
  A rising vision for AI in the open world centers on the development of systems that can complement humans for perceptual, diagnostic, and reasoning tasks. We demonstrate how an end-to-end learning strategy can be harnessed to optimize the combined performance of human-machine teams.
  arXiv  Detail & Related papers  (2020-05-01T20:00:23Z)
• Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork [54.309495231017344]
  We argue that AI systems should be trained in a human-centered manner, directly optimized for team performance. We study this proposal for a specific type of human-AI teaming, where the human overseer chooses to either accept the AI recommendation or solve the task themselves. Our experiments with linear and non-linear models on real-world, high-stakes datasets show that the most accuracy AI may not lead to highest team performance.
  arXiv  Detail & Related papers  (2020-04-27T19:06:28Z)
• Joint Inference of States, Robot Knowledge, and Human (False-)Beliefs [90.20235972293801]
  Aiming to understand how human (false-temporal)-belief-a core socio-cognitive ability unify-would affect human interactions with robots, this paper proposes to adopt a graphical model to the representation of object states, robot knowledge, and human (false-)beliefs. An inference algorithm is derived to fuse individual pg from all robots across multi-views into a joint pg, which affords more effective reasoning inference capability to overcome the errors originated from a single view.
  arXiv  Detail & Related papers  (2020-04-25T23:02:04Z)
• How human judgment impairs automated deception detection performance [0.5660207256468972]
  We tested whether a combination of supervised machine learning and human judgment could improve deception detection accuracy. Human involvement through hybrid-overrule decisions brought the accuracy back to the chance level. The decision-making strategies of humans suggest that the truth bias - the tendency to assume the other is telling the truth - could explain the detrimental effect.
  arXiv  Detail & Related papers  (2020-03-30T10:06:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.