PLACO: A Multi-Stage Framework for Cost-Effective Performance in Human-AI Teams

• URL: http://arxiv.org/abs/2605.08388v1
• Date: Fri, 08 May 2026 18:54:46 GMT
• Title: PLACO: A Multi-Stage Framework for Cost-Effective Performance in Human-AI Teams
• Abstract summary: Humans have found that using AI assistance has led to an accelerated work pace like never before.<n>In classification tasks, where the final output is a single hard label, it is crucial to address the combination of human and model output.
• Score: 7.634717272554101
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Human-AI teams play a pivotal role in improving overall system performance when neither the human nor the model can achieve such performance on their own. With the advent of powerful and accessible Generative AI models, several mundane tasks have morphed into Human-AI team tasks. From writing essays to developing advanced algorithms, humans have found that using AI assistance has led to an accelerated work pace like never before. In classification tasks, where the final output is a single hard label, it is crucial to address the combination of human and model output. Prior work elegantly solves this problem using Bayes rule, using the assumption that human and model output are conditionally independent given the ground truth. Specifically, it discusses a combination method to combine a single deterministic labeler (the human) and a probabilistic labeler (the classifier model) using the model's instance-level and the human's class-level calibrated probabilities.

Related papers

• $τ_0$-VLA: a Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation [51.05516457377339]
  Long-horizon robot manipulation requires a robot to both execute individual skills reliably and sequence them coherently over extended tasks.<n>We introduce $_0$-VLA, a hierarchical robot foundation model that formulates high-level subtask generation as a compute-scalable inference problem.<n>A low-level policy then executes the generated subtask across multiple robot embodiments.
  arXiv  Detail & Related papers  (2026-08-17T17:59:11Z)
• Modeling AI-Human Collaboration as a Multi-Agent Adaptation [0.0]
  We develop an agent-based simulation to formalize AI-human collaboration as a function of a task.<n>We show that in modular tasks, AI often substitutes for humans - delivering higher payoffs unless human expertise is very high.<n>We also show that even "hallucinatory" AI - lacking memory or structure - can improve outcomes when augmenting low-capability humans by helping escape local optima.
  arXiv  Detail & Related papers  (2025-04-29T16:19:53Z)
• Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision [84.31474052176343]
  Recent AI-assistant agents, such as ChatGPT, rely on supervised fine-tuning (SFT) with human annotations and reinforcement learning from human feedback to align the output with human intentions. This dependence can significantly constrain the true potential of AI-assistant agents due to the high cost of obtaining human supervision. We propose a novel approach called SELF-ALIGN, which combines principle-driven reasoning and the generative power of LLMs for the self-alignment of AI agents with minimal human supervision.
  arXiv  Detail & Related papers  (2023-05-04T17:59:28Z)
• Human and AI Perceptual Differences in Image Classification Errors [13.045020949359621]
  This study first analyzes the statistical distributions of mistakes from the two sources and then explores how task difficulty level affects these distributions.<n>We find that even when AI learns an excellent model from the training data, one that outperforms humans in overall accuracy, these AI models have significant and consistent differences from human perception.
  arXiv  Detail & Related papers  (2023-04-18T05:09:07Z)
• Model Predictive Control for Fluid Human-to-Robot Handovers [50.72520769938633]
  Planning motions that take human comfort into account is not a part of the human-robot handover process. We propose to generate smooth motions via an efficient model-predictive control framework. We conduct human-to-robot handover experiments on a diverse set of objects with several users.
  arXiv  Detail & Related papers  (2022-03-31T23:08:20Z)
• Investigations of Performance and Bias in Human-AI Teamwork in Hiring [30.046502708053097]
  In AI-assisted decision-making, effective hybrid teamwork (human-AI) is not solely dependent on AI performance alone. We investigate how both a model's predictive performance and bias may transfer to humans in a recommendation-aided decision task.
  arXiv  Detail & Related papers  (2022-02-21T17:58:07Z)
• Combining Human Predictions with Model Probabilities via Confusion Matrices and Calibration [11.75395256889808]
  We develop a set of algorithms that combine the probabilistic output of a model with the class-level output of a human. We show theoretically that the accuracy of our combination model is driven not only by the individual human and model accuracies, but also by the model's confidence.
  arXiv  Detail & Related papers  (2021-09-29T17:43:15Z)
• Skill Preferences: Learning to Extract and Execute Robotic Skills from Human Feedback [82.96694147237113]
  We present Skill Preferences, an algorithm that learns a model over human preferences and uses it to extract human-aligned skills from offline data. We show that SkiP enables a simulated kitchen robot to solve complex multi-step manipulation tasks.
  arXiv  Detail & Related papers  (2021-08-11T18:04:08Z)
• Humans learn too: Better Human-AI Interaction using Optimized Human Inputs [2.5991265608180396]
  Humans rely more and more on systems with AI components. The AI community typically treats human inputs as a given and optimize AI models only. In this work, human inputs are optimized for better interaction with an AI model while keeping the model fixed.
  arXiv  Detail & Related papers  (2020-09-19T16:30:37Z)
• Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork [54.309495231017344]
  We argue that AI systems should be trained in a human-centered manner, directly optimized for team performance. We study this proposal for a specific type of human-AI teaming, where the human overseer chooses to either accept the AI recommendation or solve the task themselves. Our experiments with linear and non-linear models on real-world, high-stakes datasets show that the most accuracy AI may not lead to highest team performance.
  arXiv  Detail & Related papers  (2020-04-27T19:06:28Z)
• AvgOut: A Simple Output-Probability Measure to Eliminate Dull Responses [97.50616524350123]
  We build dialogue models that are dynamically aware of what utterances or tokens are dull without any feature-engineering. The first model, MinAvgOut, directly maximizes the diversity score through the output distributions of each batch. The second model, Label Fine-Tuning (LFT), prepends to the source sequence a label continuously scaled by the diversity score to control the diversity level. The third model, RL, adopts Reinforcement Learning and treats the diversity score as a reward signal.
  arXiv  Detail & Related papers  (2020-01-15T18:32:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.