Overhearing LLM Agents: A Survey, Taxonomy, and Roadmap

• URL: http://arxiv.org/abs/2509.16325v1
• Date: Fri, 19 Sep 2025 18:11:04 GMT
• Title: Overhearing LLM Agents: A Survey, Taxonomy, and Roadmap
• Abstract summary: We study an alternative paradigm for interacting with LLM agents, which we call "overhearing agents"<n>Rather than demanding the user's attention, overhearing agents continuously monitor ambient activity and intervene only when they can provide contextual assistance.<n>Based on this taxonomy, we create a list of best practices for researchers and developers building overhearing agent systems.
• Score: 34.93417975075926
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Imagine AI assistants that enhance conversations without interrupting them: quietly providing relevant information during a medical consultation, seamlessly preparing materials as teachers discuss lesson plans, or unobtrusively scheduling meetings as colleagues debate calendars. While modern conversational LLM agents directly assist human users with tasks through a chat interface, we study this alternative paradigm for interacting with LLM agents, which we call "overhearing agents." Rather than demanding the user's attention, overhearing agents continuously monitor ambient activity and intervene only when they can provide contextual assistance. In this paper, we present the first analysis of overhearing LLM agents as a distinct paradigm in human-AI interaction and establish a taxonomy of overhearing agent interactions and tasks grounded in a survey of works on prior LLM-powered agents and exploratory HCI studies. Based on this taxonomy, we create a list of best practices for researchers and developers building overhearing agent systems. Finally, we outline the remaining research gaps and reveal opportunities for future research in the overhearing paradigm.

Related papers

• "Excuse me, may I say something..." CoLabScience, A Proactive AI Assistant for Biomedical Discovery and LLM-Expert Collaborations [20.262423767863044]
  We introduce CoLabScience, a proactive LLM assistant designed to enhance biomedical collaboration between AI systems and human experts.<n>At the core of our method is PULI, a novel framework trained with a reinforcement learning objective to determine when and how to intervene in streaming scientific discussions.
  arXiv  Detail & Related papers  (2026-04-16T23:46:22Z)
• Detection of adversarial intent in Human-AI teams using LLMs [4.008152563028669]
  Large language models (LLMs) are increasingly deployed in human-AI teams as support agents for complex tasks.<n>This paper studies their potential role as defensive supervisors within mixed human-AI teams.<n>We find that LLMs are capable of identifying malicious behavior in real-time, and without task-specific information.
  arXiv  Detail & Related papers  (2026-03-21T23:27:51Z)
• LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions [80.12078194093013]
  We present the first comprehensive survey of hallucinations in LLM-based agents.<n>We propose a new taxonomy that identifies different types of agent hallucinations occurring at different stages.<n>We conduct an in-depth examination of eighteen triggering causes underlying the emergence of agent hallucinations.
  arXiv  Detail & Related papers  (2025-09-23T13:24:48Z)
• First Steps Towards Overhearing LLM Agents: A Case Study With Dungeons & Dragons Gameplay [31.08300242866497]
  We present an alternative paradigm for interacting with LLM agents, which we call "overhearing agents"<n>These overhearing agents do not actively participate in conversation -- instead, they "listen in" on human-to-human conversations and perform background tasks or provide suggestions to assist the user.<n>We present an in-depth study using large multimodal audio-language models as overhearing agents to assist a Dungeon Master.
  arXiv  Detail & Related papers  (2025-05-28T19:34:36Z)
• Large Language Models are Autonomous Cyber Defenders [0.1884913108327873]
  Autonomous Cyber Defense (ACD) aims to automate incident response through Artificial Intelligence (AI) agents.<n>Most ACD approaches focus on single-agent scenarios and leverage Reinforcement Learning (RL)<n>Large Language Models (LLMs) can address these concerns by providing explainable actions in general security contexts.
  arXiv  Detail & Related papers  (2025-05-07T22:42:37Z)
• A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions [51.96890647837277]
  Large Language Models (LLMs) have propelled conversational AI from traditional dialogue systems into sophisticated agents capable of autonomous actions, contextual awareness, and multi-turn interactions with users.<n>This survey paper presents a desideratum for next-generation Conversational Agents - what has been achieved, what challenges persist, and what must be done for more scalable systems that approach human-level intelligence.
  arXiv  Detail & Related papers  (2025-04-07T21:01:25Z)
• Agentic Large Language Models, a survey [1.517355052203938]
  Agentic LLMs are large language models that act as agents.<n>We organize the literature according to three categories: reasoning, reflection, and retrieval.<n>Important applications are in medical diagnosis, logistics and financial market analysis.
  arXiv  Detail & Related papers  (2025-03-29T11:02:20Z)
• TAMA: A Human-AI Collaborative Thematic Analysis Framework Using Multi-Agent LLMs for Clinical Interviews [54.35097932763878]
  Thematic analysis (TA) is a widely used qualitative approach for uncovering latent meanings in unstructured text data.<n>Here, we propose TAMA: A Human-AI Collaborative Thematic Analysis framework using Multi-Agent LLMs for clinical interviews.<n>We demonstrate that TAMA outperforms existing LLM-assisted TA approaches, achieving higher thematic hit rate, coverage, and distinctiveness.
  arXiv  Detail & Related papers  (2025-03-26T15:58:16Z)
• Multi-Agent Autonomous Driving Systems with Large Language Models: A Survey of Recent Advances [36.83764809130289]
  Large Language Models (LLMs) have been integrated into Autonomous Driving Systems (ADSs)<n>LLMs face three major challenges: limited perception, insufficient collaboration, and high computational demands.<n>Recent advancements in LLM-based multi-agent ADSs have focused on improving inter-agent communication and cooperation.
  arXiv  Detail & Related papers  (2025-02-24T03:26:13Z)
• A Survey on the Memory Mechanism of Large Language Model based Agents [66.4963345269611]
  Large language model (LLM) based agents have recently attracted much attention from the research and industry communities. LLM-based agents are featured in their self-evolving capability, which is the basis for solving real-world problems. The key component to support agent-environment interactions is the memory of the agents.
  arXiv  Detail & Related papers  (2024-04-21T01:49:46Z)
• A Survey on Large Language Model based Autonomous Agents [105.2509166861984]
  Large language models (LLMs) have demonstrated remarkable potential in achieving human-level intelligence.<n>This paper delivers a systematic review of the field of LLM-based autonomous agents from a holistic perspective.<n>We present a comprehensive overview of the diverse applications of LLM-based autonomous agents in the fields of social science, natural science, and engineering.
  arXiv  Detail & Related papers  (2023-08-22T13:30:37Z)
• CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society [58.04479313658851]
  This paper explores the potential of building scalable techniques to facilitate autonomous cooperation among communicative agents. We propose a novel communicative agent framework named role-playing. Our contributions include introducing a novel communicative agent framework, offering a scalable approach for studying the cooperative behaviors and capabilities of multi-agent systems.
  arXiv  Detail & Related papers  (2023-03-31T01:09:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.