Multi-Level Testing of Conversational AI Systems

• URL: http://arxiv.org/abs/2602.03311v1
• Date: Tue, 03 Feb 2026 09:38:59 GMT
• Title: Multi-Level Testing of Conversational AI Systems
• Authors: Elena Masserini,
• Abstract summary: This thesis investigates a new family of testing approaches for conversational AI systems.<n>It focuses on the validation of their constituent elements at different levels of granularity.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conversational AI systems combine AI-based solutions with the flexibility of conversational interfaces. However, most existing testing solutions do not straightforwardly adapt to the characteristics of conversational interaction or to the behavior of AI components. To address this limitation, this Ph.D. thesis investigates a new family of testing approaches for conversational AI systems, focusing on the validation of their constituent elements at different levels of granularity, from the integration between the language and the AI components, to individual conversational agents, up to multi-agent implementations of conversational AI systems

Related papers

• AI as Teammate or Tool? A Review of Human-AI Interaction in Decision Support [0.514825619161626]
  Current AI systems remain largely passive due to an overreliance on explainability-centric designs.<n> transitioning AI to an active teammate requires adaptive, context-aware interactions.
  arXiv  Detail & Related papers  (2026-01-26T19:18:50Z)
• A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions [51.96890647837277]
  Large Language Models (LLMs) have propelled conversational AI from traditional dialogue systems into sophisticated agents capable of autonomous actions, contextual awareness, and multi-turn interactions with users.<n>This survey paper presents a desideratum for next-generation Conversational Agents - what has been achieved, what challenges persist, and what must be done for more scalable systems that approach human-level intelligence.
  arXiv  Detail & Related papers  (2025-04-07T21:01:25Z)
• Towards Anthropomorphic Conversational AI Part I: A Practical Framework [49.62013440962072]
  We introduce a multi- module framework designed to replicate the key aspects of human intelligence involved in conversations.<n>In the second stage of our approach, these conversational data, after filtering and labeling, can serve as training and testing data for reinforcement learning.
  arXiv  Detail & Related papers  (2025-02-28T03:18:39Z)
• Who Speaks Next? Multi-party AI Discussion Leveraging the Systematics of Turn-taking in Murder Mystery Games [7.946510318969307]
  This study focuses on conversational norms such as adjacency pairs and turn-taking found in conversation analysis.<n>We propose a new framework called "Murder Mystery Agents" that applies these norms to AI agents' dialogue control.<n>The proposed framework integrates next speaker selection based on adjacency pairs and a self-selection mechanism.
  arXiv  Detail & Related papers  (2024-12-06T10:45:54Z)
• WavChat: A Survey of Spoken Dialogue Models [66.82775211793547]
  Recent advancements in spoken dialogue models, exemplified by systems like GPT-4o, have captured significant attention in the speech domain. These advanced spoken dialogue models not only comprehend audio, music, and other speech-related features, but also capture stylistic and timbral characteristics in speech. Despite the progress in spoken dialogue systems, there is a lack of comprehensive surveys that systematically organize and analyze these systems.
  arXiv  Detail & Related papers  (2024-11-15T04:16:45Z)
• X-TURING: Towards an Enhanced and Efficient Turing Test for Long-Term Dialogue Agents [56.64615470513102]
  The Turing test examines whether AIs exhibit human-like behaviour in natural language conversations.<n>Traditional setting limits each participant to one message at a time and requires constant human participation.<n>This paper proposes textbftextscX-Turing, which enhances the original test with a textitburst dialogue pattern.
  arXiv  Detail & Related papers  (2024-08-19T09:57:28Z)
• Human-AI collaboration is not very collaborative yet: A taxonomy of interaction patterns in AI-assisted decision making from a systematic review [6.013543974938446]
  Leveraging Artificial Intelligence in decision support systems has disproportionately focused on technological advancements. A human-centered perspective attempts to alleviate this concern by designing AI solutions for seamless integration with existing processes.
  arXiv  Detail & Related papers  (2023-10-30T17:46:38Z)
• Expanding the Set of Pragmatic Considerations in Conversational AI [0.26206189324400636]
  We discuss several pragmatic limitations of current conversational AI systems. We label our complaints as "Turing Test Triggers" (TTTs) We develop a taxonomy of pragmatic considerations intended to identify what pragmatic competencies a conversational AI system requires.
  arXiv  Detail & Related papers  (2023-10-27T19:21:50Z)
• Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review [40.49926141538684]
  Visual Context Augmented Dialogue System (VAD) has the potential to communicate with humans by perceiving and understanding multimodal information. VAD possesses the potential to generate engaging and context-aware responses.
  arXiv  Detail & Related papers  (2022-07-02T09:31:37Z)
• End-to-end Spoken Conversational Question Answering: Task, Dataset and Model [92.18621726802726]
  In spoken question answering, the systems are designed to answer questions from contiguous text spans within the related speech transcripts. We propose a new Spoken Conversational Question Answering task (SCQA), aiming at enabling the systems to model complex dialogue flows. Our main objective is to build the system to deal with conversational questions based on the audio recordings, and to explore the plausibility of providing more cues from different modalities with systems in information gathering.
  arXiv  Detail & Related papers  (2022-04-29T17:56:59Z)
• Towards Data Distillation for End-to-end Spoken Conversational Question Answering [65.124088336738]
  We propose a new Spoken Conversational Question Answering task (SCQA) SCQA aims at enabling QA systems to model complex dialogues flow given the speech utterances and text corpora. Our main objective is to build a QA system to deal with conversational questions both in spoken and text forms.
  arXiv  Detail & Related papers  (2020-10-18T05:53:39Z)
• Attention over Parameters for Dialogue Systems [69.48852519856331]
  We learn a dialogue system that independently parameterizes different dialogue skills, and learns to select and combine each of them through Attention over Parameters (AoP) The experimental results show that this approach achieves competitive performance on a combined dataset of MultiWOZ, In-Car Assistant, and Persona-Chat.
  arXiv  Detail & Related papers  (2020-01-07T03:10:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.