The Value-Sensitive Conversational Agent Co-Design Framework

• URL: http://arxiv.org/abs/2310.11848v1
• Date: Wed, 18 Oct 2023 09:58:39 GMT
• Title: The Value-Sensitive Conversational Agent Co-Design Framework
• Authors: Malak Sadek, Rafael A. Calvo, Celine Mougenot
• Abstract summary: This paper presents the Value-Sensitive Conversational Agent (VSCA) Framework for enabling the collaborative design (co-design) of value-sensitive CAs with relevant stakeholders. The framework facilitates the co-design of three artefacts that elicit stakeholder values and have a technical utility to CA teams to guide CA implementation.
• Score: 4.9186105778865645
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Conversational agents (CAs) are gaining traction in both industry and academia, especially with the advent of generative AI and large language models. As these agents are used more broadly by members of the general public and take on a number of critical use cases and social roles, it becomes important to consider the values embedded in these systems. This consideration includes answering questions such as 'whose values get embedded in these agents?' and 'how do those values manifest in the agents being designed?' Accordingly, the aim of this paper is to present the Value-Sensitive Conversational Agent (VSCA) Framework for enabling the collaborative design (co-design) of value-sensitive CAs with relevant stakeholders. Firstly, requirements for co-designing value-sensitive CAs which were identified in previous works are summarised here. Secondly, the practical framework is presented and discussed, including its operationalisation into a design toolkit. The framework facilitates the co-design of three artefacts that elicit stakeholder values and have a technical utility to CA teams to guide CA implementation, enabling the creation of value-embodied CA prototypes. Finally, an evaluation protocol for the framework is proposed where the effects of the framework and toolkit are explored in a design workshop setting to evaluate both the process followed and the outcomes produced.

Related papers

• Co-DETECT: Collaborative Discovery of Edge Cases in Text Classification [89.62851347390959]
  Co-DETECT (Collaborative Discovery of Edge cases in TExt ClassificaTion) is a novel mixed-initiative annotation framework.<n>It integrates human expertise with automatic annotation guided by large language models.
  arXiv  Detail & Related papers  (2025-07-07T13:48:54Z)
• Manifesto from Dagstuhl Perspectives Workshop 24352 -- Conversational Agents: A Framework for Evaluation (CAFE) [59.64777874324281]
  We defined the Conversational Agents Framework for Evaluation (CAFE) for the evaluation of CONIAC systems.<n>CAFE consists of six major components: 1) goals of the system's stakeholders, 2) user tasks to be studied in the evaluation, 3) aspects of the users carrying out the tasks, 4) evaluation criteria to be considered, 5) evaluation methodology to be applied, and 6) measures for the quantitative criteria chosen.
  arXiv  Detail & Related papers  (2025-06-08T16:25:35Z)
• Creativity in LLM-based Multi-Agent Systems: A Survey [56.25583236738877]
  Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts.<n>This is the first survey dedicated to creativity in MAS.<n>We focus on text and image generation tasks, and present: (1) a taxonomy of agent proactivity and persona design; (2) an overview of generation techniques, including divergent exploration, iterative refinement, and collaborative synthesis, as well as relevant datasets and evaluation metrics; and (3) a discussion of key challenges, such as inconsistent evaluation standards, insufficient bias mitigation, coordination conflicts, and the lack of unified benchmarks.
  arXiv  Detail & Related papers  (2025-05-27T12:36:14Z)
• CReSt: A Comprehensive Benchmark for Retrieval-Augmented Generation with Complex Reasoning over Structured Documents [6.359764486371197]
  Large Language Models (LLMs) have made substantial progress in recent years, yet evaluating their capabilities in practical Retrieval-Augmented Generation (RAG) scenarios remains challenging.<n>We present CReSt (A Comprehensive Benchmark for Retrieval-Augmented Generation with Complex Reasoning over Structured Documents), a benchmark designed to assess these key dimensions holistically.<n>CReSt comprises 2,245 human-annotated examples in English and Korean, designed to capture practical RAG scenarios that require complex reasoning over structured documents.
  arXiv  Detail & Related papers  (2025-05-23T05:56:25Z)
• AgentOrca: A Dual-System Framework to Evaluate Language Agents on Operational Routine and Constraint Adherence [54.317522790545304]
  We present AgentOrca, a dual-system framework for evaluating language agents' compliance with operational constraints and routines. Our framework encodes action constraints and routines through both natural language prompts for agents and corresponding executable code serving as ground truth for automated verification. Our findings reveal notable performance gaps among state-of-the-art models, with large reasoning models like o1 demonstrating superior compliance while others show significantly lower performance.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)
• A Cooperative Multi-Agent Framework for Zero-Shot Named Entity Recognition [71.61103962200666]
  Zero-shot named entity recognition (NER) aims to develop entity recognition systems from unannotated text corpora. Recent work has adapted large language models (LLMs) for zero-shot NER by crafting specialized prompt templates. We introduce the cooperative multi-agent system (CMAS), a novel framework for zero-shot NER.
  arXiv  Detail & Related papers  (2025-02-25T23:30:43Z)
• Rubric-based Learner Modelling via Noisy Gates Bayesian Networks for Computational Thinking Skills Assessment [40.06500618820166]
  We develop a learner model for automatic skill assessment from a task-specific competence rubric. We design a network with two layers of gates, one performing disjunctive operations by noisy-OR gates and the other conjunctive operations through logical ANDs. The CT-cube skills assessment framework and the Cross Array Task (CAT) are used to exemplify it and demonstrate its feasibility.
  arXiv  Detail & Related papers  (2024-08-02T12:21:05Z)
• EAGER: Two-Stream Generative Recommender with Behavior-Semantic Collaboration [63.112790050749695]
  We introduce EAGER, a novel generative recommendation framework that seamlessly integrates both behavioral and semantic information. We validate the effectiveness of EAGER on four public benchmarks, demonstrating its superior performance compared to existing methods.
  arXiv  Detail & Related papers  (2024-06-20T06:21:56Z)
• Reframing the Relationship in Out-of-Distribution Detection [4.182518087792777]
  We introduce a novel approach that integrates the agent paradigm into the Out-of-distribution (OOD) detection task. Our proposed method, Concept Matching with Agent (CMA), employs neutral prompts as agents to augment the CLIP-based OOD detection process. Our extensive experimental results showcase the superior performance of CMA over both zero-shot and training-required methods.
  arXiv  Detail & Related papers  (2024-05-27T02:27:28Z)
• Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents [28.406492378232695]
  Foundation model based agents derive their autonomy from the capabilities of foundation models. This paper presents a pattern-oriented reference architecture that serves as guidance when designing foundation model based agents.
  arXiv  Detail & Related papers  (2023-11-22T04:21:47Z)
• Coherent Entity Disambiguation via Modeling Topic and Categorical Dependency [87.16283281290053]
  Previous entity disambiguation (ED) methods adopt a discriminative paradigm, where prediction is made based on matching scores between mention context and candidate entities. We propose CoherentED, an ED system equipped with novel designs aimed at enhancing the coherence of entity predictions. We achieve new state-of-the-art results on popular ED benchmarks, with an average improvement of 1.3 F1 points.
  arXiv  Detail & Related papers  (2023-11-06T16:40:13Z)
• Rethinking Word-Level Auto-Completion in Computer-Aided Translation [76.34184928621477]
  Word-Level Auto-Completion (WLAC) plays a crucial role in Computer-Assisted Translation. It aims at providing word-level auto-completion suggestions for human translators. We introduce a measurable criterion to answer this question and discover that existing WLAC models often fail to meet this criterion. We propose an effective approach to enhance WLAC performance by promoting adherence to the criterion.
  arXiv  Detail & Related papers  (2023-10-23T03:11:46Z)
• Semantic Communications for Artificial Intelligence Generated Content (AIGC) Toward Effective Content Creation [75.73229320559996]
  This paper develops a conceptual model for the integration of AIGC and SemCom. A novel framework that employs AIGC technology is proposed as an encoder and decoder for semantic information. The framework can adapt to different types of content generated, the required quality, and the semantic information utilized.
  arXiv  Detail & Related papers  (2023-08-09T13:17:21Z)
• A Framework for Auditing Multilevel Models using Explainability Methods [2.578242050187029]
  An audit framework for technical assessment of regressions is proposed. The focus is on three aspects, model, discrimination, and transparency and explainability. It is demonstrated that popular explainability methods, such as SHAP and LIME, underperform in accuracy when interpreting these models.
  arXiv  Detail & Related papers  (2022-07-04T17:53:21Z)
• Exploring the Trade-off between Plausibility, Change Intensity and Adversarial Power in Counterfactual Explanations using Multi-objective Optimization [73.89239820192894]
  We argue that automated counterfactual generation should regard several aspects of the produced adversarial instances. We present a novel framework for the generation of counterfactual examples.
  arXiv  Detail & Related papers  (2022-05-20T15:02:53Z)
• Towards a multi-stakeholder value-based assessment framework for algorithmic systems [76.79703106646967]
  We develop a value-based assessment framework that visualizes closeness and tensions between values. We give guidelines on how to operationalize them, while opening up the evaluation and deliberation process to a wide range of stakeholders.
  arXiv  Detail & Related papers  (2022-05-09T19:28:32Z)
• A Computational Model of the Institutional Analysis and Development Framework [0.0]
  This work presents the first attempt to turn the IAD framework into a computational model. We define the Action Situation Language -- or ASL -- whose syntax is tailored to the components of the IAD framework and that we use to write descriptions of social interactions. These models can be analyzed with the standard tools of game theory to predict which outcomes are being most incentivized, and evaluated according to their socially relevant properties.
  arXiv  Detail & Related papers  (2021-05-27T13:53:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.