Explainability for Transparent Conversational Information-Seeking

• URL: http://arxiv.org/abs/2405.03303v1
• Date: Mon, 06 May 2024 09:25:14 GMT
• Title: Explainability for Transparent Conversational Information-Seeking
• Authors: Weronika Łajewska, Damiano Spina, Johanne Trippas, Krisztian Balog,
• Abstract summary: This study explores different methods of explaining the responses. By exploring transparency across explanation type, quality, and presentation mode, this research aims to bridge the gap between system-generated responses and responses verifiable by the user.
• Score: 13.790574266700006
• License:
• Abstract: The increasing reliance on digital information necessitates advancements in conversational search systems, particularly in terms of information transparency. While prior research in conversational information-seeking has concentrated on improving retrieval techniques, the challenge remains in generating responses useful from a user perspective. This study explores different methods of explaining the responses, hypothesizing that transparency about the source of the information, system confidence, and limitations can enhance users' ability to objectively assess the response. By exploring transparency across explanation type, quality, and presentation mode, this research aims to bridge the gap between system-generated responses and responses verifiable by the user. We design a user study to answer questions concerning the impact of (1) the quality of explanations enhancing the response on its usefulness and (2) ways of presenting explanations to users. The analysis of the collected data reveals lower user ratings for noisy explanations, although these scores seem insensitive to the quality of the response. Inconclusive results on the explanations presentation format suggest that it may not be a critical factor in this setting.

Related papers

• Can Users Detect Biases or Factual Errors in Generated Responses in Conversational Information-Seeking? [13.790574266700006]
  We investigate the limitations of response generation in conversational information-seeking systems. The study addresses the problem of query answerability and the challenge of response incompleteness. Our analysis reveals that it is easier for users to detect response incompleteness than query answerability.
  arXiv  Detail & Related papers  (2024-10-28T20:55:00Z)
• Rethinking the Evaluation of Dialogue Systems: Effects of User Feedback on Crowdworkers and LLMs [57.16442740983528]
  In ad-hoc retrieval, evaluation relies heavily on user actions, including implicit feedback. The role of user feedback in annotators' assessment of turns in a conversational perception has been little studied. We focus on how the evaluation of task-oriented dialogue systems ( TDSs) is affected by considering user feedback, explicit or implicit, as provided through the follow-up utterance of a turn being evaluated.
  arXiv  Detail & Related papers  (2024-04-19T16:45:50Z)
• Diffexplainer: Towards Cross-modal Global Explanations with Diffusion Models [51.21351775178525]
  DiffExplainer is a novel framework that, leveraging language-vision models, enables multimodal global explainability. It employs diffusion models conditioned on optimized text prompts, synthesizing images that maximize class outputs. The analysis of generated visual descriptions allows for automatic identification of biases and spurious features.
  arXiv  Detail & Related papers  (2024-04-03T10:11:22Z)
• PAQA: Toward ProActive Open-Retrieval Question Answering [34.883834970415734]
  This work aims to tackle the challenge of generating relevant clarifying questions by taking into account the inherent ambiguities present in both user queries and documents. We propose PAQA, an extension to the existing AmbiNQ dataset, incorporating clarifying questions. We then evaluate various models and assess how passage retrieval impacts ambiguity detection and the generation of clarifying questions.
  arXiv  Detail & Related papers  (2024-02-26T14:40:34Z)
• Asking Multimodal Clarifying Questions in Mixed-Initiative Conversational Search [89.1772985740272]
  In mixed-initiative conversational search systems, clarifying questions are used to help users who struggle to express their intentions in a single query. We hypothesize that in scenarios where multimodal information is pertinent, the clarification process can be improved by using non-textual information. We collect a dataset named Melon that contains over 4k multimodal clarifying questions, enriched with over 14k images. Several analyses are conducted to understand the importance of multimodal contents during the query clarification phase.
  arXiv  Detail & Related papers  (2024-02-12T16:04:01Z)
• InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification [60.10193972862099]
  This work proposes a framework to characterize and recover simplification-induced information loss in form of question-and-answer pairs. QA pairs are designed to help readers deepen their knowledge of a text.
  arXiv  Detail & Related papers  (2024-01-29T19:00:01Z)
• FeedbackMap: a tool for making sense of open-ended survey responses [1.0660480034605242]
  This demo introduces FeedbackMap, a web-based tool that uses natural language processing techniques to facilitate the analysis of open-ended survey responses. We discuss the importance of examining survey results from multiple perspectives and the potential biases introduced by summarization methods.
  arXiv  Detail & Related papers  (2023-06-26T23:38:24Z)
• Explaining Explainability: Towards Deeper Actionable Insights into Deep Learning through Second-order Explainability [70.60433013657693]
  Second-order explainable AI (SOXAI) was recently proposed to extend explainable AI (XAI) from the instance level to the dataset level. We demonstrate for the first time, via example classification and segmentation cases, that eliminating irrelevant concepts from the training set based on actionable insights from SOXAI can enhance a model's performance.
  arXiv  Detail & Related papers  (2023-06-14T23:24:01Z)
• Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions? [50.29862466940209]
  We introduce InfoSeek, a visual question answering dataset tailored for information-seeking questions. We analyze various pre-trained visual question answering models and gain insights into their characteristics. We show that accurate visual entity recognition can be used to improve performance on InfoSeek by retrieving relevant documents.
  arXiv  Detail & Related papers  (2023-02-23T00:33:54Z)
• Guided Transformer: Leveraging Multiple External Sources for Representation Learning in Conversational Search [36.64582291809485]
  Asking clarifying questions in response to ambiguous or faceted queries has been recognized as a useful technique for various information retrieval systems. In this paper, we enrich the representations learned by Transformer networks using a novel attention mechanism from external information sources. Our experiments use a public dataset for search clarification and demonstrate significant improvements compared to competitive baselines.
  arXiv  Detail & Related papers  (2020-06-13T03:24:53Z)
• Review-guided Helpful Answer Identification in E-commerce [38.276241153439955]
  Product-specific community question answering platforms can greatly help address the concerns of potential customers. The user-provided answers on such platforms often vary a lot in their qualities. Helpfulness votes from the community can indicate the overall quality of the answer, but they are often missing.
  arXiv  Detail & Related papers  (2020-03-13T11:34:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.