CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures

• URL: http://arxiv.org/abs/2410.05235v2
• Date: Tue, 8 Oct 2024 13:12:04 GMT
• Title: CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures
• Authors: Ekaterina Sviridova, Anar Yeginbergen, Ainara Estarrona, Elena Cabrio, Serena Villata, Rodrigo Agerri,
• Abstract summary: We present the first multilingual dataset for Medical Question Answering where correct and incorrect diagnoses for a clinical case are enriched with a natural language explanation written by doctors. This dataset consists of 558 clinical cases in four languages (English, Spanish, French, Italian) with explanations, where we annotated 5021 claims, 2313 premises, 2431 support relations, and 1106 attack relations.
• Score: 19.242920846826895
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Explaining Artificial Intelligence (AI) decisions is a major challenge nowadays in AI, in particular when applied to sensitive scenarios like medicine and law. However, the need to explain the rationale behind decisions is a main issue also for human-based deliberation as it is important to justify \textit{why} a certain decision has been taken. Resident medical doctors for instance are required not only to provide a (possibly correct) diagnosis, but also to explain how they reached a certain conclusion. Developing new tools to aid residents to train their explanation skills is therefore a central objective of AI in education. In this paper, we follow this direction, and we present, to the best of our knowledge, the first multilingual dataset for Medical Question Answering where correct and incorrect diagnoses for a clinical case are enriched with a natural language explanation written by doctors. These explanations have been manually annotated with argument components (i.e., premise, claim) and argument relations (i.e., attack, support), resulting in the Multilingual CasiMedicos-Arg dataset which consists of 558 clinical cases in four languages (English, Spanish, French, Italian) with explanations, where we annotated 5021 claims, 2313 premises, 2431 support relations, and 1106 attack relations. We conclude by showing how competitive baselines perform over this challenging dataset for the argument mining task.

Related papers

• Explainable AI: Definition and attributes of a good explanation for health AI [0.18846515534317265]
  understanding how and why an AI system makes a recommendation may require complex explanations of its inner workings and reasoning processes. To fully realize the potential of AI, it is critical to address two fundamental questions about explanations for safety-critical AI applications. The research outputs include (1) a definition of what constitutes an explanation in health-AI and (2) a comprehensive list of attributes that characterize a good explanation in health-AI.
  arXiv  Detail & Related papers  (2024-09-09T16:56:31Z)
• Diagnostic Reasoning in Natural Language: Computational Model and Application [68.47402386668846]
  We investigate diagnostic abductive reasoning (DAR) in the context of language-grounded tasks (NL-DAR) We propose a novel modeling framework for NL-DAR based on Pearl's structural causal models. We use the resulting dataset to investigate the human decision-making process in NL-DAR.
  arXiv  Detail & Related papers  (2024-09-09T06:55:37Z)
• The Explanation Necessity for Healthcare AI [3.8953842074141387]
  We propose a novel categorization system with four distinct classes of explanation necessity. Three key factors are considered: the robustness of the evaluation protocol, the variability of expert observations, and the representation dimensionality of the application.
  arXiv  Detail & Related papers  (2024-05-31T22:20:10Z)
• InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification [60.10193972862099]
  This work proposes a framework to characterize and recover simplification-induced information loss in form of question-and-answer pairs. QA pairs are designed to help readers deepen their knowledge of a text.
  arXiv  Detail & Related papers  (2024-01-29T19:00:01Z)
• Explanatory Argument Extraction of Correct Answers in Resident Medical Exams [5.399800035598185]
  We present a new dataset which includes not only explanatory arguments for the correct answer, but also arguments to reason why the incorrect answers are not correct. This new benchmark allows us to setup a novel extractive task which consists of identifying the explanation of the correct answer written by medical doctors.
  arXiv  Detail & Related papers  (2023-12-01T13:22:35Z)
• Generating Explanations in Medical Question-Answering by Expectation Maximization Inference over Evidence [33.018873142559286]
  We propose a novel approach for generating natural language explanations for answers predicted by medical QA systems. Our system extract knowledge from medical textbooks to enhance the quality of explanations during the explanation generation process.
  arXiv  Detail & Related papers  (2023-10-02T16:00:37Z)
• HiTZ@Antidote: Argumentation-driven Explainable Artificial Intelligence for Digital Medicine [7.089952396422835]
  ANTIDOTE fosters an integrated vision of explainable AI, where low-level characteristics of the deep learning process are combined with higher level schemes proper of the human argumentation capacity. As a first result of the project, we publish the Antidote CasiMedicos dataset to facilitate research on explainable AI in general, and argumentation in the medical domain in particular.
  arXiv  Detail & Related papers  (2023-06-09T16:50:02Z)
• Informing clinical assessment by contextualizing post-hoc explanations of risk prediction models in type-2 diabetes [50.8044927215346]
  We consider a comorbidity risk prediction scenario and focus on contexts regarding the patients clinical state. We employ several state-of-the-art LLMs to present contexts around risk prediction model inferences and evaluate their acceptability. Our paper is one of the first end-to-end analyses identifying the feasibility and benefits of contextual explanations in a real-world clinical use case.
  arXiv  Detail & Related papers  (2023-02-11T18:07:11Z)
• What Do End-Users Really Want? Investigation of Human-Centered XAI for Mobile Health Apps [69.53730499849023]
  We present a user-centered persona concept to evaluate explainable AI (XAI) Results show that users' demographics and personality, as well as the type of explanation, impact explanation preferences. Our insights bring an interactive, human-centered XAI closer to practical application.
  arXiv  Detail & Related papers  (2022-10-07T12:51:27Z)
• CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark [51.38557174322772]
  We present the first Chinese Biomedical Language Understanding Evaluation benchmark. It is a collection of natural language understanding tasks including named entity recognition, information extraction, clinical diagnosis normalization, single-sentence/sentence-pair classification. We report empirical results with the current 11 pre-trained Chinese models, and experimental results show that state-of-the-art neural models perform by far worse than the human ceiling.
  arXiv  Detail & Related papers  (2021-06-15T12:25:30Z)
• Interpretable Multi-Step Reasoning with Knowledge Extraction on Complex Healthcare Question Answering [89.76059961309453]
  HeadQA dataset contains multiple-choice questions authorized for the public healthcare specialization exam. These questions are the most challenging for current QA systems. We present a Multi-step reasoning with Knowledge extraction framework (MurKe) We are striving to make full use of off-the-shelf pre-trained models.
  arXiv  Detail & Related papers  (2020-08-06T02:47:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.