MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluation

• URL: http://arxiv.org/abs/2308.08147v1
• Date: Wed, 16 Aug 2023 04:56:55 GMT
• Title: MDDial: A Multi-turn Differential Diagnosis Dialogue Dataset with Reliability Evaluation
• Authors: Srija Macherla, Man Luo, Mihir Parmar, Chitta Baral
• Abstract summary: Building end-to-end ADD dialogue systems requires dialogue training datasets. There is no publicly available ADD dialogue dataset in English. We introduce MDDial, the first differential diagnosis dialogue dataset in English.
• Score: 46.82607230465541
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Dialogue systems for Automatic Differential Diagnosis (ADD) have a wide range of real-life applications. These dialogue systems are promising for providing easy access and reducing medical costs. Building end-to-end ADD dialogue systems requires dialogue training datasets. However, to the best of our knowledge, there is no publicly available ADD dialogue dataset in English (although non-English datasets exist). Driven by this, we introduce MDDial, the first differential diagnosis dialogue dataset in English which can aid to build and evaluate end-to-end ADD dialogue systems. Additionally, earlier studies present the accuracy of diagnosis and symptoms either individually or as a combined weighted score. This method overlooks the connection between the symptoms and the diagnosis. We introduce a unified score for the ADD system that takes into account the interplay between symptoms and diagnosis. This score also indicates the system's reliability. To the end, we train two moderate-size of language models on MDDial. Our experiments suggest that while these language models can perform well on many natural language understanding tasks, including dialogue tasks in the general domain, they struggle to relate relevant symptoms and disease and thus have poor performance on MDDial. MDDial will be released publicly to aid the study of ADD dialogue research.

Related papers

• Medical Dialogue Generation via Dual Flow Modeling [9.328694317877169]
  Medical dialogue systems (MDS) aim to provide patients with medical services, such as diagnosis and prescription. Previous studies mainly addressed this by extracting the mentioned medical entities as critical dialogue history information. In this work, we argue that it is also essential to capture the transitions of the medical entities and the doctor's dialogue acts in each turn.
  arXiv  Detail & Related papers  (2023-05-29T14:23:34Z)
• Enhancing Task Bot Engagement with Synthesized Open-Domain Dialog [89.35658776144638]
  It is essential to build a system that can handle both TOD and ODD and access different knowledge sources. We propose a framework for automatically generating dialogues that combine knowledge-grounded ODDs and TODs in various settings. We introduce a unified model PivotBot that is capable of appropriately adopting TOD and ODD modes and accessing different knowledge sources.
  arXiv  Detail & Related papers  (2022-12-20T05:51:47Z)
• D4: a Chinese Dialogue Dataset for Depression-Diagnosis-Oriented Chat [25.852922703368133]
  In a depression-diagnosis-directed clinical session, doctors initiate a conversation with ample emotional support that guides the patients to expose their symptoms. Due to the social stigma associated with mental illness, the dialogue data related to depression consultation and diagnosis are rarely disclosed. We construct a Chinese dialogue dataset for Depression-Diagnosis-Oriented Chat which simulates the dialogue between doctors and patients during the diagnosis of depression.
  arXiv  Detail & Related papers  (2022-05-24T03:54:22Z)
• "My nose is running.""Are you also coughing?": Building A Medical Diagnosis Agent with Interpretable Inquiry Logics [80.55587329326046]
  We propose a more interpretable decision process to implement the dialogue manager of DSMD. We devise a model with highly transparent components to conduct the inference. Experiments show that our method obtains 7.7%, 10.0%, 3.0% absolute improvement in diagnosis accuracy.
  arXiv  Detail & Related papers  (2022-04-29T09:02:23Z)
• Prompt-based Generative Approach towards Multi-Hierarchical Medical Dialogue State Tracking [5.586690662128686]
  The dialogue state tracking (DST) module in the medical dialogue system interprets utterances into a machine-readable structure for downstream tasks. We first define a multi-hierarchical state structure and then propose a Prompt-based Generative Approach. Our approach outperforms other DST methods and is rather effective in the scenario with little data.
  arXiv  Detail & Related papers  (2022-03-18T13:28:27Z)
• UniDS: A Unified Dialogue System for Chit-Chat and Task-oriented Dialogues [59.499965460525694]
  We propose a unified dialogue system (UniDS) with the two aforementioned skills. We design a unified dialogue data schema, compatible for both chit-chat and task-oriented dialogues. We train UniDS with mixed dialogue data from a pretrained chit-chat dialogue model.
  arXiv  Detail & Related papers  (2021-10-15T11:56:47Z)
• Knowledge Grounded Conversational Symptom Detection with Graph Memory Networks [5.788153402669881]
  We build a system that can interact with patients through dialog to detect and collect clinical symptoms automatically. Given a set of explicit symptoms provided by the patient to initiate a dialog for diagnosing, the system is trained to collect implicit symptoms by asking questions. After getting the reply from the patient for each question, the system also decides whether current information is enough for a human doctor to make a diagnosis.
  arXiv  Detail & Related papers  (2021-01-24T18:50:16Z)
• MedDG: An Entity-Centric Medical Consultation Dataset for Entity-Aware Medical Dialogue Generation [86.38736781043109]
  We build and release a large-scale high-quality Medical Dialogue dataset related to 12 types of common Gastrointestinal diseases named MedDG. We propose two kinds of medical dialogue tasks based on MedDG dataset. One is the next entity prediction and the other is the doctor response generation. Experimental results show that the pre-train language models and other baselines struggle on both tasks with poor performance in our dataset.
  arXiv  Detail & Related papers  (2020-10-15T03:34:33Z)
• Rethinking Dialogue State Tracking with Reasoning [76.0991910623001]
  This paper proposes to track dialogue states gradually with reasoning over dialogue turns with the help of the back-end data. Empirical results demonstrate that our method significantly outperforms the state-of-the-art methods by 38.6% in terms of joint belief accuracy for MultiWOZ 2.1.
  arXiv  Detail & Related papers  (2020-05-27T02:05:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.