Uncertainty-Aware Text-to-Program for Question Answering on Structured Electronic Health Records

• URL: http://arxiv.org/abs/2203.06918v1
• Date: Mon, 14 Mar 2022 08:12:16 GMT
• Title: Uncertainty-Aware Text-to-Program for Question Answering on Structured Electronic Health Records
• Authors: Daeyoung Kim, Seongsu Bae, Seungho Kim, Edward Choi
• Abstract summary: We design the program-based model (NLQ2Program) for EHR-QA as the first step towards the future direction. We tackle MIMICSPARQL*, the graph-based EHR-QA dataset, via a program-based approach in a semi-supervised manner. For a reliable EHR-QA model, we apply the uncertainty decomposition method to measure the ambiguity in the input question.
• Score: 8.272573489245717
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Question Answering on Electronic Health Records (EHR-QA) has a significant impact on the healthcare domain, and it is being actively studied. Previous research on structured EHR-QA focuses on converting natural language queries into query language such as SQL or SPARQL (NLQ2Query), so the problem scope is limited to pre-defined data types by the specific query language. In order to expand the EHR-QA task beyond this limitation to handle multi-modal medical data and solve complex inference in the future, more primitive systemic language is needed. In this paper, we design the program-based model (NLQ2Program) for EHR-QA as the first step towards the future direction. We tackle MIMICSPARQL*, the graph-based EHR-QA dataset, via a program-based approach in a semi-supervised manner in order to overcome the absence of gold programs. Without the gold program, our proposed model shows comparable performance to the previous state-of-the-art model, which is an NLQ2Query model (0.9\% gain). In addition, for a reliable EHR-QA model, we apply the uncertainty decomposition method to measure the ambiguity in the input question. We empirically confirmed data uncertainty is most indicative of the ambiguity in the input question.

Related papers

• LG AI Research & KAIST at EHRSQL 2024: Self-Training Large Language Models with Pseudo-Labeled Unanswerable Questions for a Reliable Text-to-SQL System on EHRs [58.59113843970975]
  Text-to-answer models are pivotal for making Electronic Health Records accessible to healthcare professionals without knowledge. We present a self-training strategy using pseudo-labeled un-answerable questions to enhance the reliability of text-to-answer models for EHRs.
  arXiv  Detail & Related papers  (2024-05-18T03:25:44Z)
• Uncertainty-aware Language Modeling for Selective Question Answering [107.47864420630923]
  We present an automatic large language model (LLM) conversion approach that produces uncertainty-aware LLMs. Our approach is model- and data-agnostic, is computationally-efficient, and does not rely on external models or systems.
  arXiv  Detail & Related papers  (2023-11-26T22:47:54Z)
• Toward Unsupervised Realistic Visual Question Answering [70.67698100148414]
  We study the problem of realistic VQA (RVQA), where a model has to reject unanswerable questions (UQs) and answer answerable ones (AQs) We first point out 2 drawbacks in current RVQA research, where (1) datasets contain too many unchallenging UQs and (2) a large number of annotated UQs are required for training. We propose a new testing dataset, RGQA, which combines AQs from an existing VQA dataset with around 29K human-annotated UQs. This combines pseudo UQs obtained by randomly pairing images and questions, with an
  arXiv  Detail & Related papers  (2023-03-09T06:58:29Z)
• RoMQA: A Benchmark for Robust, Multi-evidence, Multi-answer Question Answering [87.18962441714976]
  We introduce RoMQA, the first benchmark for robust, multi-evidence, multi-answer question answering (QA) We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, and find that RoMQA is challenging. Our results show that RoMQA is a challenging benchmark for large language models, and provides a quantifiable test to build more robust QA methods.
  arXiv  Detail & Related papers  (2022-10-25T21:39:36Z)
• Question Answering for Complex Electronic Health Records Database using Unified Encoder-Decoder Architecture [8.656936724622145]
  We design UniQA, a unified-decoder architecture for EHR-QA where natural language questions are converted to queries such as SPARQL. We also propose input masking (IM), a simple and effective method to cope with complex medical terms and various typos and better learn the SPARQL syntax. UniQA demonstrated a significant performance improvement against the previous state-of-the-art model for MIMIC* (14.2% gain), the most complex NLQ2 dataset in the EHR domain, and its typo-ridden versions.
  arXiv  Detail & Related papers  (2021-11-14T05:01:38Z)
• Knowledge Graph-based Question Answering with Electronic Health Records [30.901617020638124]
  Question Answering (QA) is a widely-used framework for developing and evaluating an intelligent machine. This paper proposes a graph-based EHR QA where natural language queries are converted to SPARQL. All datasets are open-sourced to encourage further EHR QA research in both directions.
  arXiv  Detail & Related papers  (2020-10-19T11:31:20Z)
• Hierarchical Deep Multi-modal Network for Medical Visual Question Answering [25.633660028022195]
  We propose a hierarchical deep multi-modal network that analyzes and classifies end-user questions/queries. We integrate the QS model to the hierarchical deep multi-modal neural network to generate proper answers to the queries related to medical images.
  arXiv  Detail & Related papers  (2020-09-27T07:24:41Z)
• Harvesting and Refining Question-Answer Pairs for Unsupervised QA [95.9105154311491]
  We introduce two approaches to improve unsupervised Question Answering (QA) First, we harvest lexically and syntactically divergent questions from Wikipedia to automatically construct a corpus of question-answer pairs (named as RefQA) Second, we take advantage of the QA model to extract more appropriate answers, which iteratively refines data over RefQA.
  arXiv  Detail & Related papers  (2020-05-06T15:56:06Z)
• Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering [98.48363619128108]
  We propose an unsupervised approach to training QA models with generated pseudo-training data. We show that generating questions for QA training by applying a simple template on a related, retrieved sentence rather than the original context sentence improves downstream QA performance.
  arXiv  Detail & Related papers  (2020-04-24T17:57:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.