A preliminary study on evaluating Consultation Notes with Post-Editing

• URL: http://arxiv.org/abs/2104.04402v1
• Date: Fri, 9 Apr 2021 14:42:00 GMT
• Title: A preliminary study on evaluating Consultation Notes with Post-Editing
• Authors: Francesco Moramarco, Alex Papadopoulos Korfiatis, Aleksandar Savkov, Ehud Reiter
• Abstract summary: We propose a semi-automatic approach whereby physicians post-edit generated notes before submitting them. We conduct a preliminary study on the time saving of automatically generated consultation notes with post-editing. We time this and find that it is faster than writing the note from scratch.
• Score: 67.30200768442926
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Automatic summarisation has the potential to aid physicians in streamlining clerical tasks such as note taking. But it is notoriously difficult to evaluate these systems and demonstrate that they are safe to be used in a clinical setting. To circumvent this issue, we propose a semi-automatic approach whereby physicians post-edit generated notes before submitting them. We conduct a preliminary study on the time saving of automatically generated consultation notes with post-editing. Our evaluators are asked to listen to mock consultations and to post-edit three generated notes. We time this and find that it is faster than writing the note from scratch. We present insights and lessons learnt from this experiment.

Related papers

• Improving Clinical Note Generation from Complex Doctor-Patient Conversation [20.2157016701399]
  We present three key contributions to the field of clinical note generation using large language models (LLMs) First, we introduce CliniKnote, a dataset consisting of 1,200 complex doctor-patient conversations paired with their full clinical notes. Second, we propose K-SOAP, which enhances traditional SOAPcitepodder20soap (Subjective, Objective, Assessment, and Plan) notes by adding a keyword section at the top, allowing for quick identification of essential information. Third, we develop an automatic pipeline to generate K-SOAP notes from doctor-patient conversations and benchmark various modern LLMs using various
  arXiv  Detail & Related papers  (2024-08-26T18:39:31Z)
• Conceptualizing Machine Learning for Dynamic Information Retrieval of Electronic Health Record Notes [6.1656026560972]
  This work conceptualizes the use of EHR audit logs for machine learning as a source of supervision of note relevance in a specific clinical context. We show that our methods can achieve an AUC of 0.963 for predicting which notes will be read in an individual note writing session.
  arXiv  Detail & Related papers  (2023-08-09T21:04:19Z)
• Consultation Checklists: Standardising the Human Evaluation of Medical Note Generation [58.54483567073125]
  We propose a protocol that aims to increase objectivity by grounding evaluations in Consultation Checklists. We observed good levels of inter-annotator agreement in a first evaluation study using the protocol.
  arXiv  Detail & Related papers  (2022-11-17T10:54:28Z)
• User-Driven Research of Medical Note Generation Software [49.85146209418244]
  We present three rounds of user studies carried out in the context of developing a medical note generation system. We discuss the participating clinicians' impressions and views of how the system ought to be adapted to be of value to them. We describe a three-week test run of the system in a live telehealth clinical practice.
  arXiv  Detail & Related papers  (2022-05-05T10:18:06Z)
• Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation [56.25869366777579]
  In recent years, machine learning models have rapidly become better at generating clinical consultation notes. We present an extensive human evaluation study where 5 clinicians listen to 57 mock consultations, write their own notes, post-edit a number of automatically generated notes, and extract all the errors. We find that a simple, character-based Levenshtein distance metric performs on par if not better than common model-based metrics like BertScore.
  arXiv  Detail & Related papers  (2022-04-01T14:04:16Z)
• PriMock57: A Dataset Of Primary Care Mock Consultations [66.29154510369372]
  We detail the development of a public access, high quality dataset comprising of57 mocked primary care consultations. Our work illustrates how the dataset can be used as a benchmark for conversational medical ASR as well as consultation note generation from transcripts.
  arXiv  Detail & Related papers  (2022-04-01T10:18:28Z)
• Towards more patient friendly clinical notes through language models and ontologies [57.51898902864543]
  We present a novel approach to automated medical text based on word simplification and language modelling. We use a new dataset pairs of publicly available medical sentences and a version of them simplified by clinicians. Our method based on a language model trained on medical forum data generates simpler sentences while preserving both grammar and the original meaning.
  arXiv  Detail & Related papers  (2021-12-23T16:11:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.