Exploring the Consistency, Quality and Challenges in Manual and Automated Coding of Free-text Diagnoses from Hospital Outpatient Letters

• URL: http://arxiv.org/abs/2311.10856v1
• Date: Fri, 17 Nov 2023 20:32:24 GMT
• Title: Exploring the Consistency, Quality and Challenges in Manual and Automated Coding of Free-text Diagnoses from Hospital Outpatient Letters
• Authors: Warren Del-Pinto, George Demetriou, Meghna Jani, Rikesh Patel, Leanne Gray, Alex Bulcock, Niels Peek, Andrew S. Kanter, William G Dixon, Goran Nenadic
• Abstract summary: This work evaluates the quality and consistency of both manual and automated clinical coding of diagnoses from hospital outpatient letters. A gold standard was constructed by a panel of clinicians from a subset of the annotated diagnoses. Results indicate that humans slightly out-performed automated coding, while both performed notably better when there was only a single diagnosis contained in the free-text description.
• Score: 4.743464936070594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Coding of unstructured clinical free-text to produce interoperable structured data is essential to improve direct care, support clinical communication and to enable clinical research.However, manual clinical coding is difficult and time consuming, which motivates the development and use of natural language processing for automated coding. This work evaluates the quality and consistency of both manual and automated clinical coding of diagnoses from hospital outpatient letters. Using 100 randomly selected letters, two human clinicians performed coding of diagnosis lists to SNOMED CT. Automated coding was also performed using IMO's Concept Tagger. A gold standard was constructed by a panel of clinicians from a subset of the annotated diagnoses. This was used to evaluate the quality and consistency of both manual and automated coding via (1) a distance-based metric, treating SNOMED CT as a graph, and (2) a qualitative metric agreed upon by the panel of clinicians. Correlation between the two metrics was also evaluated. Comparing human and computer-generated codes to the gold standard, the results indicate that humans slightly out-performed automated coding, while both performed notably better when there was only a single diagnosis contained in the free-text description. Automated coding was considered acceptable by the panel of clinicians in approximately 90% of cases.

Related papers

• Assisted morbidity coding: the SISCO.web use case for identifying the main diagnosis in Hospital Discharge Records [0.0]
  The paper aims to present the SISCO.web approach designed to support physicians in filling in Hospital Discharge Records with proper diagnoses and procedures codes. The web service leverages NLP algorithms, specific coding rules, as well as ad hoc decision trees to identify the main condition.
  arXiv  Detail & Related papers  (2024-12-11T16:08:25Z)
• Clinical Evaluation of Medical Image Synthesis: A Case Study in Wireless Capsule Endoscopy [63.39037092484374]
  This study focuses on the clinical evaluation of medical Synthetic Data Generation using Artificial Intelligence (AI) models. The paper contributes by a) presenting a protocol for the systematic evaluation of synthetic images by medical experts and b) applying it to assess TIDE-II, a novel variational autoencoder-based model for high-resolution WCE image synthesis. The results show that TIDE-II generates clinically relevant WCE images, helping to address data scarcity and enhance diagnostic tools.
  arXiv  Detail & Related papers  (2024-10-31T19:48:50Z)
• A Data-Driven Guided Decoding Mechanism for Diagnostic Captioning [11.817595076396925]
  Diagnostic Captioning (DC) automatically generates a diagnostic text from one or more medical images of a patient. We propose a new data-driven guided decoding method that incorporates medical information into the beam search of the diagnostic text generation process. We evaluate the proposed method on two medical datasets using four DC systems that range from generic image-to-text systems with CNN encoders to pre-trained Large Language Models.
  arXiv  Detail & Related papers  (2024-06-20T10:08:17Z)
• Automated Clinical Coding for Outpatient Departments [14.923343535929515]
  This paper is the first to investigate how well state-of-the-art deep learning-based clinical coding approaches work in the outpatient setting at hospital scale. We collect a large outpatient dataset comprising over 7 million notes documenting over half a million patients. We adapt four state-of-the-art clinical coding approaches to this setting and evaluate their potential to assist coders.
  arXiv  Detail & Related papers  (2023-12-21T02:28:29Z)
• Consultation Checklists: Standardising the Human Evaluation of Medical Note Generation [58.54483567073125]
  We propose a protocol that aims to increase objectivity by grounding evaluations in Consultation Checklists. We observed good levels of inter-annotator agreement in a first evaluation study using the protocol.
  arXiv  Detail & Related papers  (2022-11-17T10:54:28Z)
• GrabQC: Graph based Query Contextualization for automated ICD coding [16.096824533334352]
  We propose textbfGrabQC, a textbfGraph textbfbased textbfQuery textbfContextualization method that automatically extracts queries from the clinical text. We perform experiments on two datasets of clinical text in three different setups to assert the effectiveness of our approach.
  arXiv  Detail & Related papers  (2022-07-14T10:27:25Z)
• Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation [56.25869366777579]
  In recent years, machine learning models have rapidly become better at generating clinical consultation notes. We present an extensive human evaluation study where 5 clinicians listen to 57 mock consultations, write their own notes, post-edit a number of automatically generated notes, and extract all the errors. We find that a simple, character-based Levenshtein distance metric performs on par if not better than common model-based metrics like BertScore.
  arXiv  Detail & Related papers  (2022-04-01T14:04:16Z)
• Towards The Automatic Coding of Medical Transcripts to Improve Patient-Centered Communication [0.0]
  We adopt three machine learning algorithms to categorize lines in transcripts into corresponding codes. There is evidence to distinguish the codes, and this is considered to be sufficient for training of human annotators.
  arXiv  Detail & Related papers  (2021-09-22T04:37:05Z)
• Inheritance-guided Hierarchical Assignment for Clinical Automatic Diagnosis [50.15205065710629]
  Clinical diagnosis, which aims to assign diagnosis codes for a patient based on the clinical note, plays an essential role in clinical decision-making. We propose a novel framework to combine the inheritance-guided hierarchical assignment and co-occurrence graph propagation for clinical automatic diagnosis.
  arXiv  Detail & Related papers  (2021-01-27T13:16:51Z)
• Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation [48.87254340298189]
  We construct a new dataset named MedLane to support the development and evaluation of automated clinical language simplification approaches. We propose a new model called DECLARE that follows the human annotation procedure and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-12-04T06:09:02Z)
• CodeBLEU: a Method for Automatic Evaluation of Code Synthesis [57.87741831987889]
  In the area of code synthesis, the commonly used evaluation metric is BLEU or perfect accuracy. We introduce a new automatic evaluation metric, dubbed CodeBLEU. It absorbs the strength of BLEU in the n-gram match and further injects code syntax via abstract syntax trees (AST) and code semantics via data-flow.
  arXiv  Detail & Related papers  (2020-09-22T03:10:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.