CHiLL: Zero-shot Custom Interpretable Feature Extraction from Clinical Notes with Large Language Models

• URL: http://arxiv.org/abs/2302.12343v2
• Date: Thu, 19 Oct 2023 18:16:57 GMT
• Title: CHiLL: Zero-shot Custom Interpretable Feature Extraction from Clinical Notes with Large Language Models
• Abstract summary: CHiLL prompts LLMs with expert-crafted queries to generate interpretable features from health records. The resulting noisy labels are then used to train a simple linear classifier. We find that linear models using automatically extracted features are comparably performant to models using reference features, and provide greater interpretability than linear models using "Bag-of-Words" features.
• Score: 24.048687797399637
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We propose CHiLL (Crafting High-Level Latents), an approach for natural-language specification of features for linear models. CHiLL prompts LLMs with expert-crafted queries to generate interpretable features from health records. The resulting noisy labels are then used to train a simple linear classifier. Generating features based on queries to an LLM can empower physicians to use their domain expertise to craft features that are clinically meaningful for a downstream task of interest, without having to manually extract these from raw EHR. We are motivated by a real-world risk prediction task, but as a reproducible proxy, we use MIMIC-III and MIMIC-CXR data and standard predictive tasks (e.g., 30-day readmission) to evaluate this approach. We find that linear models using automatically extracted features are comparably performant to models using reference features, and provide greater interpretability than linear models using "Bag-of-Words" features. We verify that learned feature weights align well with clinical expectations.

Related papers

• Future Querying: Can LLMs Serve as Implicit Medical World Models? [3.3341114691426257]
  We introduce future querying, a paradigm that probes whether large language models (LLMs) can function as implicit medical world models.<n>Our framework operates on unstructured clinical documentation using endpoint-agnostic training.<n>We show that small, locally fine-tuned open-weight models can match or approach larger proprietary systems.
  arXiv  Detail & Related papers  (2026-08-24T13:41:12Z)
• FeatEHR-LLM: Leveraging Large Language Models for Feature Engineering in Electronic Health Records [12.958792839639605]
  FeatEHR-LLM is a framework that generates meaningful features from irregularly sampled EHR time series.<n>It operates exclusively on dataset schemas and task descriptions rather than raw patient records.<n>It achieves the highest mean AUROC on 7 out of 8 tasks, with improvements of up to 6 percentage points over strong baselines.
  arXiv  Detail & Related papers  (2026-04-24T13:21:01Z)
• Harnessing Large Language Models for Precision Querying and Retrieval-Augmented Knowledge Extraction in Clinical Data Science [3.4325249294405555]
  This study applies Large Language Models (LLMs) to two foundational Electronic Health Record (EHR) data science tasks.<n>We test the ability of LLMs to interact accurately with large structured datasets for analytics.<n>We present a flexible evaluation framework that automatically generates synthetic question and answer pairs tailored to the characteristics of each dataset or task.
  arXiv  Detail & Related papers  (2026-01-28T14:57:36Z)
• Utilizing Large Language Models for Machine Learning Explainability [37.31918138232927]
  This study explores the explainability capabilities of large language models (LLMs), when employed to autonomously generate machine learning (ML) solutions.<n>Three state-of-the-art LLMs are prompted to design training pipelines for four common classifiers: Random Forest, XGBoost, Multilayer Perceptron, and Long Short-Term Memory networks.<n>The generated models are evaluated in terms of predictive performance (recall, precision, and F1-score) and explainability using SHAP (SHapley Additive exPlanations)
  arXiv  Detail & Related papers  (2025-10-08T11:46:23Z)
• Large Language Models as Universal Predictors? An Empirical Study on Small Tabular Datasets [0.0]
  Large Language Models (LLMs) can perform predictive tasks over structured inputs without explicit fine-tuning on downstream tasks.<n>We investigate the empirical function approximation capability of LLMs on small-scale structured datasets for classification, regression and clustering tasks.<n>Our findings suggest that LLMs can serve as general-purpose predictive engines for structured data, with clear strengths in classification and significant limitations in regression and clustering.
  arXiv  Detail & Related papers  (2025-08-24T15:00:51Z)
• Tabular Feature Discovery With Reasoning Type Exploration [5.030210915367596]
  Large language models (LLMs) have been used to automatically generate new features by leveraging their vast knowledge.<n>We propose a novel method REFeat, which guides an LLM to discover diverse and informative features by leveraging multiple types of reasoning.
  arXiv  Detail & Related papers  (2025-06-25T12:18:34Z)
• Efficient Model Selection for Time Series Forecasting via LLMs [52.31535714387368]
  We propose to leverage Large Language Models (LLMs) as a lightweight alternative for model selection. Our method eliminates the need for explicit performance matrices by utilizing the inherent knowledge and reasoning capabilities of LLMs.
  arXiv  Detail & Related papers  (2025-04-02T20:33:27Z)
• Self-Regularization with Latent Space Explanations for Controllable LLM-based Classification [29.74457390987092]
  We propose a novel framework to identify and regularize unintended features in large language models (LLMs) latent spaces. We evaluate the proposed framework on three real-world tasks, including toxic chat detection, reward modeling, and disease diagnosis.
  arXiv  Detail & Related papers  (2025-02-19T22:27:59Z)
• When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications? [8.89829757177796]
  We examine the effectiveness of vector representations from last hidden states of Large Language Models for medical diagnostics and prognostics. We focus on instruction-tuned LLMs in a zero-shot setting to represent abnormal physiological data and evaluate their utilities as feature extractors. Although findings suggest the raw data features still prevails in medical ML tasks, zero-shot LLM embeddings demonstrate competitive results.
  arXiv  Detail & Related papers  (2024-08-15T03:56:40Z)
• ClinicRealm: Re-evaluating Large Language Models with Conventional Machine Learning for Non-Generative Clinical Prediction Tasks [22.539696532725607]
  Large Language Models (LLMs) are increasingly deployed in medicine.<n>However, their utility in non-generative clinical prediction remains under-evaluated.<n>Our ClinicRealm study addresses this by benchmarking 9 GPT-based LLMs, 5 BERT-based models, and 7 traditional methods on unstructured clinical notes and structured Electronic Health Records.
  arXiv  Detail & Related papers  (2024-07-26T06:09:10Z)
• LLM-Select: Feature Selection with Large Language Models [64.5099482021597]
  Large language models (LLMs) are capable of selecting the most predictive features, with performance rivaling the standard tools of data science. Our findings suggest that LLMs may be useful not only for selecting the best features for training but also for deciding which features to collect in the first place.
  arXiv  Detail & Related papers  (2024-07-02T22:23:40Z)
• LLMs can learn self-restraint through iterative self-reflection [57.26854891567574]
  Large Language Models (LLMs) must be capable of dynamically adapting their behavior based on their level of knowledge and uncertainty associated with specific topics. This adaptive behavior, which we refer to as self-restraint, is non-trivial to teach. We devise a utility function that can encourage the model to produce responses only when it is confident in them.
  arXiv  Detail & Related papers  (2024-05-15T13:35:43Z)
• Large Language Models Can Automatically Engineer Features for Few-Shot Tabular Learning [35.03338699349037]
  We propose a novel in-context learning framework, FeatLLM, which employs Large Language Models as feature engineers. FeatLLM generates high-quality rules, significantly (10% on average) outperforming alternatives such as TabLLM and STUNT.
  arXiv  Detail & Related papers  (2024-04-15T06:26:08Z)
• Natural Language Programming in Medicine: Administering Evidence Based Clinical Workflows with Autonomous Agents Powered by Generative Large Language Models [29.05425041393475]
  Generative Large Language Models (LLMs) hold significant promise in healthcare. This study assessed the potential of LLMs to function as autonomous agents in a simulated tertiary care medical center.
  arXiv  Detail & Related papers  (2024-01-05T15:09:57Z)
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning [52.257422715393574]
  We introduce a self-guided methodology for Large Language Models (LLMs) to autonomously discern and select cherry samples from open-source datasets. Our key innovation, the Instruction-Following Difficulty (IFD) metric, emerges as a pivotal metric to identify discrepancies between a model's expected responses and its intrinsic generation capability.
  arXiv  Detail & Related papers  (2023-08-23T09:45:29Z)
• Evaluating and Explaining Large Language Models for Code Using Syntactic Structures [74.93762031957883]
  This paper introduces ASTxplainer, an explainability method specific to Large Language Models for code. At its core, ASTxplainer provides an automated method for aligning token predictions with AST nodes. We perform an empirical evaluation on 12 popular LLMs for code using a curated dataset of the most popular GitHub projects.
  arXiv  Detail & Related papers  (2023-08-07T18:50:57Z)
• Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Models [59.89454513692417]
  Tabular data is often hidden in text, particularly in medical diagnostic reports. We propose a novel, simple, and effective methodology for extracting structured tabular data from textual medical reports, called TEMED-LLM. We demonstrate that our approach significantly outperforms state-of-the-art text classification models in medical diagnostics.
  arXiv  Detail & Related papers  (2023-06-08T09:12:28Z)
• An Iterative Optimizing Framework for Radiology Report Summarization with ChatGPT [80.33783969507458]
  The 'Impression' section of a radiology report is a critical basis for communication between radiologists and other physicians. Recent studies have achieved promising results in automatic impression generation using large-scale medical text data. These models often require substantial amounts of medical text data and have poor generalization performance.
  arXiv  Detail & Related papers  (2023-04-17T17:13:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.