LLMs Understand Glass-Box Models, Discover Surprises, and Suggest Repairs

• URL: http://arxiv.org/abs/2308.01157v2
• Date: Mon, 7 Aug 2023 17:06:56 GMT
• Title: LLMs Understand Glass-Box Models, Discover Surprises, and Suggest Repairs
• Authors: Benjamin J. Lengerich, Sebastian Bordt, Harsha Nori, Mark E. Nunnally, Yin Aphinyanaphongs, Manolis Kellis, Rich Caruana
• Abstract summary: We show that large language models (LLMs) are remarkably good at working with interpretable models. By adopting a hierarchical approach to reasoning, LLMs can provide comprehensive model-level summaries. We present the package $textttTalkToEBM$ as an open-source LLM-GAM interface.
• Score: 10.222281712562705
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We show that large language models (LLMs) are remarkably good at working with interpretable models that decompose complex outcomes into univariate graph-represented components. By adopting a hierarchical approach to reasoning, LLMs can provide comprehensive model-level summaries without ever requiring the entire model to fit in context. This approach enables LLMs to apply their extensive background knowledge to automate common tasks in data science such as detecting anomalies that contradict prior knowledge, describing potential reasons for the anomalies, and suggesting repairs that would remove the anomalies. We use multiple examples in healthcare to demonstrate the utility of these new capabilities of LLMs, with particular emphasis on Generalized Additive Models (GAMs). Finally, we present the package $\texttt{TalkToEBM}$ as an open-source LLM-GAM interface.

Related papers

• Can a Large Language Model Learn Matrix Functions In Context? [3.7478782183628634]
  Large Language Models (LLMs) have demonstrated the ability to solve complex tasks through In-Context Learning (ICL) This paper explores the capacity of LLMs to solve non-linear numerical computations, with specific emphasis on functions of the Singular Value Decomposition.
  arXiv  Detail & Related papers  (2024-11-24T00:33:43Z)
• Guide-to-Explain for Controllable Summarization [11.904090197598505]
  controllable summarization with large language models (LLMs) remains underexplored. We propose a guide-to-explain framework (GTE) for controllable summarization. Our framework enables the model to identify misaligned attributes in the initial draft and guides it in explaining errors in the previous output.
  arXiv  Detail & Related papers  (2024-11-19T12:36:02Z)
• Unveiling the Generalization Power of Fine-Tuned Large Language Models [81.70754292058258]
  We investigate whether fine-tuning affects the intrinsic generalization ability intrinsic to Large Language Models (LLMs) Our main findings reveal that models fine-tuned on generation and classification tasks exhibit dissimilar behaviors in generalizing to different domains and tasks. We observe that integrating the in-context learning strategy during fine-tuning on generation tasks can enhance the model's generalization ability.
  arXiv  Detail & Related papers  (2024-03-14T08:18:59Z)
• Data Science with LLMs and Interpretable Models [19.4969442162327]
  Large language models (LLMs) are remarkably good at working with interpretable models. We show that LLMs can describe, interpret, and debug Generalized Additive Models (GAMs)
  arXiv  Detail & Related papers  (2024-02-22T12:04:15Z)
• On the Out-Of-Distribution Generalization of Multimodal Large Language Models [24.431960338495184]
  We investigate the generalization boundaries of current Multimodal Large Language Models (MLLMs) We evaluate their zero-shot generalization across synthetic images, real-world distributional shifts, and specialized datasets like medical and molecular imagery. We show that in-context learning can significantly enhance MLLMs' generalization, opening new avenues for overcoming generalization barriers.
  arXiv  Detail & Related papers  (2024-02-09T18:21:51Z)
• Rethinking Interpretability in the Era of Large Language Models [76.1947554386879]
  Large language models (LLMs) have demonstrated remarkable capabilities across a wide array of tasks. The capability to explain in natural language allows LLMs to expand the scale and complexity of patterns that can be given to a human. These new capabilities raise new challenges, such as hallucinated explanations and immense computational costs.
  arXiv  Detail & Related papers  (2024-01-30T17:38:54Z)
• Knowledge Fusion of Large Language Models [73.28202188100646]
  This paper introduces the notion of knowledge fusion for large language models (LLMs) We externalize their collective knowledge and unique strengths, thereby elevating the capabilities of the target model beyond those of any individual source LLM. Our findings confirm that the fusion of LLMs can improve the performance of the target model across a range of capabilities such as reasoning, commonsense, and code generation.
  arXiv  Detail & Related papers  (2024-01-19T05:02:46Z)
• Supervised Knowledge Makes Large Language Models Better In-context Learners [94.89301696512776]
  Large Language Models (LLMs) exhibit emerging in-context learning abilities through prompt engineering. The challenge of improving the generalizability and factuality of LLMs in natural language understanding and question answering remains under-explored. We propose a framework that enhances the reliability of LLMs as it: 1) generalizes out-of-distribution data, 2) elucidates how LLMs benefit from discriminative models, and 3) minimizes hallucinations in generative tasks.
  arXiv  Detail & Related papers  (2023-12-26T07:24:46Z)
• Local Large Language Models for Complex Structured Medical Tasks [0.0]
  This paper introduces an approach that combines the language reasoning capabilities of large language models with the benefits of local training to tackle complex, domain-specific tasks. Specifically, the authors demonstrate their approach by extracting structured condition codes from pathology reports.
  arXiv  Detail & Related papers  (2023-08-03T12:36:13Z)
• Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback [127.75419038610455]
  Large language models (LLMs) are able to generate human-like, fluent responses for many downstream tasks. This paper proposes a LLM-Augmenter system, which augments a black-box LLM with a set of plug-and-play modules.
  arXiv  Detail & Related papers  (2023-02-24T18:48:43Z)
• Augmenting Interpretable Models with LLMs during Training [73.40079895413861]
  We propose Augmented Interpretable Models (Aug-imodels) to build efficient and interpretable models. Aug-imodels use LLMs during fitting but not during inference, allowing complete transparency. We explore two instantiations of Aug-imodels in natural-language processing: (i) Aug-GAM, which augments a generalized additive model with decoupled embeddings from an LLM and (ii) Aug-Tree, which augments a decision tree with LLM feature expansions.
  arXiv  Detail & Related papers  (2022-09-23T18:36:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.