Explaining Datasets in Words: Statistical Models with Natural Language Parameters

• URL: http://arxiv.org/abs/2409.08466v1
• Date: Fri, 13 Sep 2024 01:40:20 GMT
• Title: Explaining Datasets in Words: Statistical Models with Natural Language Parameters
• Authors: Ruiqi Zhong, Heng Wang, Dan Klein, Jacob Steinhardt,
• Abstract summary: We introduce a family of statistical models -- including clustering, time series, and classification models -- parameterized by natural language predicates. We apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other.
• Score: 66.69456696878842
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: To make sense of massive data, we often fit simplified models and then interpret the parameters; for example, we cluster the text embeddings and then interpret the mean parameters of each cluster. However, these parameters are often high-dimensional and hard to interpret. To make model parameters directly interpretable, we introduce a family of statistical models -- including clustering, time series, and classification models -- parameterized by natural language predicates. For example, a cluster of text about COVID could be parameterized by the predicate "discusses COVID". To learn these statistical models effectively, we develop a model-agnostic algorithm that optimizes continuous relaxations of predicate parameters with gradient descent and discretizes them by prompting language models (LMs). Finally, we apply our framework to a wide range of problems: taxonomizing user chat dialogues, characterizing how they evolve across time, finding categories where one language model is better than the other, clustering math problems based on subareas, and explaining visual features in memorable images. Our framework is highly versatile, applicable to both textual and visual domains, can be easily steered to focus on specific properties (e.g. subareas), and explains sophisticated concepts that classical methods (e.g. n-gram analysis) struggle to produce.

Related papers

• CAST: Corpus-Aware Self-similarity Enhanced Topic modelling [16.562349140796115]
  We introduce CAST: Corpus-Aware Self-similarity Enhanced Topic modelling, a novel topic modelling method. We find self-similarity to be an effective metric to prevent functional words from acting as candidate topic words. Our approach significantly enhances the coherence and diversity of generated topics, as well as the topic model's ability to handle noisy data.
  arXiv  Detail & Related papers  (2024-10-19T15:27:11Z)
• Detecting Statements in Text: A Domain-Agnostic Few-Shot Solution [1.3654846342364308]
  State-of-the-art approaches usually involve fine-tuning models on large annotated datasets, which are costly to produce. We propose and release a qualitative and versatile few-shot learning methodology as a common paradigm for any claim-based textual classification task. We illustrate this methodology in the context of three tasks: climate change contrarianism detection, topic/stance classification and depression-relates symptoms detection.
  arXiv  Detail & Related papers  (2024-05-09T12:03:38Z)
• Language Models for Text Classification: Is In-Context Learning Enough? [54.869097980761595]
  Recent foundational language models have shown state-of-the-art performance in many NLP tasks in zero- and few-shot settings. An advantage of these models over more standard approaches is the ability to understand instructions written in natural language (prompts) This makes them suitable for addressing text classification problems for domains with limited amounts of annotated instances.
  arXiv  Detail & Related papers  (2024-03-26T12:47:39Z)
• Contextualized Machine Learning [40.415518395978204]
  Contextualized Machine Learning estimates heterogeneous functions by applying deep learning to the meta-relationship between contextual information and context-specific parametric models. We present the open-source PyTorch package ContextualizedML.
  arXiv  Detail & Related papers  (2023-10-17T15:23:00Z)
• Reimagining Retrieval Augmented Language Models for Answering Queries [23.373952699385427]
  We present a reality check on large language models and inspect the promise of retrieval augmented language models in comparison. Such language models are semi-parametric, where models integrate model parameters and knowledge from external data sources to make their predictions.
  arXiv  Detail & Related papers  (2023-06-01T18:08:51Z)
• Conversational Semantic Parsing using Dynamic Context Graphs [68.72121830563906]
  We consider the task of conversational semantic parsing over general purpose knowledge graphs (KGs) with millions of entities, and thousands of relation-types. We focus on models which are capable of interactively mapping user utterances into executable logical forms.
  arXiv  Detail & Related papers  (2023-05-04T16:04:41Z)
• Language Model Cascades [72.18809575261498]
  Repeated interactions at test-time with a single model, or the composition of multiple models together, further expands capabilities. Cases with control flow and dynamic structure require techniques from probabilistic programming. We formalize several existing techniques from this perspective, including scratchpads / chain of thought, verifiers, STaR, selection-inference, and tool use.
  arXiv  Detail & Related papers  (2022-07-21T07:35:18Z)
• A Unified Understanding of Deep NLP Models for Text Classification [88.35418976241057]
  We have developed a visual analysis tool, DeepNLPVis, to enable a unified understanding of NLP models for text classification. The key idea is a mutual information-based measure, which provides quantitative explanations on how each layer of a model maintains the information of input words in a sample. A multi-level visualization, which consists of a corpus-level, a sample-level, and a word-level visualization, supports the analysis from the overall training set to individual samples.
  arXiv  Detail & Related papers  (2022-06-19T08:55:07Z)
• Low-Rank Constraints for Fast Inference in Structured Models [110.38427965904266]
  This work demonstrates a simple approach to reduce the computational and memory complexity of a large class of structured models. Experiments with neural parameterized structured models for language modeling, polyphonic music modeling, unsupervised grammar induction, and video modeling show that our approach matches the accuracy of standard models at large state spaces.
  arXiv  Detail & Related papers  (2022-01-08T00:47:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.