Combining Prediction and Interpretation in Decision Trees (PrInDT) -- a Linguistic Example

• URL: http://arxiv.org/abs/2103.02336v2
• Date: Fri, 5 Mar 2021 17:37:51 GMT
• Title: Combining Prediction and Interpretation in Decision Trees (PrInDT) -- a Linguistic Example
• Authors: Claus Weihs and Sarah Buschfeld
• Abstract summary: We show that conditional inference trees and ensembles are suitable methods for modeling linguistic variation. As against earlier linguistic applications, however, we claim that their suitability is strongly increased if we combine prediction and interpretation.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this paper, we show that conditional inference trees and ensembles are suitable methods for modeling linguistic variation. As against earlier linguistic applications, however, we claim that their suitability is strongly increased if we combine prediction and interpretation. To that end, we have developed a statistical method, PrInDT (Prediction and Interpretation with Decision Trees), which we introduce and discuss in the present paper.

Related papers

• Variational Reasoning for Language Models [93.08197299751197]
  We introduce a variational reasoning framework for language models that treats thinking traces as latent variables.<n>We show that rejection sampling finetuning and binary-reward RL, including GRPO, can be interpreted as local forward-KL objectives.
  arXiv  Detail & Related papers  (2025-09-26T17:58:10Z)
• Refining Syntactic Distinctions Using Decision Trees: A Paper on Postnominal 'That' in Complement vs. Relative Clauses [0.0]
  We first tested the performance of the TreeTagger English model developed by Helmut Schmid with test files at our disposal.<n>We distinguished between the two uses of "that," both as a relative pronoun and as a complementizer.<n>We proposed an improved model by retraining TreeTagger and compared the newly trained model with Schmid's baseline model.
  arXiv  Detail & Related papers  (2025-09-13T15:41:13Z)
• Leveraging Predictive Equivalence in Decision Trees [15.961209879141066]
  Decision trees are widely used for interpretable machine learning.<n>We present a representation of decision trees that does not exhibit predictive equivalence.<n>We show that decision trees are surprisingly robust to test-time missingness of feature values.
  arXiv  Detail & Related papers  (2025-06-17T03:11:30Z)
• Rethinking the Relationship between the Power Law and Hierarchical Structures [3.8063235092267993]
  This study examines the validity of the argument for syntactic structures using English corpora.<n>Using English corpora, we analyze the mutual information, deviations from probabilistic context-free grammars, and other properties in parse trees.<n>Our results indicate that the assumptions do not hold for syntactic structures and that it is difficult to apply the proposed argument to child languages and animal signals.
  arXiv  Detail & Related papers  (2025-05-08T06:41:46Z)
• Extending Explainable Ensemble Trees (E2Tree) to regression contexts [1.5186937600119894]
  E2Tree is a novel methodology for explaining random forests. It accounts for the effects of predictor variables on the response. It also accounts for associations between the predictor variables through the computation and use of dissimilarity measures.
  arXiv  Detail & Related papers  (2024-09-10T11:42:55Z)
• Regularized Conventions: Equilibrium Computation as a Model of Pragmatic Reasoning [72.21876989058858]
  We present a model of pragmatic language understanding, where utterances are produced and understood by searching for regularized equilibria of signaling games. In this model speakers and listeners search for contextually appropriate utterance--meaning mappings that are both close to game-theoretically optimal conventions and close to a shared, ''default'' semantics.
  arXiv  Detail & Related papers  (2023-11-16T09:42:36Z)
• Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models [2.5863812709449543]
  We propose a novel line to constructing metrics for understanding the mechanisms of pretrained language models. Based on the experimental results, we propose a speculation regarding the working mechanism of BERT-like pretrained language models.
  arXiv  Detail & Related papers  (2023-10-19T04:16:40Z)
• Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictions [9.909170013118775]
  This work presents a linear decomposition of final hidden states from autoregressive language models based on each initial input token. Using the change in next-word probability as a measure of importance, this work first examines which context words make the biggest contribution to language model predictions.
  arXiv  Detail & Related papers  (2023-05-17T23:55:32Z)
• Towards Linguistically Informed Multi-Objective Pre-Training for Natural Language Inference [0.38233569758620045]
  We introduce a linguistically enhanced combination of pre-training methods for transformers. The pre-training objectives include POS-tagging, synset prediction based on semantic knowledge graphs, and parent prediction based on dependency parse trees. Our approach achieves competitive results on the Natural Language Inference task, compared to the state of the art.
  arXiv  Detail & Related papers  (2022-12-14T10:50:13Z)
• A Latent-Variable Model for Intrinsic Probing [93.62808331764072]
  We propose a novel latent-variable formulation for constructing intrinsic probes. We find empirical evidence that pre-trained representations develop a cross-lingually entangled notion of morphosyntax.
  arXiv  Detail & Related papers  (2022-01-20T15:01:12Z)
• Instance-Based Neural Dependency Parsing [56.63500180843504]
  We develop neural models that possess an interpretable inference process for dependency parsing. Our models adopt instance-based inference, where dependency edges are extracted and labeled by comparing them to edges in a training set.
  arXiv  Detail & Related papers  (2021-09-28T05:30:52Z)
• Probabilistic modelling of rational communication with conditionals [0.0]
  We take a probabilistic approach to pragmatic reasoning about conditionals. We show that our model uniformly explains a number of inferences attested in the literature.
  arXiv  Detail & Related papers  (2021-05-12T08:21:25Z)
• On the Branching Bias of Syntax Extracted from Pre-trained Language Models [47.82102426290707]
  We propose quantitatively measuring the branching bias by comparing the performance gap on a language and its reversed language. We analyze the impacts of three factors on the branching bias, namely parsing algorithms, feature definitions, and language models.
  arXiv  Detail & Related papers  (2020-10-06T03:09:14Z)
• Constructing a Family Tree of Ten Indo-European Languages with Delexicalized Cross-linguistic Transfer Patterns [57.86480614673034]
  We formalize the delexicalized transfer as interpretable tree-to-string and tree-to-tree patterns. This allows us to quantitatively probe cross-linguistic transfer and extend inquiries of Second Language Acquisition.
  arXiv  Detail & Related papers  (2020-07-17T15:56:54Z)
• Exploiting Syntactic Structure for Better Language Modeling: A Syntactic Distance Approach [78.77265671634454]
  We make use of a multi-task objective, i.e., the models simultaneously predict words as well as ground truth parse trees in a form called "syntactic distances" Experimental results on the Penn Treebank and Chinese Treebank datasets show that when ground truth parse trees are provided as additional training signals, the model is able to achieve lower perplexity and induce trees with better quality.
  arXiv  Detail & Related papers  (2020-05-12T15:35:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.