Modeling Probabilistic Reduction using Information Theory and Naive Discriminative Learning

• URL: http://arxiv.org/abs/2506.09641v1
• Date: Wed, 11 Jun 2025 11:54:49 GMT
• Title: Modeling Probabilistic Reduction using Information Theory and Naive Discriminative Learning
• Authors: Anna Stein, Kevin Tang,
• Abstract summary: We examine three models using the Buckeye corpus: one with NDL-derived predictors using information-theoretic formulas, one with traditional NDL predictors, and one with N-gram probabilistic predictors.<n>Results show that the N-gram model outperforms both NDL models, challenging the assumption that NDL is more effective due to its cognitive motivation.
• Score: 0.9668407688201361
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: This study compares probabilistic predictors based on information theory with Naive Discriminative Learning (NDL) predictors in modeling acoustic word duration, focusing on probabilistic reduction. We examine three models using the Buckeye corpus: one with NDL-derived predictors using information-theoretic formulas, one with traditional NDL predictors, and one with N-gram probabilistic predictors. Results show that the N-gram model outperforms both NDL models, challenging the assumption that NDL is more effective due to its cognitive motivation. However, incorporating information-theoretic formulas into NDL improves model performance over the traditional model. This research highlights a) the need to incorporate not only frequency and contextual predictability but also average contextual predictability, and b) the importance of combining information-theoretic metrics of predictability and information derived from discriminative learning in modeling acoustic reduction.

Related papers

• A Comprehensive Taxonomy of Negation for NLP and Neural Retrievers [61.086220009192424]
  We introduce a taxonomy of negation that derives from philosophical, linguistic, and logical definitions.<n>We generate two benchmark datasets that can be used to evaluate the performance of neural information retrieval models.<n>We propose a logic-based classification mechanism that can be used to analyze the performance of retrieval models on existing datasets.
  arXiv  Detail & Related papers  (2025-07-30T02:44:20Z)
• Variational Graph Convolutional Neural Networks [72.67088029389764]
  Uncertainty can help improve the explainability of Graph Convolutional Networks.<n>Uncertainty can also be used in critical applications to verify the results of the model.
  arXiv  Detail & Related papers  (2025-07-02T13:28:37Z)
• An Interpretable Alternative to Neural Representation Learning for Rating Prediction -- Transparent Latent Class Modeling of User Reviews [8.392465185798713]
  We present a transparent probabilistic model that organizes user and product latent classes based on the review information. We evaluate our results in terms of both capacity for interpretability and predictive performances in comparison with popular text-based neural approaches.
  arXiv  Detail & Related papers  (2024-06-17T07:07:42Z)
• When predict can also explain: few-shot prediction to select better neural latents [3.6218162133579703]
  Co-smoothing is used to estimate latent variables and predict observations along held-out channels.<n>In this study, we reveal the limitations of the co-smoothing prediction framework and propose a remedy.<n>We present a novel prediction metric designed to yield latent variables that more accurately reflect the ground truth.
  arXiv  Detail & Related papers  (2024-05-23T10:48:30Z)
• CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding [62.075029712357]
  This work introduces the Cognitive Diffusion Probabilistic Models (CogDPM) CogDPM features a precision estimation method based on the hierarchical sampling capabilities of diffusion models and weight the guidance with precision weights estimated by the inherent property of diffusion models. We apply CogDPM to real-world prediction tasks using the United Kindom precipitation and surface wind datasets.
  arXiv  Detail & Related papers  (2024-05-03T15:54:50Z)
• A PAC-Bayesian Perspective on the Interpolating Information Criterion [54.548058449535155]
  We show how a PAC-Bayes bound is obtained for a general class of models, characterizing factors which influence performance in the interpolating regime. We quantify how the test error for overparameterized models achieving effectively zero training error depends on the quality of the implicit regularization imposed by e.g. the combination of model, parameter-initialization scheme.
  arXiv  Detail & Related papers  (2023-11-13T01:48:08Z)
• Neural Additive Models for Location Scale and Shape: A Framework for Interpretable Neural Regression Beyond the Mean [1.0923877073891446]
  Deep neural networks (DNNs) have proven to be highly effective in a variety of tasks. Despite this success, the inner workings of DNNs are often not transparent. This lack of interpretability has led to increased research on inherently interpretable neural networks.
  arXiv  Detail & Related papers  (2023-01-27T17:06:13Z)
• Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions [59.284907093349425]
  Large amounts of training data are one of the major reasons for the high performance of state-of-the-art NLP models. We provide a language for describing how training data influences predictions, through a causal framework. Our framework bypasses the need to retrain expensive models and allows us to estimate causal effects based on observational data alone.
  arXiv  Detail & Related papers  (2022-07-28T17:36:24Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Double Robust Representation Learning for Counterfactual Prediction [68.78210173955001]
  We propose a novel scalable method to learn double-robust representations for counterfactual predictions. We make robust and efficient counterfactual predictions for both individual and average treatment effects. The algorithm shows competitive performance with the state-of-the-art on real world and synthetic data.
  arXiv  Detail & Related papers  (2020-10-15T16:39:26Z)
• Estimating Model Uncertainty of Neural Networks in Sparse Information Form [39.553268191681376]
  We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) The key insight of our work is that the information matrix tends to be sparse in its spectrum. We show that the information form can be scalably applied to represent model uncertainty in DNNs.
  arXiv  Detail & Related papers  (2020-06-20T18:09:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.