The Harmonic Structure of Information Contours

• URL: http://arxiv.org/abs/2506.03902v1
• Date: Wed, 04 Jun 2025 12:56:30 GMT
• Title: The Harmonic Structure of Information Contours
• Authors: Eleftheria Tsipidi, Samuel Kiegeland, Franz Nowak, Tianyang Xu, Ethan Wilcox, Alex Warstadt, Ryan Cotterell, Mario Giulianelli,
• Abstract summary: We find consistent evidence of periodic patterns in information rate in texts in English, Spanish, German, Dutch, Basque, and Brazilian Portuguese.<n>Many dominant frequencies align with discourse structure, suggesting these oscillations reflect meaningful linguistic organization.
• Score: 54.38365999922221
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The uniform information density (UID) hypothesis proposes that speakers aim to distribute information evenly throughout a text, balancing production effort and listener comprehension difficulty. However, language typically does not maintain a strictly uniform information rate; instead, it fluctuates around a global average. These fluctuations are often explained by factors such as syntactic constraints, stylistic choices, or audience design. In this work, we explore an alternative perspective: that these fluctuations may be influenced by an implicit linguistic pressure towards periodicity, where the information rate oscillates at regular intervals, potentially across multiple frequencies simultaneously. We apply harmonic regression and introduce a novel extension called time scaling to detect and test for such periodicity in information contours. Analyzing texts in English, Spanish, German, Dutch, Basque, and Brazilian Portuguese, we find consistent evidence of periodic patterns in information rate. Many dominant frequencies align with discourse structure, suggesting these oscillations reflect meaningful linguistic organization. Beyond highlighting the connection between information rate and discourse structure, our approach offers a general framework for uncovering structural pressures at various levels of linguistic granularity.

Related papers

• Towards Explainable Bilingual Multimodal Misinformation Detection and Localization [64.37162720126194]
  BiMi is a framework that jointly performs region-level localization, cross-modal and cross-lingual consistency detection, and natural language explanation for misinformation analysis.<n>BiMiBench is a benchmark constructed by systematically editing real news images and subtitles.<n>BiMi outperforms strong baselines by up to +8.9 in classification accuracy, +15.9 in localization accuracy, and +2.5 in explanation BERTScore.
  arXiv  Detail & Related papers  (2025-06-28T15:43:06Z)
• Using Information Theory to Characterize Prosodic Typology: The Case of Tone, Pitch-Accent and Stress-Accent [22.63155507847401]
  We predict that languages that use prosody to make lexical distinctions should exhibit a higher mutual information between word identity and prosody, compared to languages that don't.<n>We use a dataset of speakers reading sentences aloud in ten languages across five language families to estimate the mutual information between the text and their pitch curves.
  arXiv  Detail & Related papers  (2025-05-12T15:25:17Z)
• Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias [76.85949078144098]
  This paper focuses on textual hallucinations, where diffusion models correctly generate individual symbols but assemble them in a nonsensical manner.<n>We observe that such phenomenon is attributed it to the network's local generation bias.<n>We also theoretically analyze the training dynamics for a specific case involving a two-layer learning parity points on a hypercube.
  arXiv  Detail & Related papers  (2025-03-05T15:28:50Z)
• Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Experts [4.796752450839119]
  We develop a framework to generalize classifiers over time on multiple languages.<n>Our analysis shows classification performance varies over time across different languages.<n>Our study provides analytic insights and addresses the need for time-aware models.
  arXiv  Detail & Related papers  (2025-02-12T22:30:18Z)
• Surprise! Uniform Information Density Isn't the Whole Story: Predicting Surprisal Contours in Long-form Discourse [54.08750245737734]
  We propose that speakers modulate information rate based on location within a hierarchically-structured model of discourse. We find that hierarchical predictors are significant predictors of a discourse's information contour and that deeply nested hierarchical predictors are more predictive than shallow ones.
  arXiv  Detail & Related papers  (2024-10-21T14:42:37Z)
• On the Role of Context in Reading Time Prediction [50.87306355705826]
  We present a new perspective on how readers integrate context during real-time language comprehension. Our proposals build on surprisal theory, which posits that the processing effort of a linguistic unit is an affine function of its in-context information content.
  arXiv  Detail & Related papers  (2024-09-12T15:52:22Z)
• Decoding Multilingual Topic Dynamics and Trend Identification through ARIMA Time Series Analysis on Social Networks: A Novel Data Translation Framework Enhanced by LDA/HDP Models [0.08246494848934444]
  We focus on dialogues within Tunisian social networks during the Coronavirus Pandemic and other notable themes like sports and politics. We start by aggregating a varied multilingual corpus of comments relevant to these subjects. We then introduce our No-English-to-English Machine Translation approach to handle linguistic differences.
  arXiv  Detail & Related papers  (2024-03-18T00:01:10Z)
• Putting Context in Context: the Impact of Discussion Structure on Text Classification [13.15873889847739]
  We propose a series of experiments on a large dataset for stance detection in English. We evaluate the contribution of different types of contextual information. We show that structural information can be highly beneficial to text classification but only under certain circumstances.
  arXiv  Detail & Related papers  (2024-02-05T12:56:22Z)
• How to Handle Different Types of Out-of-Distribution Scenarios in Computational Argumentation? A Comprehensive and Fine-Grained Field Study [59.13867562744973]
  This work systematically assesses LMs' capabilities for out-of-distribution (OOD) scenarios. We find that the efficacy of such learning paradigms varies with the type of OOD. Specifically, while ICL excels for domain shifts, prompt-based fine-tuning surpasses for topic shifts.
  arXiv  Detail & Related papers  (2023-09-15T11:15:47Z)
• Generic Temporal Reasoning with Differential Analysis and Explanation [61.96034987217583]
  We introduce a novel task named TODAY that bridges the gap with temporal differential analysis. TODAY evaluates whether systems can correctly understand the effect of incremental changes. We show that TODAY's supervision style and explanation annotations can be used in joint learning.
  arXiv  Detail & Related papers  (2022-12-20T17:40:03Z)
• Locally Typical Sampling [84.62530743899025]
  We show that today's probabilistic language generators fall short when it comes to producing coherent and fluent text.<n>We propose a simple and efficient procedure for enforcing this criterion when generating from probabilistic models.
  arXiv  Detail & Related papers  (2022-02-01T18:58:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.