Quantifying the redundancy between prosody and text
- URL: http://arxiv.org/abs/2311.17233v1
- Date: Tue, 28 Nov 2023 21:15:24 GMT
- Title: Quantifying the redundancy between prosody and text
- Authors: Lukas Wolf, Tiago Pimentel, Evelina Fedorenko, Ryan Cotterell, Alex
Warstadt, Ethan Wilcox, Tamar Regev
- Abstract summary: We use large language models to estimate how much information is redundant between prosody and the words themselves.
We find a high degree of redundancy between the information carried by the words and prosodic information across several prosodic features.
Still, we observe that prosodic features can not be fully predicted from text, suggesting that prosody carries information above and beyond the words.
- Score: 67.07817268372743
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Prosody -- the suprasegmental component of speech, including pitch, loudness,
and tempo -- carries critical aspects of meaning. However, the relationship
between the information conveyed by prosody vs. by the words themselves remains
poorly understood. We use large language models (LLMs) to estimate how much
information is redundant between prosody and the words themselves. Using a
large spoken corpus of English audiobooks, we extract prosodic features aligned
to individual words and test how well they can be predicted from LLM
embeddings, compared to non-contextual word embeddings. We find a high degree
of redundancy between the information carried by the words and prosodic
information across several prosodic features, including intensity, duration,
pauses, and pitch contours. Furthermore, a word's prosodic information is
redundant with both the word itself and the context preceding as well as
following it. Still, we observe that prosodic features can not be fully
predicted from text, suggesting that prosody carries information above and
beyond the words. Along with this paper, we release a general-purpose data
processing pipeline for quantifying the relationship between linguistic
information and extra-linguistic features.
Related papers
- Improving Mandarin Prosodic Structure Prediction with Multi-level
Contextual Information [68.89000132126536]
This work proposes to use inter-utterance linguistic information to improve the performance of prosodic structure prediction (PSP)
Our method achieves better F1 scores in predicting prosodic word (PW), prosodic phrase (PPH) and intonational phrase (IPH)
arXiv Detail & Related papers (2023-08-31T09:19:15Z) - Neighboring Words Affect Human Interpretation of Saliency Explanations [65.29015910991261]
Word-level saliency explanations are often used to communicate feature-attribution in text-based models.
Recent studies found that superficial factors such as word length can distort human interpretation of the communicated saliency scores.
We investigate how the marking of a word's neighboring words affect the explainee's perception of the word's importance in the context of a saliency explanation.
arXiv Detail & Related papers (2023-05-04T09:50:25Z) - Evaluation of Automatically Constructed Word Meaning Explanations [0.0]
We present a new tool that derives explanations automatically based on collective information from very large corpora.
We show that the presented approach allows to create explanations that contain data useful for understanding the word meaning in approximately 90% of cases.
arXiv Detail & Related papers (2023-02-27T09:47:55Z) - Representing `how you say' with `what you say': English corpus of
focused speech and text reflecting corresponding implications [10.103202030679844]
In speech communication, how something is said (paralinguistic information) is as crucial as what is said (linguistic information)
Current speech translation systems return the same translations if the utterances are linguistically identical.
We propose mapping paralinguistic information into the linguistic domain within the source language using lexical and grammatical devices.
arXiv Detail & Related papers (2022-03-29T12:29:22Z) - Pareto Probing: Trading Off Accuracy for Complexity [87.09294772742737]
We argue for a probe metric that reflects the fundamental trade-off between probe complexity and performance.
Our experiments with dependency parsing reveal a wide gap in syntactic knowledge between contextual and non-contextual representations.
arXiv Detail & Related papers (2020-10-05T17:27:31Z) - Speakers Fill Lexical Semantic Gaps with Context [65.08205006886591]
We operationalise the lexical ambiguity of a word as the entropy of meanings it can take.
We find significant correlations between our estimate of ambiguity and the number of synonyms a word has in WordNet.
This suggests that, in the presence of ambiguity, speakers compensate by making contexts more informative.
arXiv Detail & Related papers (2020-10-05T17:19:10Z) - Paragraph-level Commonsense Transformers with Recurrent Memory [77.4133779538797]
We train a discourse-aware model that incorporates paragraph-level information to generate coherent commonsense inferences from narratives.
Our results show that PARA-COMET outperforms the sentence-level baselines, particularly in generating inferences that are both coherent and novel.
arXiv Detail & Related papers (2020-10-04T05:24:12Z) - Prosody leaks into the memories of words [2.309770674164469]
The average predictability (aka informativity) of a word in context has been shown to condition word duration.
This study extends past work in two directions; it investigated informativity effects in another large language, Mandarin Chinese.
Results indicated that words with low informativity have shorter durations, replicating the effect found in English.
arXiv Detail & Related papers (2020-05-29T17:58:33Z) - Heaps' law and Heaps functions in tagged texts: Evidences of their
linguistic relevance [0.0]
We study the relationship between vocabulary size and text length in a corpus of $75$ literary works in English.
We analyze the progressive appearance of new words of each tag along each individual text.
arXiv Detail & Related papers (2020-01-07T17:05:16Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.